从超参数调整作业创建用于管道的模型

Question

我正在尝试将超参数调整作业中的最佳估计器实施到管道对象中以部署端点。

我已尽最大努力阅读文档以将调优作业的结果包含在管道中，但我在创建 Model() class 对象时遇到了问题。

# This is the hyperparameter tuning job
tuner.fit({'train': s3_train, 'validation': s3_val}, 
include_cls_metadata=False)


#With a standard Model (Not from the tuner) the process was as follows:
scikit_learn_inferencee_model_name = sklearn_preprocessor.create_model()
xgb_model_name = Model(model_data=xgb_model.model_data, image=xgb_image)


model_name = 'xgb-inference-pipeline-' + timestamp_prefix
endpoint_name = 'xgb-inference-pipeline-ep-' + timestamp_prefix
sm_model = PipelineModel(
    name=model_name, 
    role=role, 
    models=[
        scikit_learn_inferencee_model_name, 
        xgb_model_name])

sm_model.deploy(initial_instance_count=1, instance_type='ml.c4.xlarge', 
endpoint_name=endpoint_name)

我希望能够使用调优作业的结果干净地实例化模型对象并将其传递到 PipelineModel 对象中。任何指导表示赞赏。

Answer 1

我认为您的方向是正确的。你有什么错误吗？参考此 notebook 从调谐器实例化模型并在推理管道中使用。

正在根据评论编辑之前的回复。要从超参数调整作业的最佳训练作业创建模型，您可以使用以下代码片段

from sagemaker.tuner import HyperparameterTuner
from sagemaker.estimator import Estimator
from sagemaker.model import Model

# Attach to an existing hyperparameter tuning job.
xgb_tuning_job_name = 'my_xgb_hpo_tuning_job_name'
xgb_tuner = HyperparameterTuner.attach(xgb_tuning_job_name)

# Get the best XGBoost training job name from the HPO job
xgb_best_training_job = xgb_tuner.best_training_job()
print(xgb_best_training_job)

# Attach estimator to the best training job name
xgb_best_estimator = Estimator.attach(xgb_best_training_job)

# Create model to be passed to the inference pipeline
xgb_model = Model(model_data=xgb_best_estimator.model_data,
                  role=sagemaker.get_execution_role(),
                  image=xgb_best_estimator.image_name)

从超参数调整作业创建用于管道的模型

Creating a model for use in a pipeline from a hyperparameter tuning job

python

pipeline

amazon-web-services

amazon-sagemaker