一个 运行 sagemaker notebook 可以在 visual studio 代码中本地编码吗
can one run sagemaker notebook code locally in visual studio code
下面的代码在云中的 sagemaker notebook 中运行良好。在本地,我也有通过 aws cli 创建的 aws 凭据。就个人而言,我不喜欢笔记本(除非我做一些 EDA 之类的)。所以我想知道是否也可以从本地机器(例如 visual studio 代码)启动这段代码,因为它只告诉 sagemaker 无论如何要做什么?我想这只是验证和获取会话对象的问题?谢谢!
import boto3
import os
import sagemaker
from sagemaker import get_execution_role
from sagemaker.inputs import TrainingInput
from sagemaker.serializers import CSVSerializer
from sagemaker import image_uris
region_name = boto3.session.Session().region_name
s3_bucket_name = 'bucket_name'
# this image cannot be used below !!! there must be an issue with sagemaker ?
training_image_name = image_uris.retrieve(framework='xgboost', region=region_name, version='latest')
role = get_execution_role()
s3_prefix = 'my_model'
train_file_name = 'sagemaker_train.csv'
val_file_name = 'sagemaker_val.csv'
sagemaker_session = sagemaker.Session()
s3_input_train = TrainingInput(s3_data='s3://{}/{}/{}'.format(s3_bucket_name, s3_prefix, train_file_name), content_type='csv')
s3_input_val = TrainingInput(s3_data='s3://{}/{}/{}'.format(s3_bucket_name, s3_prefix, val_file_name), content_type='csv')
hyperparameters = {
"max_depth":"5",
"eta":"0.2",
"gamma":"4",
"min_child_weight":"6",
"subsample":"0.7",
"objective":"reg:squarederror",
"num_round":"10"}
output_path = 's3://{}/{}/output'.format(s3_bucket_name, s3_prefix)
estimator = sagemaker.estimator.Estimator(image_uri=sagemaker.image_uris.retrieve("xgboost", region_name, "1.2-2"),
hyperparameters=hyperparameters,
role=role,
instance_count=1,
instance_type='ml.m5.2xlarge',
volume_size=1, # 1 GB
output_path=output_path)
estimator.fit({'train': s3_input_train, 'validation': s3_input_val})
在本地机器上,
- 确保安装 AWS CLI:https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
- 创建访问密钥 ID 和秘密访问密钥以在本地访问 Sagemaker 服务:https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html
- 在本地使用
aws configure
命令设置这些凭据。
- 除了获得执行角色外,代码应该可以正常工作。您可以在代码中对 Sagemaker 角色进行硬编码(不是最佳实践),也可以将其存储在参数存储中并从那里访问它。
下面的代码在云中的 sagemaker notebook 中运行良好。在本地,我也有通过 aws cli 创建的 aws 凭据。就个人而言,我不喜欢笔记本(除非我做一些 EDA 之类的)。所以我想知道是否也可以从本地机器(例如 visual studio 代码)启动这段代码,因为它只告诉 sagemaker 无论如何要做什么?我想这只是验证和获取会话对象的问题?谢谢!
import boto3
import os
import sagemaker
from sagemaker import get_execution_role
from sagemaker.inputs import TrainingInput
from sagemaker.serializers import CSVSerializer
from sagemaker import image_uris
region_name = boto3.session.Session().region_name
s3_bucket_name = 'bucket_name'
# this image cannot be used below !!! there must be an issue with sagemaker ?
training_image_name = image_uris.retrieve(framework='xgboost', region=region_name, version='latest')
role = get_execution_role()
s3_prefix = 'my_model'
train_file_name = 'sagemaker_train.csv'
val_file_name = 'sagemaker_val.csv'
sagemaker_session = sagemaker.Session()
s3_input_train = TrainingInput(s3_data='s3://{}/{}/{}'.format(s3_bucket_name, s3_prefix, train_file_name), content_type='csv')
s3_input_val = TrainingInput(s3_data='s3://{}/{}/{}'.format(s3_bucket_name, s3_prefix, val_file_name), content_type='csv')
hyperparameters = {
"max_depth":"5",
"eta":"0.2",
"gamma":"4",
"min_child_weight":"6",
"subsample":"0.7",
"objective":"reg:squarederror",
"num_round":"10"}
output_path = 's3://{}/{}/output'.format(s3_bucket_name, s3_prefix)
estimator = sagemaker.estimator.Estimator(image_uri=sagemaker.image_uris.retrieve("xgboost", region_name, "1.2-2"),
hyperparameters=hyperparameters,
role=role,
instance_count=1,
instance_type='ml.m5.2xlarge',
volume_size=1, # 1 GB
output_path=output_path)
estimator.fit({'train': s3_input_train, 'validation': s3_input_val})
在本地机器上,
- 确保安装 AWS CLI:https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
- 创建访问密钥 ID 和秘密访问密钥以在本地访问 Sagemaker 服务:https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html
- 在本地使用
aws configure
命令设置这些凭据。 - 除了获得执行角色外,代码应该可以正常工作。您可以在代码中对 Sagemaker 角色进行硬编码(不是最佳实践),也可以将其存储在参数存储中并从那里访问它。