在 SageMaker 中调用 fit 方法时如何防止 NoCredentialsError？

Question

我是 Python SageMaker 的新手（我的背景是 C#）。目前，我有一个问题，因为最后一个方法调用（我的意思是 fit 方法）导致 "NoCredentialsError"。我不明白这个。 AWS 凭证已设置，我确实使用它们与 AWS 通信，例如与 S3 通信。我怎样才能防止这个错误？

import io
import os
import gzip
import pickle
import urllib.request
import boto3
import sagemaker
import sagemaker.amazon.common as smac

DOWNLOADED_FILENAME = 'C:/Users/Daan/PycharmProjects/downloads/mnist.pkl.gz'
if not os.path.exists(DOWNLOADED_FILENAME):
    urllib.request.urlretrieve("http://deeplearning.net/data/mnist/mnist.pkl.gz", DOWNLOADED_FILENAME)

with gzip.open(DOWNLOADED_FILENAME, 'rb') as f:
    train_set, valid_set, test_set = pickle.load(f, encoding='latin1')
vectors = train_set[0].T
buf = io.BytesIO()
smac.write_numpy_to_dense_tensor(buf, vectors)
buf.seek(0)
key = 'recordio-pb-data'
bucket_name = 'SOMEKINDOFBUCKETNAME'
prefix = 'sagemaker/pca'
path = os.path.join(prefix, 'train', key)
print(path)

session = boto3.session.Session(aws_access_key_id='SECRET',aws_secret_access_key='SECRET',region_name='eu-west-1')
client = boto3.client('sagemaker',region_name='eu-west-1',aws_access_key_id='SECRET',aws_secret_access_key='SECRET')
region='eu-west-1'
sagemakerSession= sagemaker.Session(sagemaker_client=client,boto_session=session)
s3_resource=session.resource('s3')
bucket = s3_resource.Bucket(bucket_name)
current_bucket = bucket.Object(path)

train_data = 's3://{}/{}/train/{}'.format(bucket_name, prefix, key)
print('uploading training data location: {}'.format(train_data))
current_bucket.upload_fileobj(buf)

output_location = 's3://{}/{}/output'.format('SOMEBUCKETNAME', prefix)
print('training artifacts will be uploaded to: {}'.format(output_location))

region='eu-west-1'

containers = {'us-west-2': 'SOMELOCATION',
              'us-east-1': 'SOMELOCATION',
              'us-east-2': 'SOMELOCATION',
              'eu-west-1': 'SOMELOCATION'}
container = containers[region]

role='AmazonSageMaker-ExecutionRole-SOMEVALUE'
pca = sagemaker.estimator.Estimator(container,
                                    role,
                                    train_instance_count=1,
                                    train_instance_type='ml.c4.xlarge',
                                    output_path=output_location,
                                    sagemaker_session=sagemakerSession)


pca.set_hyperparameters(feature_dim=50000,
                        num_components=10,
                        subtract_mean=True,
                        algorithm_mode='randomized',
                        mini_batch_size=200)

pca.fit(inputs=train_data)

print('END')

Answer 1

我不确定您是否屏蔽了实际的访问 ID 和密钥，或者这就是您的身份运行。

session = boto3.session.Session(aws_access_key_id='SECRET',aws_secret_access_key='SECRET',region_name='eu-west-1')
client = boto3.client('sagemaker',region_name='eu-west-1',aws_access_key_id='SECRET',aws_secret_access_key='SECRET')

我希望您在上面的代码行中提供实际的 aws_access_key_id 和 aws_secret_access_key。

在代码中指定相同而不是硬编码的另一种方法是在您的配置文件目录中创建一个凭据文件，即

在 Mac ~/.aws/

并在 Windows "%UserProfile%\.aws"

该文件是一个纯文本文件，名称为“credentials”（不带引号）。文件包含

[default]
aws_access_key_id=XXXXXXXXXXXXXX
aws_secret_access_key=YYYYYYYYYYYYYYYYYYYYYYYYYYY

AWS CLI 将从上述位置获取并使用它。您还可以使用非默认配置文件并使用

传递配置文件

os.environ["AWS_PROFILE"] = "profile-name"

希望对您有所帮助。

在 SageMaker 中调用 fit 方法时如何防止 NoCredentialsError？

How to prevent a NoCredentialsError when calling the fit method in SageMaker?

python

amazon-sagemaker