使用 sagemaker 将数据上传到 S3

Question

当我尝试将数据上传到 S3 存储桶时，SageMaker 出现问题。我收到此错误：

NameError                                 Traceback (most recent call last)
<ipython-input-26-d21b1cb0fcab> in <module>()
     19 download('http://data.mxnet.io/data/caltech-256/caltech-256-60-train.rec')
     20 
---> 21 upload_to_s3('train', 'caltech-256-60-train.rec')

<ipython-input-26-d21b1cb0fcab> in upload_to_s3(channel, file)
     13     data = open(file, "rb")
     14     key = channel + '/' + file
---> 15     s3.Bucket(bucket).put_object(Key=key, Body=data)
     16 
     17 

NameError: name 'bucket' is not defined

这是脚本：

import os
import urllib.request
import boto3

def download(url):
    filename = url.split("/")[-1]
    if not os.path.exists(filename):
        urllib.request.urlretrieve(url, filename)


def upload_to_s3(channel, file):
    s3 = boto3.resource('s3')
    data = open(file, "rb")
    key = channel + '/' + file
    s3.Bucket(bucket).put_object(Key=key, Body=data)


# caltech-256 download('http://data.mxnet.io/data/caltech-256/caltech-256-60-train.rec')

upload_to_s3('train', 'caltech-256-60-train.rec')

Answer 1

正如错误所说，变量bucket未定义。你可能想做类似

的事情

bucket = <name of already created bucket in s3>

在你打电话之前

s3.Bucket(bucket).put_object(Key=key, Body=data)

使用 sagemaker 将数据上传到 S3

upload data to S3 with sagemaker

amazon-s3

amazon-sagemaker