使用 Python urllib2 将 zip 文件直接上传到 AWS S3

Question

我正在尝试使用 Python 脚本将 zip 文件直接上传到 S3，但是运行出现了一些 Unicode 解码错误。

我所做的是生成一个预签名的 S3 Link，然后将数据上传到它。我知道 link 工作正常，因为当我使用 curl 这样做时上传工作正常：

curl -v -H "Content-Type: application/zip" -T /Path/To/Local/File.zip https://MySignedAWSS3Link

但是，当我使用下面的代码在 Python 中尝试此操作时，出现错误。

infile2 = open('/Path/To/Local/File.zip', 'rb')
filedata2 = infile2.read()
request2 = urllib2.Request("https://MySignedAWSS3Link",data=filedata2)
request2.add_header('Content-Type', 'application/zip')
request2.get_method = lambda: 'PUT'
url2 = opener.open(request2)

我在 Python 中得到以下 error/traceback：

> Traceback (most recent call last):
  File "putFiles.py", line 44, in <module>
    url2 = opener.open(request2)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 404, in open
    response = self._open(req, data)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 422, in _open
    '_open', req)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 382, in _call_chain
    result = func(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1222, in https_open
    return self.do_open(httplib.HTTPSConnection, req)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1181, in do_open
    h.request(req.get_method(), req.get_selector(), req.data, headers)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 973, in request
    self._send_request(method, url, body, headers)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1007, in _send_request
    self.endheaders(body)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 969, in endheaders
    self._send_output(message_body)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 827, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0xca in position 10: ordinal not in range(128)

我做错了什么？

Answer 1

根据您的回溯，添加到您的请求中的一些字符串是 unicode。但是，根据您的脚本示例，所有字符串都是 ascii 编码的（因为您使用的是 Python2.7).

除非您将 Python 默认编码设置为 UTF-8（或 OS 为您设置），否则您会遇到奇怪的事情。

我希望这对你有用（将所有字符串转换为 ascii）：

infile2 = open('/Path/To/Local/File.zip', 'rb')
filedata2 = infile2.read()
request2 = urllib2.Request("https://MySignedAWSS3Link".encode('utf-8'),data=filedata2)
request2.add_header(str('Content-Type'), str('application/zip'))
request2.get_method = lambda: str('PUT')
url2 = opener.open(request2)

更新: This q/a 也可能对你有帮助

使用 Python urllib2 将 zip 文件直接上传到 AWS S3

Uploading a zip file directly to AWS S3 using Python urllib2

python

amazon-s3

zip

urllib2

python-unicode