botocore s3 put 由于编码而出现哈希文件问题？

Question

我无法弄清楚为什么加载了编码 utf-8 的文件（其内容为 "DELETE ME LATER"）在散列时在 botocore 中导致异常。

with io.open('deleteme','r', encoding='utf-8') as f:
  try:
    resp=client.put_object(
    Body=f,
    Bucket='s3-bucket-actual-name-for-real',
    Key='testing/a/put'
    )
    print('deleteme exists')
    print(resp)
  except:
    print('deleteme could not put')
    raise

生产：

deleteme could not put
Traceback (most recent call last): File
"./test_operator.py", line 41, in
Key='testing/a/put' File "/Users/lamblin/VEnvs/awscli/lib/python3.6/site-packages/botocore/client.py",
line 312, in _api_call
return self._make_api_call(operation_name, kwargs) File "/Users/lamblin/VEnvs/awscli/lib/python3.6/site-packages/botocore/client.py",
line 582, in _make_api_call request_signer=self._request_signer, context=request_context) File
"/Users/lamblin/VEnvs/awscli/lib/python3.6/site-packages/botocore/hooks.py",
line 242, in emit_until_response
responses = self._emit(event_name, kwargs, stop_on_response=True) File
"/Users/lamblin/VEnvs/awscli/lib/python3.6/site-packages/botocore/hooks.py",
line 210, in _emit
response = handler(**kwargs) File "/Users/lamblin/VEnvs/awscli/lib/python3.6/site-packages/botocore/handlers.py",
line 201, in conditionally_calculate_md5
calculate_md5(params, **kwargs) File "/Users/lamblin/VEnvs/awscli/lib/python3.6/site-packages/botocore/handlers.py",
line 179, in calculate_md5
binary_md5 = _calculate_md5_from_file(body) File "/Users/lamblin/VEnvs/awscli/lib/python3.6/site-packages/botocore/handlers.py",
line 193, in _calculate_md5_from_file md5.update(chunk)
TypeError: Unicode-objects must be encoded before hashing

现在可以通过使用 'rb' 打开文件来避免这种情况，但是文件对象 f 显然使用了编码吗？

Answer 1

Now this can be avoided by opening the file with 'rb' but, isn't the file object f clearly using an encoding?

在mode='r'中为io.open指定的编码用于解码内容。所以当你迭代f时，内容已经被Python从bytes转换为str（文本）。

要直接与 botocore 交互，请使用 'rb' 模式打开文件，并删除编码 kwarg。当 botocore 必须做的第一件事是传输内容时，将其解码为文本是没有意义的，只是再次编码回字节。

botocore s3 put 由于编码而出现哈希文件问题？

botocore s3 put has issue hashing file due to encoding?

python

amazon-s3

botocore