如何将 boto 的 s3 bucket.copy_key 与 KMS SSE 一起使用？
How to use boto's s3 bucket.copy_key with KMS SSE?
我正在使用 boto 版本 2.38.0 尝试将 s3 上的密钥复制 region-to-region 到不同区域中不同存储桶中的相同密钥。源密钥使用 KMS 密钥加密，目标也应使用 KMS 密钥加密，尽管是不同的密钥。
我可以使用以下代码正确复制到未加密的目标密钥：
import boto
from boto import connect_s3
if not boto.config.get('s3', 'use-sigv4'):
boto.config.add_section('s3')
boto.config.set('s3', 'use-sigv4', 'True')
src_bucket_name = 'my-bucket'
dest_bucket_name = 'my-bucket-2'
conn_std = connect_s3(host='s3.amazonaws.com')
conn_norcal = connect_s3(host='s3-us-west-1.amazonaws.com')
src = conn_std.get_bucket(src_bucket_name)
dest = conn_norcal.get_bucket(dest_bucket_name)
for src_item in src_items:
dest.copy_key(src_item.key, src.name, src_item.key)
但是，如果我尝试使用此处指定的 headers (http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectCOPY.html) 通过 KMS 密钥启用 SSE，我会收到 SignatureDoesNotMatch 错误。
因此，例如，如果我将 copy_key 行更改为：
dest.copy_key(src_item.key, src.name, src_item.key, headers={
'x-amz-server-side-encryption':'aws:kms',
'x-amz-server-side-encryption-aws-kms-key-id': 'e2009179-a4fc-4a0a-99c6-490b7bb1ebbb'
})
然后我得到如下错误：
## Note: I replaced all sensitive details with [...],
## and used my-bucket and my-bucket-2 with testing.txt.
Traceback (most recent call last):
[...]
File "/Library/Python/2.7/site-packages/boto/s3/bucket.py", line 888, in copy_key
response.reason, body)
boto.exception.S3ResponseError: S3ResponseError: 403 Forbidden
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>[...]</AWSAccessKeyId><StringToSign>AWS4-HMAC-SHA256
20150818T231356Z
20150818/us-west-1/s3/aws4_request
e2466b937a1a45c13fa055db0df7b89476ff568f2beef309743ad937b351ce30</StringToSign><SignatureProvided>7df5b18a8015c325e9363c84c8a919e60392f3b7ef20e48b70c45eb0785a7431</SignatureProvided><StringToSignBytes>41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 0a 32 30 31 35 30 38 31 38 54 32 33 31 33 35 36 5a 0a 32 30 31 35 30 38 31 38 2f 75 73 2d 77 65 73 74 2d 31 2f 73 33 2f 61 77 73 34 5f 72 65 71 75 65 73 74 0a 65 32 34 36 36 62 39 33 37 61 31 61 34 35 63 31 33 66 61 30 35 35 64 62 30 64 66 37 62 38 39 34 37 36 66 66 35 36 38 66 32 62 65 65 66 33 30 39 37 34 33 61 64 39 33 37 62 33 35 31 63 65 33 30</StringToSignBytes><CanonicalRequest>PUT
/testing.txt
host:my-bucket-2.s3-us-west-1.amazonaws.com
user-agent:Boto/2.38.0 Python/2.7.6 Darwin/14.3.0
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-copy-source:my-bucket/testing.txt
x-amz-date:20150818T231356Z
x-amz-metadata-directive:COPY
x-amz-server-side-encryption:aws:kms
x-amz-server-side-encryption-aws-kms-key-id:e2009179-a4fc-4a0a-99c6-490b7bb1ebbb
x-amz-storage-class:STANDARD
host;user-agent;x-amz-content-sha256;x-amz-copy-source;x-amz-date;x-amz-metadata-directive;x-amz-server-side-encryption;x-amz-server-side-encryption-aws-kms-key-id;x-amz-storage-class
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855</CanonicalRequest><CanonicalRequestBytes>50 55 54 0a 2f 61 65 74 6e 61 2f 6d 65 74 61 64 61 74 61 2f 61 65 74 6e 61 5f 63 6f 6c 75 6d 6e 5f 64 65 74 61 69 6c 73 2e 74 78 74 0a 0a 68 6f 73 74 3a 71 75 61 72 74 65 74 68 65 61 6c 74 68 2d 73 65 63 75 72 65 2d 62 61 63 6b 75 70 2e 73 33 2d 75 73 2d 77 65 73 74 2d 31 2e 61 6d 61 7a 6f 6e 61 77 73 2e 63 6f 6d 0a 75 73 65 72 2d 61 67 65 6e 74 3a 42 6f 74 6f 2f 32 2e 33 38 2e 30 20 50 79 74 68 6f 6e 2f 32 2e 37 2e 36 20 44 61 72 77 69 6e 2f 31 34 2e 33 2e 30 0a 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35 0a 78 2d 61 6d 7a 2d 63 6f 70 79 2d 73 6f 75 72 63 65 3a 71 75 61 72 74 65 74 68 65 61 6c 74 68 2d 73 65 63 75 72 65 2f 61 65 74 6e 61 2f 6d 65 74 61 64 61 74 61 2f 61 65 74 6e 61 5f 63 6f 6c 75 6d 6e 5f 64 65 74 61 69 6c 73 2e 74 78 74 0a 78 2d 61 6d 7a 2d 64 61 74 65 3a 32 30 31 35 30 38 31 38 54 32 33 31 33 35 36 5a 0a 78 2d 61 6d 7a 2d 6d 65 74 61 64 61 74 61 2d 64 69 72 65 63 74 69 76 65 3a 43 4f 50 59 0a 78 2d 61 6d 7a 2d 73 65 72 76 65 72 2d 73 69 64 65 2d 65 6e 63 72 79 70 74 69 6f 6e 3a 61 77 73 3a 6b 6d 73 0a 78 2d 61 6d 7a 2d 73 65 72 76 65 72 2d 73 69 64 65 2d 65 6e 63 72 79 70 74 69 6f 6e 2d 61 77 73 2d 6b 6d 73 2d 6b 65 79 2d 69 64 3a 65 32 30 30 39 31 37 39 2d 61 34 66 63 2d 34 61 30 61 2d 39 39 63 36 2d 34 39 30 62 37 62 62 31 65 33 62 32 0a 78 2d 61 6d 7a 2d 73 74 6f 72 61 67 65 2d 63 6c 61 73 73 3a 53 54 41 4e 44 41 52 44 0a 0a 68 6f 73 74 3b 75 73 65 72 2d 61 67 65 6e 74 3b 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3b 78 2d 61 6d 7a 2d 63 6f 70 79 2d 73 6f 75 72 63 65 3b 78 2d 61 6d 7a 2d 64 61 74 65 3b 78 2d 61 6d 7a 2d 6d 65 74 61 64 61 74 61 2d 64 69 72 65 63 74 69 76 65 3b 78 2d 61 6d 7a 2d 73 65 72 76 65 72 2d 73 69 64 65 2d 65 6e 63 72 79 70 74 69 6f 6e 3b 78 2d 61 6d 7a 2d 73 65 72 76 65 72 2d 73 69 64 65 2d 65 6e 63 72 79 70 74 69 6f 6e 2d 61 77 73 2d 6b 6d 73 2d 6b 65 79 2d 69 64 3b 78 2d 61 6d 7a 2d 73 74 6f 72 61 67 65 2d 63 6c 61 73 73 0a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35</CanonicalRequestBytes><RequestId>D0092E395ACDBBBB</RequestId><HostId>jnUTMiF5ohPZBWhz9EboX66iym/9nFQUMUEB2Nk/LsuYiSorG5z3sxkrqIQIPt+1DDhCZxuBvUg=</HostId></Error>
我查看了 "Troubleshooting AWS Signature Version 4 Errors" 页面 (http://docs.aws.amazon.com/general/latest/gr/signature-v4-troubleshooting.html)，它可能会提供一些提示，但这似乎有助于解决 自己的自定义问题 签名代码，而不是 boto SDK 的。
我是不是做错了什么？如何正确地将密钥从一个区域复制到另一个区域（没有本地中介），同时还指定用于服务器端加密的 KMS 加密密钥？
看起来这确实是 boto 中的一个错误，其中 headers 在规范表示中排序不正确。他们最终得到完全排序 'key:value' 而不是仅按 'key' 排序，这最终会使用某些键集产生不同的结果。
不幸的是，这些 PR 仍然很出色：
https://github.com/boto/boto/pull/3176
https://github.com/boto/boto/pull/3032
我在boto3中重新实现了上传，发现代码在那里可以正常工作：
import boto3
# note, be sure to put '[s3] use-sigv4 = True' in ~/.boto
src_bucket_name = 'my-bucket'
dest_bucket_name = 'my-bucket-2'
test_key = '/testing.txt'
kms_key = '1e7ff509-6f87-412f-a7af-07ff72ea1111'
dest_bucket_region = 'us-west-1'
client = boto3.client('s3', dest_bucket_region)
response = client.copy_object(
Bucket=dest_bucket_name,
CopySource=src_bucket_name+test_key, # apparently must not be urlencoded, contrary to documentation
Key=test_key,
ServerSideEncryption='aws:kms',
StorageClass='STANDARD',
SSEKMSKeyId=kms_key
)
猜猜使用最新最好的是值得的！
我正在使用 boto 版本 2.38.0 尝试将 s3 上的密钥复制 region-to-region 到不同区域中不同存储桶中的相同密钥。源密钥使用 KMS 密钥加密，目标也应使用 KMS 密钥加密，尽管是不同的密钥。
我可以使用以下代码正确复制到未加密的目标密钥：
import boto
from boto import connect_s3
if not boto.config.get('s3', 'use-sigv4'):
boto.config.add_section('s3')
boto.config.set('s3', 'use-sigv4', 'True')
src_bucket_name = 'my-bucket'
dest_bucket_name = 'my-bucket-2'
conn_std = connect_s3(host='s3.amazonaws.com')
conn_norcal = connect_s3(host='s3-us-west-1.amazonaws.com')
src = conn_std.get_bucket(src_bucket_name)
dest = conn_norcal.get_bucket(dest_bucket_name)
for src_item in src_items:
dest.copy_key(src_item.key, src.name, src_item.key)
但是，如果我尝试使用此处指定的 headers (http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectCOPY.html) 通过 KMS 密钥启用 SSE，我会收到 SignatureDoesNotMatch 错误。
因此，例如，如果我将 copy_key 行更改为：
dest.copy_key(src_item.key, src.name, src_item.key, headers={
'x-amz-server-side-encryption':'aws:kms',
'x-amz-server-side-encryption-aws-kms-key-id': 'e2009179-a4fc-4a0a-99c6-490b7bb1ebbb'
})
然后我得到如下错误：
## Note: I replaced all sensitive details with [...],
## and used my-bucket and my-bucket-2 with testing.txt.
Traceback (most recent call last):
[...]
File "/Library/Python/2.7/site-packages/boto/s3/bucket.py", line 888, in copy_key
response.reason, body)
boto.exception.S3ResponseError: S3ResponseError: 403 Forbidden
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>[...]</AWSAccessKeyId><StringToSign>AWS4-HMAC-SHA256
20150818T231356Z
20150818/us-west-1/s3/aws4_request
e2466b937a1a45c13fa055db0df7b89476ff568f2beef309743ad937b351ce30</StringToSign><SignatureProvided>7df5b18a8015c325e9363c84c8a919e60392f3b7ef20e48b70c45eb0785a7431</SignatureProvided><StringToSignBytes>41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 0a 32 30 31 35 30 38 31 38 54 32 33 31 33 35 36 5a 0a 32 30 31 35 30 38 31 38 2f 75 73 2d 77 65 73 74 2d 31 2f 73 33 2f 61 77 73 34 5f 72 65 71 75 65 73 74 0a 65 32 34 36 36 62 39 33 37 61 31 61 34 35 63 31 33 66 61 30 35 35 64 62 30 64 66 37 62 38 39 34 37 36 66 66 35 36 38 66 32 62 65 65 66 33 30 39 37 34 33 61 64 39 33 37 62 33 35 31 63 65 33 30</StringToSignBytes><CanonicalRequest>PUT
/testing.txt
host:my-bucket-2.s3-us-west-1.amazonaws.com
user-agent:Boto/2.38.0 Python/2.7.6 Darwin/14.3.0
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-copy-source:my-bucket/testing.txt
x-amz-date:20150818T231356Z
x-amz-metadata-directive:COPY
x-amz-server-side-encryption:aws:kms
x-amz-server-side-encryption-aws-kms-key-id:e2009179-a4fc-4a0a-99c6-490b7bb1ebbb
x-amz-storage-class:STANDARD
host;user-agent;x-amz-content-sha256;x-amz-copy-source;x-amz-date;x-amz-metadata-directive;x-amz-server-side-encryption;x-amz-server-side-encryption-aws-kms-key-id;x-amz-storage-class
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855</CanonicalRequest><CanonicalRequestBytes>50 55 54 0a 2f 61 65 74 6e 61 2f 6d 65 74 61 64 61 74 61 2f 61 65 74 6e 61 5f 63 6f 6c 75 6d 6e 5f 64 65 74 61 69 6c 73 2e 74 78 74 0a 0a 68 6f 73 74 3a 71 75 61 72 74 65 74 68 65 61 6c 74 68 2d 73 65 63 75 72 65 2d 62 61 63 6b 75 70 2e 73 33 2d 75 73 2d 77 65 73 74 2d 31 2e 61 6d 61 7a 6f 6e 61 77 73 2e 63 6f 6d 0a 75 73 65 72 2d 61 67 65 6e 74 3a 42 6f 74 6f 2f 32 2e 33 38 2e 30 20 50 79 74 68 6f 6e 2f 32 2e 37 2e 36 20 44 61 72 77 69 6e 2f 31 34 2e 33 2e 30 0a 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35 0a 78 2d 61 6d 7a 2d 63 6f 70 79 2d 73 6f 75 72 63 65 3a 71 75 61 72 74 65 74 68 65 61 6c 74 68 2d 73 65 63 75 72 65 2f 61 65 74 6e 61 2f 6d 65 74 61 64 61 74 61 2f 61 65 74 6e 61 5f 63 6f 6c 75 6d 6e 5f 64 65 74 61 69 6c 73 2e 74 78 74 0a 78 2d 61 6d 7a 2d 64 61 74 65 3a 32 30 31 35 30 38 31 38 54 32 33 31 33 35 36 5a 0a 78 2d 61 6d 7a 2d 6d 65 74 61 64 61 74 61 2d 64 69 72 65 63 74 69 76 65 3a 43 4f 50 59 0a 78 2d 61 6d 7a 2d 73 65 72 76 65 72 2d 73 69 64 65 2d 65 6e 63 72 79 70 74 69 6f 6e 3a 61 77 73 3a 6b 6d 73 0a 78 2d 61 6d 7a 2d 73 65 72 76 65 72 2d 73 69 64 65 2d 65 6e 63 72 79 70 74 69 6f 6e 2d 61 77 73 2d 6b 6d 73 2d 6b 65 79 2d 69 64 3a 65 32 30 30 39 31 37 39 2d 61 34 66 63 2d 34 61 30 61 2d 39 39 63 36 2d 34 39 30 62 37 62 62 31 65 33 62 32 0a 78 2d 61 6d 7a 2d 73 74 6f 72 61 67 65 2d 63 6c 61 73 73 3a 53 54 41 4e 44 41 52 44 0a 0a 68 6f 73 74 3b 75 73 65 72 2d 61 67 65 6e 74 3b 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3b 78 2d 61 6d 7a 2d 63 6f 70 79 2d 73 6f 75 72 63 65 3b 78 2d 61 6d 7a 2d 64 61 74 65 3b 78 2d 61 6d 7a 2d 6d 65 74 61 64 61 74 61 2d 64 69 72 65 63 74 69 76 65 3b 78 2d 61 6d 7a 2d 73 65 72 76 65 72 2d 73 69 64 65 2d 65 6e 63 72 79 70 74 69 6f 6e 3b 78 2d 61 6d 7a 2d 73 65 72 76 65 72 2d 73 69 64 65 2d 65 6e 63 72 79 70 74 69 6f 6e 2d 61 77 73 2d 6b 6d 73 2d 6b 65 79 2d 69 64 3b 78 2d 61 6d 7a 2d 73 74 6f 72 61 67 65 2d 63 6c 61 73 73 0a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35</CanonicalRequestBytes><RequestId>D0092E395ACDBBBB</RequestId><HostId>jnUTMiF5ohPZBWhz9EboX66iym/9nFQUMUEB2Nk/LsuYiSorG5z3sxkrqIQIPt+1DDhCZxuBvUg=</HostId></Error>
我查看了 "Troubleshooting AWS Signature Version 4 Errors" 页面 (http://docs.aws.amazon.com/general/latest/gr/signature-v4-troubleshooting.html)，它可能会提供一些提示，但这似乎有助于解决 自己的自定义问题 签名代码，而不是 boto SDK 的。
我是不是做错了什么？如何正确地将密钥从一个区域复制到另一个区域（没有本地中介），同时还指定用于服务器端加密的 KMS 加密密钥？
看起来这确实是 boto 中的一个错误，其中 headers 在规范表示中排序不正确。他们最终得到完全排序 'key:value' 而不是仅按 'key' 排序，这最终会使用某些键集产生不同的结果。
不幸的是，这些 PR 仍然很出色：
https://github.com/boto/boto/pull/3176
https://github.com/boto/boto/pull/3032
我在boto3中重新实现了上传，发现代码在那里可以正常工作：
import boto3
# note, be sure to put '[s3] use-sigv4 = True' in ~/.boto
src_bucket_name = 'my-bucket'
dest_bucket_name = 'my-bucket-2'
test_key = '/testing.txt'
kms_key = '1e7ff509-6f87-412f-a7af-07ff72ea1111'
dest_bucket_region = 'us-west-1'
client = boto3.client('s3', dest_bucket_region)
response = client.copy_object(
Bucket=dest_bucket_name,
CopySource=src_bucket_name+test_key, # apparently must not be urlencoded, contrary to documentation
Key=test_key,
ServerSideEncryption='aws:kms',
StorageClass='STANDARD',
SSEKMSKeyId=kms_key
)
猜猜使用最新最好的是值得的！