Python - Fine Uploader 服务器端 AWS 版本 4 签名请求
Python - Fine Uploader Server Side AWS Version 4 signing request
从 python web2py 创建请求后。我从 fine uploader
收到以下错误
"The request signature we calculated does not match the signature you provided. Check your key and signing method."
这是我的服务器端代码
def _sign(key, msg):
return hmac.new(key, msg.encode("utf-8"), hashlib.sha256).digest()
def getV4Signature(date_stamp, regionName, policy):
kDate = _sign(('AWS4' + AWS_SECRET_KEY).encode('utf-8'), date_stamp)
kRegion = _sign(kDate, regionName)
kService = _sign(kRegion, 's3')
kSigning = _sign(kService, 'aws4_request')
kSignature = _sign(kSigning, policy)
return binascii.hexlify(kSignature)
我的回答是假设您正在使用 python3。它适用于文件分块或常规上传的 AWS v4 签名。首先，在您的 javascript 中，确保您在 signature 部分中指定了 version:4：
<script type="text/javascript">
// execute the code after the document is loaded
document.addEventListener("DOMContentLoaded", function() {
// The code
(function() {
var uploader = new qq.s3.FineUploader({
debug: true,
element: document.getElementById('fine-uploader'),
cors: {
expected: true
},
objectProperties: {
bucket: '<your bucket>',
region: 'cn-north-1',
acl: 'private',
key: "uuid",
},
request: {
endpoint: 'https://<your bucket>.s3.cn-north-1.amazonaws.com.cn',
accessKey: '<your public access key>',
},
signature: {
endpoint: '{{ url_for('data.s3_signature') }}',
version: 4,
customHeaders: {
"X-CSRF-Token": $("meta[name='csrf-token']").attr("content")
}
},
uploadSuccess: {
endpoint: '{{ url_for('data.s3_success') }}'
},
iframeSupport: {
localBlankPagePath: '/success.html'
},
chunking: {
enabled: true,
concurrent: {
enabled: true
}
},
resume: {
enabled: true
},
retry: {
enableAuto: true // defaults to false
},
deleteFile: {
enabled: true,
endpoint: '{{ url_for('data.s3_delete', key=key) }}'
}
})
}());
});
</script>
接下来，三种用于签名的辅助方法和一种用于从 fineuploader 发送的保单日期数据中获取日期的方法：
def hash_sha256(msg:str):
"""
Generate a SHA256 hash and return the base16 Uicode string.
msg -- A Unicode message to hash.
"""
return binascii.hexlify(hashlib.sha256(
bytearray(msg.strip(), 'utf-8')).digest()).decode('utf-8')
def sign_sha256(key, msg):
"""
Generate an SHA256 HMAC, encoding msg to UTF-8 if not
already encoded.
key -- signing key. bytes.
msg -- message to sign. unicode or bytes.
"""
if isinstance(msg, text_type):
msg = msg.encode('utf-8')
return hmac.new(key, msg, hashlib.sha256).digest()
def generate_key(cls, secret_key, region, service, date,
intermediates=False):
"""
Generate the signing key string as bytes.
If intermediate is set to True, returns a 4-tuple containing the key
and the intermediate keys:
( signing_key, date_key, region_key, service_key )
The intermediate keys can be used for testing against examples from
Amazon.
"""
init_key = ('AWS4' + secret_key).encode('utf-8')
date_key = cls.sign_sha256(init_key, date)
region_key = cls.sign_sha256(date_key, region)
service_key = cls.sign_sha256(region_key, service)
key = cls.sign_sha256(service_key, 'aws4_request')
if intermediates:
return (key, date_key, region_key, service_key)
else:
return key
def get_condition(list_of_dicts, condition):
"""input a list_of_dicts as found in policy['conditions'] and then iterate
the dict keys in the list to get the condition."""
for d in list_of_dicts:
for k in d.keys():
if condition in k:
return d.get(k)
接下来，我将向您推荐服务器端 python 示例，如 https://github.com/FineUploader/server-examples/blob/master/python/python3-flask-fine-uploader-s3/s3-sign-srv.py 所示，但此示例用于 v2 签名，因此我们需要修改 [=该示例中的 16=] 和 sign_headers() 来处理 AWS v4 签名要求。
def sign_policy(policy):
""" Sign (aws v4) and return the policy document for a simple upload.
http://aws.amazon.com/articles/1434/#signyours3postform """
policy_data = json.loads(policy)
yyyymmdd = get_condition(policy_data['conditions'], 'x-amz-date')[:8]
encoded_policy = base64.b64encode(
bytearray(json.dumps(policy_data), 'utf-8'))
signing_key = generate_key(AWS_CLIENT_SECRET_KEY, S3_REGION_NAME,
's3', yyyymmdd)
signature = sign_sha256(signing_key, encoded_policy.decode('utf-8'))
return {
'policy': encoded_policy.decode("utf-8"),
'signature': binascii.hexlify(signature).decode("utf-8")
}
def sign_headers(headers):
""" Sign and return the headers for a chunked upload.
The headers sent from fineupload have the majority of what is needed. But
you must follow the process laid out in amazon docs to finish it.
https://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
"""
# cut the first three lines from the headers from fine uploader and sign the remaining as hashed_canonical_request
canonical_request = headers[70:]
# split the headers so you can build the string_to_sign
split_headers = headers.splitlines()
# grab the date from the header
yyyymmdd = split_headers[2][:8]
# hash the canonical request and then return the base16 string
hashed_canonical_request = hash_sha256(canonical_request)
# build the string_to_sign
string_to_sign = split_headers[0] + '\n' + split_headers[1] + '\n' + split_headers[2] + '\n' + hashed_canonical_request
# create the signing key
signing_key = generate_key(AWS_CLIENT_SECRET_KEY, S3_REGION_NAME, 's3', yyyymmdd)
# create the signature using the signing_key and string_to_sign
signature = sign_sha256(signing_key, string_to_sign)
return {
'signature': binascii.hexlify(signature).decode("utf-8")
}
如果您已按照此处的说明完成操作，并且星星也对您有利，那么它应该会起作用。但是，如果您仍然有错误，我发现使用 chrome 开发工具来查看从 amazon s3 返回的关于 403 错误的 xml 数据非常有帮助。它看起来像这样（在应该标记为红色的 403 错误行的“网络”选项卡上找到）：
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>[your public key]</AWSAccessKeyId><StringToSign>AWS4-HMAC-SHA256
20180725T101212Z
20180725/cn-north-1/s3/aws4_request
e7a7e92e17d8a3ac6228bb02139a499904db50a493ea6c336d847d4d94a5c320</StringToSign><SignatureProvided>2afc0bc1316732c9cd9bdc75c0aafdde70c3c96c0211991c610cf0c1bed33d71</SignatureProvided><StringToSignBytes>41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 0a 32 30 31 38 30 37 32 35 54 31 30 31 32 31 32 5a 0a 32 30 31 38 30 37 32 35 2f 63 6e 2d 6e 6f 72 74 68 2d 31 2f 73 33 2f 61 77 73 34 5f 72 65 71 75 65 73 74 0a 65 37 61 37 65 39 32 65 31 37 64 38 61 33 61 63 36 32 32 38 62 62 30 32 31 33 39 61 34 39 39 39 30 34 64 62 35 30 61 34 39 33 65 61 36 63 33 33 36 64 38 34 37 64 34 64 39 34 61 35 63 33 32 30</StringToSignBytes><CanonicalRequest>
POST
/39f0808e-3d0b-48d4-a3db-f171a3cb2943.mp4
uploads=
host:<your bucket>.s3.cn-north-1.amazonaws.com.cn
x-amz-acl:private
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20180725T101212Z
x-amz-meta-qqfilename:flux-field_pro_854x480p.mp4
host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-meta-qqfilename
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855</CanonicalRequest><CanonicalRequestBytes>50 4f 53 54 0a 2f 33 39 66 30 38 30 38 65 2d 33 64 30 62 2d 34 38 64 34 2d 61 33 64 62 2d 66 31 37 31 61 33 63 62 32 39 34 33 2e 6d 70 34 0a 75 70 6c 6f 61 64 73 3d 0a 68 6f 73 74 3a 61 6f 62 71 2e 73 33 2e 63 6e 2d 6e 6f 72 74 68 2d 31 2e 61 6d 61 7a 6f 6e 61 77 73 2e 63 6f 6d 2e 63 6e 0a 78 2d 61 6d 7a 2d 61 63 6c 3a 70 72 69 76 61 74 65 0a 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35 0a 78 2d 61 6d 7a 2d 64 61 74 65 3a 32 30 31 38 30 37 32 35 54 31 30 31 32 31 32 5a 0a 78 2d 61 6d 7a 2d 6d 65 74 61 2d 71 71 66 69 6c 65 6e 61 6d 65 3a 66 6c 75 78 2d 66 69 65 6c 64 5f 70 72 6f 5f 38 35 34 78 34 38 30 70 2e 6d 70 34 0a 0a 68 6f 73 74 3b 78 2d 61 6d 7a 2d 61 63 6c 3b 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3b 78 2d 61 6d 7a 2d 64 61 74 65 3b 78 2d 61 6d 7a 2d 6d 65 74 61 2d 71 71 66 69 6c 65 6e 61 6d 65 0a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35</CanonicalRequestBytes><RequestId>056AA9A01FBFF8FB</RequestId><HostId>RzQYHes10dAU3rrnhyDRRwN4NzuNxn3JrVcjlfK8NEqagFh0DZ0gkT56bMrYNwDTcU2iuZQohaY=</HostId></Error>
您在 sign_headers() 方法中创建的 canonical_request 必须与 xml 数据的 s3 错误 <CanonicalRequest> 部分中显示的内容相匹配。如果它有任何不同，那么你会得到一个错误。同样，sign_headers() 中的 string_to_sign 也必须匹配 xml 中 <StringToSign> 的错误响应。祝你好运。
从 python web2py 创建请求后。我从 fine uploader
收到以下错误"The request signature we calculated does not match the signature you provided. Check your key and signing method."
这是我的服务器端代码
def _sign(key, msg):
return hmac.new(key, msg.encode("utf-8"), hashlib.sha256).digest()
def getV4Signature(date_stamp, regionName, policy):
kDate = _sign(('AWS4' + AWS_SECRET_KEY).encode('utf-8'), date_stamp)
kRegion = _sign(kDate, regionName)
kService = _sign(kRegion, 's3')
kSigning = _sign(kService, 'aws4_request')
kSignature = _sign(kSigning, policy)
return binascii.hexlify(kSignature)
我的回答是假设您正在使用 python3。它适用于文件分块或常规上传的 AWS v4 签名。首先，在您的 javascript 中，确保您在 signature 部分中指定了 version:4：
<script type="text/javascript">
// execute the code after the document is loaded
document.addEventListener("DOMContentLoaded", function() {
// The code
(function() {
var uploader = new qq.s3.FineUploader({
debug: true,
element: document.getElementById('fine-uploader'),
cors: {
expected: true
},
objectProperties: {
bucket: '<your bucket>',
region: 'cn-north-1',
acl: 'private',
key: "uuid",
},
request: {
endpoint: 'https://<your bucket>.s3.cn-north-1.amazonaws.com.cn',
accessKey: '<your public access key>',
},
signature: {
endpoint: '{{ url_for('data.s3_signature') }}',
version: 4,
customHeaders: {
"X-CSRF-Token": $("meta[name='csrf-token']").attr("content")
}
},
uploadSuccess: {
endpoint: '{{ url_for('data.s3_success') }}'
},
iframeSupport: {
localBlankPagePath: '/success.html'
},
chunking: {
enabled: true,
concurrent: {
enabled: true
}
},
resume: {
enabled: true
},
retry: {
enableAuto: true // defaults to false
},
deleteFile: {
enabled: true,
endpoint: '{{ url_for('data.s3_delete', key=key) }}'
}
})
}());
});
</script>
接下来，三种用于签名的辅助方法和一种用于从 fineuploader 发送的保单日期数据中获取日期的方法：
def hash_sha256(msg:str):
"""
Generate a SHA256 hash and return the base16 Uicode string.
msg -- A Unicode message to hash.
"""
return binascii.hexlify(hashlib.sha256(
bytearray(msg.strip(), 'utf-8')).digest()).decode('utf-8')
def sign_sha256(key, msg):
"""
Generate an SHA256 HMAC, encoding msg to UTF-8 if not
already encoded.
key -- signing key. bytes.
msg -- message to sign. unicode or bytes.
"""
if isinstance(msg, text_type):
msg = msg.encode('utf-8')
return hmac.new(key, msg, hashlib.sha256).digest()
def generate_key(cls, secret_key, region, service, date,
intermediates=False):
"""
Generate the signing key string as bytes.
If intermediate is set to True, returns a 4-tuple containing the key
and the intermediate keys:
( signing_key, date_key, region_key, service_key )
The intermediate keys can be used for testing against examples from
Amazon.
"""
init_key = ('AWS4' + secret_key).encode('utf-8')
date_key = cls.sign_sha256(init_key, date)
region_key = cls.sign_sha256(date_key, region)
service_key = cls.sign_sha256(region_key, service)
key = cls.sign_sha256(service_key, 'aws4_request')
if intermediates:
return (key, date_key, region_key, service_key)
else:
return key
def get_condition(list_of_dicts, condition):
"""input a list_of_dicts as found in policy['conditions'] and then iterate
the dict keys in the list to get the condition."""
for d in list_of_dicts:
for k in d.keys():
if condition in k:
return d.get(k)
接下来，我将向您推荐服务器端 python 示例，如 https://github.com/FineUploader/server-examples/blob/master/python/python3-flask-fine-uploader-s3/s3-sign-srv.py 所示，但此示例用于 v2 签名，因此我们需要修改 [=该示例中的 16=] 和 sign_headers() 来处理 AWS v4 签名要求。
def sign_policy(policy):
""" Sign (aws v4) and return the policy document for a simple upload.
http://aws.amazon.com/articles/1434/#signyours3postform """
policy_data = json.loads(policy)
yyyymmdd = get_condition(policy_data['conditions'], 'x-amz-date')[:8]
encoded_policy = base64.b64encode(
bytearray(json.dumps(policy_data), 'utf-8'))
signing_key = generate_key(AWS_CLIENT_SECRET_KEY, S3_REGION_NAME,
's3', yyyymmdd)
signature = sign_sha256(signing_key, encoded_policy.decode('utf-8'))
return {
'policy': encoded_policy.decode("utf-8"),
'signature': binascii.hexlify(signature).decode("utf-8")
}
def sign_headers(headers):
""" Sign and return the headers for a chunked upload.
The headers sent from fineupload have the majority of what is needed. But
you must follow the process laid out in amazon docs to finish it.
https://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
"""
# cut the first three lines from the headers from fine uploader and sign the remaining as hashed_canonical_request
canonical_request = headers[70:]
# split the headers so you can build the string_to_sign
split_headers = headers.splitlines()
# grab the date from the header
yyyymmdd = split_headers[2][:8]
# hash the canonical request and then return the base16 string
hashed_canonical_request = hash_sha256(canonical_request)
# build the string_to_sign
string_to_sign = split_headers[0] + '\n' + split_headers[1] + '\n' + split_headers[2] + '\n' + hashed_canonical_request
# create the signing key
signing_key = generate_key(AWS_CLIENT_SECRET_KEY, S3_REGION_NAME, 's3', yyyymmdd)
# create the signature using the signing_key and string_to_sign
signature = sign_sha256(signing_key, string_to_sign)
return {
'signature': binascii.hexlify(signature).decode("utf-8")
}
如果您已按照此处的说明完成操作，并且星星也对您有利，那么它应该会起作用。但是，如果您仍然有错误，我发现使用 chrome 开发工具来查看从 amazon s3 返回的关于 403 错误的 xml 数据非常有帮助。它看起来像这样（在应该标记为红色的 403 错误行的“网络”选项卡上找到）：
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>[your public key]</AWSAccessKeyId><StringToSign>AWS4-HMAC-SHA256
20180725T101212Z
20180725/cn-north-1/s3/aws4_request
e7a7e92e17d8a3ac6228bb02139a499904db50a493ea6c336d847d4d94a5c320</StringToSign><SignatureProvided>2afc0bc1316732c9cd9bdc75c0aafdde70c3c96c0211991c610cf0c1bed33d71</SignatureProvided><StringToSignBytes>41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 0a 32 30 31 38 30 37 32 35 54 31 30 31 32 31 32 5a 0a 32 30 31 38 30 37 32 35 2f 63 6e 2d 6e 6f 72 74 68 2d 31 2f 73 33 2f 61 77 73 34 5f 72 65 71 75 65 73 74 0a 65 37 61 37 65 39 32 65 31 37 64 38 61 33 61 63 36 32 32 38 62 62 30 32 31 33 39 61 34 39 39 39 30 34 64 62 35 30 61 34 39 33 65 61 36 63 33 33 36 64 38 34 37 64 34 64 39 34 61 35 63 33 32 30</StringToSignBytes><CanonicalRequest>
POST
/39f0808e-3d0b-48d4-a3db-f171a3cb2943.mp4
uploads=
host:<your bucket>.s3.cn-north-1.amazonaws.com.cn
x-amz-acl:private
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20180725T101212Z
x-amz-meta-qqfilename:flux-field_pro_854x480p.mp4
host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-meta-qqfilename
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855</CanonicalRequest><CanonicalRequestBytes>50 4f 53 54 0a 2f 33 39 66 30 38 30 38 65 2d 33 64 30 62 2d 34 38 64 34 2d 61 33 64 62 2d 66 31 37 31 61 33 63 62 32 39 34 33 2e 6d 70 34 0a 75 70 6c 6f 61 64 73 3d 0a 68 6f 73 74 3a 61 6f 62 71 2e 73 33 2e 63 6e 2d 6e 6f 72 74 68 2d 31 2e 61 6d 61 7a 6f 6e 61 77 73 2e 63 6f 6d 2e 63 6e 0a 78 2d 61 6d 7a 2d 61 63 6c 3a 70 72 69 76 61 74 65 0a 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35 0a 78 2d 61 6d 7a 2d 64 61 74 65 3a 32 30 31 38 30 37 32 35 54 31 30 31 32 31 32 5a 0a 78 2d 61 6d 7a 2d 6d 65 74 61 2d 71 71 66 69 6c 65 6e 61 6d 65 3a 66 6c 75 78 2d 66 69 65 6c 64 5f 70 72 6f 5f 38 35 34 78 34 38 30 70 2e 6d 70 34 0a 0a 68 6f 73 74 3b 78 2d 61 6d 7a 2d 61 63 6c 3b 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3b 78 2d 61 6d 7a 2d 64 61 74 65 3b 78 2d 61 6d 7a 2d 6d 65 74 61 2d 71 71 66 69 6c 65 6e 61 6d 65 0a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35</CanonicalRequestBytes><RequestId>056AA9A01FBFF8FB</RequestId><HostId>RzQYHes10dAU3rrnhyDRRwN4NzuNxn3JrVcjlfK8NEqagFh0DZ0gkT56bMrYNwDTcU2iuZQohaY=</HostId></Error>
您在 sign_headers() 方法中创建的 canonical_request 必须与 xml 数据的 s3 错误 <CanonicalRequest> 部分中显示的内容相匹配。如果它有任何不同，那么你会得到一个错误。同样，sign_headers() 中的 string_to_sign 也必须匹配 xml 中 <StringToSign> 的错误响应。祝你好运。