Python - Fine Uploader 服务器端 AWS 版本 4 签名请求

Python - Fine Uploader Server Side AWS Version 4 signing request

从 python web2py 创建请求后。我从 fine uploader

收到以下错误

"The request signature we calculated does not match the signature you provided. Check your key and signing method."

这是我的服务器端代码

def _sign(key, msg):
    return hmac.new(key, msg.encode("utf-8"), hashlib.sha256).digest()

def getV4Signature(date_stamp, regionName, policy):
    kDate = _sign(('AWS4' + AWS_SECRET_KEY).encode('utf-8'), date_stamp)
    kRegion = _sign(kDate, regionName)
    kService = _sign(kRegion, 's3')
    kSigning = _sign(kService, 'aws4_request')
    kSignature = _sign(kSigning, policy)
    return binascii.hexlify(kSignature)

我的回答是假设您正在使用 python3。它适用于文件分块或常规上传的 AWS v4 签名。首先,在您的 javascript 中,确保您在 signature 部分中指定了 version:4

 <script type="text/javascript">
     // execute the code after the document is loaded
     document.addEventListener("DOMContentLoaded", function() {
         // The code
        (function() {
            var uploader = new qq.s3.FineUploader({
                debug: true,
                element: document.getElementById('fine-uploader'),
                cors: {
                  expected: true
                },
                objectProperties: {
                      bucket: '<your bucket>',
                      region: 'cn-north-1',
                      acl: 'private',
                      key: "uuid",
                    },
                request: {
                    endpoint: 'https://<your bucket>.s3.cn-north-1.amazonaws.com.cn',
                    accessKey: '<your public access key>',

                },
                signature: {
                    endpoint: '{{ url_for('data.s3_signature') }}',
                    version: 4,
                    customHeaders: {
                              "X-CSRF-Token": $("meta[name='csrf-token']").attr("content")
                            }
                },
                uploadSuccess: {
                    endpoint: '{{ url_for('data.s3_success') }}'
                },
                iframeSupport: {
                    localBlankPagePath: '/success.html'
                },
                chunking: {
                    enabled: true,
                    concurrent: {
                        enabled: true
                    }
                },
                resume: {
                    enabled: true
                },
                retry: {
                   enableAuto: true // defaults to false
                },
                deleteFile: {
                    enabled: true,
                    endpoint: '{{ url_for('data.s3_delete', key=key) }}'
                }
            })
        }());
    });
    </script>

接下来,三种用于签名的辅助方法和一种用于从 fineuploader 发送的保单日期数据中获取日期的方法:

def hash_sha256(msg:str):
    """
    Generate a SHA256 hash and return the base16 Uicode string.

    msg -- A Unicode message to hash.

    """
    return binascii.hexlify(hashlib.sha256(
        bytearray(msg.strip(), 'utf-8')).digest()).decode('utf-8')

def sign_sha256(key, msg):
    """
    Generate an SHA256 HMAC, encoding msg to UTF-8 if not
    already encoded.

    key -- signing key. bytes.
    msg -- message to sign. unicode or bytes.

    """
    if isinstance(msg, text_type):
        msg = msg.encode('utf-8')
    return hmac.new(key, msg, hashlib.sha256).digest()

def generate_key(cls, secret_key, region, service, date,
                 intermediates=False):
    """
    Generate the signing key string as bytes.

    If intermediate is set to True, returns a 4-tuple containing the key
    and the intermediate keys:

    ( signing_key, date_key, region_key, service_key )

    The intermediate keys can be used for testing against examples from
    Amazon.

    """
    init_key = ('AWS4' + secret_key).encode('utf-8')
    date_key = cls.sign_sha256(init_key, date)
    region_key = cls.sign_sha256(date_key, region)
    service_key = cls.sign_sha256(region_key, service)
    key = cls.sign_sha256(service_key, 'aws4_request')
    if intermediates:
        return (key, date_key, region_key, service_key)
    else:
        return key

def get_condition(list_of_dicts, condition):
    """input a list_of_dicts as found in policy['conditions'] and then iterate
    the dict keys in the list to get the condition."""
    for d in list_of_dicts:
        for k in d.keys():
            if condition in k:
                return d.get(k)

接下来,我将向您推荐服务器端 python 示例,如 https://github.com/FineUploader/server-examples/blob/master/python/python3-flask-fine-uploader-s3/s3-sign-srv.py 所示,但此示例用于 v2 签名,因此我们需要修改 [=该示例中的 16=] 和 sign_headers() 来处理 AWS v4 签名要求。

def sign_policy(policy):
    """ Sign (aws v4) and return the policy document for a simple upload.
    http://aws.amazon.com/articles/1434/#signyours3postform """
    policy_data = json.loads(policy)
    yyyymmdd = get_condition(policy_data['conditions'], 'x-amz-date')[:8]
    encoded_policy = base64.b64encode(
        bytearray(json.dumps(policy_data), 'utf-8'))
    signing_key = generate_key(AWS_CLIENT_SECRET_KEY, S3_REGION_NAME,
                                                's3', yyyymmdd)
    signature = sign_sha256(signing_key, encoded_policy.decode('utf-8'))

    return {
        'policy': encoded_policy.decode("utf-8"),
        'signature': binascii.hexlify(signature).decode("utf-8")
    }


def sign_headers(headers):
    """ Sign and return the headers for a chunked upload.
    The headers sent from fineupload have the majority of what is needed. But
    you must follow the process laid out in amazon docs to finish it.

    https://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
    """
    # cut the first three lines from the headers from fine uploader and sign the remaining as hashed_canonical_request
    canonical_request = headers[70:]
    # split the headers so you can build the string_to_sign
    split_headers = headers.splitlines()
    # grab the date from the header
    yyyymmdd = split_headers[2][:8]
    # hash the canonical request and then return the base16 string
    hashed_canonical_request =  hash_sha256(canonical_request)
    # build the string_to_sign
    string_to_sign = split_headers[0] + '\n' + split_headers[1] + '\n' + split_headers[2] + '\n' + hashed_canonical_request
    # create the signing key
    signing_key = generate_key(AWS_CLIENT_SECRET_KEY, S3_REGION_NAME, 's3', yyyymmdd)
    # create the signature using the signing_key and string_to_sign
    signature = sign_sha256(signing_key, string_to_sign)

    return {
        'signature': binascii.hexlify(signature).decode("utf-8")
    }

如果您已按照此处的说明完成操作,并且星星也对您有利,那么它应该会起作用。但是,如果您仍然有错误,我发现使用 chrome 开发工具来查看从 amazon s3 返回的关于 403 错误的 xml 数据非常有帮助。它看起来像这样(在应该标记为红色的 403 错误行的“网络”选项卡上找到):

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>[your public key]</AWSAccessKeyId><StringToSign>AWS4-HMAC-SHA256
20180725T101212Z
20180725/cn-north-1/s3/aws4_request
e7a7e92e17d8a3ac6228bb02139a499904db50a493ea6c336d847d4d94a5c320</StringToSign><SignatureProvided>2afc0bc1316732c9cd9bdc75c0aafdde70c3c96c0211991c610cf0c1bed33d71</SignatureProvided><StringToSignBytes>41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 0a 32 30 31 38 30 37 32 35 54 31 30 31 32 31 32 5a 0a 32 30 31 38 30 37 32 35 2f 63 6e 2d 6e 6f 72 74 68 2d 31 2f 73 33 2f 61 77 73 34 5f 72 65 71 75 65 73 74 0a 65 37 61 37 65 39 32 65 31 37 64 38 61 33 61 63 36 32 32 38 62 62 30 32 31 33 39 61 34 39 39 39 30 34 64 62 35 30 61 34 39 33 65 61 36 63 33 33 36 64 38 34 37 64 34 64 39 34 61 35 63 33 32 30</StringToSignBytes><CanonicalRequest>
POST
/39f0808e-3d0b-48d4-a3db-f171a3cb2943.mp4
uploads=
host:<your bucket>.s3.cn-north-1.amazonaws.com.cn
x-amz-acl:private
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20180725T101212Z
x-amz-meta-qqfilename:flux-field_pro_854x480p.mp4

host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-meta-qqfilename
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855</CanonicalRequest><CanonicalRequestBytes>50 4f 53 54 0a 2f 33 39 66 30 38 30 38 65 2d 33 64 30 62 2d 34 38 64 34 2d 61 33 64 62 2d 66 31 37 31 61 33 63 62 32 39 34 33 2e 6d 70 34 0a 75 70 6c 6f 61 64 73 3d 0a 68 6f 73 74 3a 61 6f 62 71 2e 73 33 2e 63 6e 2d 6e 6f 72 74 68 2d 31 2e 61 6d 61 7a 6f 6e 61 77 73 2e 63 6f 6d 2e 63 6e 0a 78 2d 61 6d 7a 2d 61 63 6c 3a 70 72 69 76 61 74 65 0a 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35 0a 78 2d 61 6d 7a 2d 64 61 74 65 3a 32 30 31 38 30 37 32 35 54 31 30 31 32 31 32 5a 0a 78 2d 61 6d 7a 2d 6d 65 74 61 2d 71 71 66 69 6c 65 6e 61 6d 65 3a 66 6c 75 78 2d 66 69 65 6c 64 5f 70 72 6f 5f 38 35 34 78 34 38 30 70 2e 6d 70 34 0a 0a 68 6f 73 74 3b 78 2d 61 6d 7a 2d 61 63 6c 3b 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3b 78 2d 61 6d 7a 2d 64 61 74 65 3b 78 2d 61 6d 7a 2d 6d 65 74 61 2d 71 71 66 69 6c 65 6e 61 6d 65 0a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35</CanonicalRequestBytes><RequestId>056AA9A01FBFF8FB</RequestId><HostId>RzQYHes10dAU3rrnhyDRRwN4NzuNxn3JrVcjlfK8NEqagFh0DZ0gkT56bMrYNwDTcU2iuZQohaY=</HostId></Error>

您在 sign_headers() 方法中创建的 canonical_request 必须与 xml 数据的 s3 错误 <CanonicalRequest> 部分中显示的内容相匹配。如果它有任何不同,那么你会得到一个错误。同样,sign_headers() 中的 string_to_sign 也必须匹配 xml 中 <StringToSign> 的错误响应。祝你好运。