Python3, hmac returns 输出错误?! ARM71, 32位

Python3, hmac returns wrong output?! ARM71, 32bit

我最近在这里问了另一个问题,关于 python,flask with a badsignature exception for session () ,我已经处理这个问题一段时间了,看到另一个问题。

为我的 Flask 应用程序签署客户端会话时 hmac 用于散列签名。在我的个人计算机上,这工作正常,但是当将它移动到为此应用程序构建的嵌入式设备时,hmac 偶尔无法散列密钥(错误输出),这使得会话数据无效。

我做了一个小测试程序 运行 hmac.new() & .update() 多次 (10000) 没有任何错误(在目标机器上)。但是当在 flask 应用程序内部进行调用时,错误发生在大约 60% 的调用中。

调用是在 itsdangerous.py 的 "derive_key" 方法内部进行的,看起来像这样:

  def derive_key(self):
        """This method is called to derive the key.  If you're unhappy with
        the default key derivation choices you can override them here.
        Keep in mind that the key derivation in itsdangerous is not intended
        to be used as a security method to make a complex key out of a short
        password.  Instead you should use large random secret keys.
        """
        salt = want_bytes(self.salt)
        if self.key_derivation == 'concat':
            return self.digest_method(salt + self.secret_key).digest()
        elif self.key_derivation == 'django-concat':
            return self.digest_method(salt + b'signer' +
                self.secret_key).digest()
        elif self.key_derivation == 'hmac':
            mac = hmac.new(self.secret_key, digestmod=self.digest_method)
            print("mac1:", binascii.hexlify(mac.digest()))  #1
            mac.update(salt)
            print("mac2:", binascii.hexlify(mac.digest()))  #2
            return mac.digest()
        elif self.key_derivation == 'none':
            return self.secret_key
        else:
            raise TypeError('Unknown key derivation method')

digestmod = hashlib.sha1

使用 secret_key = b'testing' 和 salt=b'cookie-session' 预期输出为:

mac1: b'6ab6fc891eefd3b78743ea28b1803811561a7c9b'
mac2: b'd58bd52b4ced54374ea5baca0b6aa52b0e03af74'

但很多时候这些值不同。 我还看到 mac1 和 mac2 的输出是相等的!就像盐没有修改结果。

我也在这里问过这个问题: https://github.com/pallets/flask/issues/1808

此应用程序 运行 在 ARM7、32 位上。 使用 yocto 安装的所有库。

更新: 对于 derive_key() 的每次调用,我还打印 salt 和 key:几个请求的输出如下:

...: OPEN THE SESSION       OK!
...: DERIVE KEY:
...: Salt: b'cookie-session'
...: Key: b'testing'
...: mac1: b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
...: mac2: b'\xd5\x8b\xd5+L\xedT7N\xa5\xba\xca\x0bj\xa5+\x0e\x03\xaft'
...: .-.-.-.-.-.-.
...: SAVES THE SESSION      OK!
...: DERIVE KEY:
...: Salt: b'cookie-session'
...: Key: b'testing'
...: mac1: b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
...: mac2: b'\xd5\x8b\xd5+L\xedT7N\xa5\xba\xca\x0bj\xa5+\x0e\x03\xaft'
...: .-.-.-.-.-.-.
...: OPEN THE SESSION       NOT OKAY!!
...: DERIVE KEY:
...: Salt: b'cookie-session'
...: Key: b'testing'
...: mac1: b'\xc8D\xf0\x95\xc5R\x9f\xe3n\xc7\xa2 `7\xa9\xdb\xdd\xd8F\x85'
...: mac2: b'\x156\xbf\xb6\x97}m\xe9[\xe0\xea\xd15\xb4\xff\x00\xf9\x14B\x0c'
...: .-.-.-.-.-.-.
...: SAVES THE SESSION      OK!
...: DERIVE KEY:
...: Salt: b'cookie-session'
...: Key: b'testing'
...: mac1: b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
...: mac2: b'\xd5\x8b\xd5+L\xedT7N\xa5\xba\xca\x0bj\xa5+\x0e\x03\xaft'
...: .-.-.-.-.-.-.
...: OPEN THE SESSION       OK!
...: DERIVE KEY:
...: Salt: b'cookie-session'
...: Key: b'testing'
...: mac1: b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
...: mac2: b'\xd5\x8b\xd5+L\xedT7N\xa5\xba\xca\x0bj\xa5+\x0e\x03\xaft'
...: .-.-.-.-.-.-.
...: SAVES THE SESSION      OK!
...: DERIVE KEY:
...: Salt: b'cookie-session'
...: Key: b'testing'
...: mac1: b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
...: mac2: b'\xd5\x8b\xd5+L\xedT7N\xa5\xba\xca\x0bj\xa5+\x0e\x03\xaft'
...: .-.-.-.-.-.-.
...: OPEN THE SESSION       OK!
...: DERIVE KEY:
...: Salt: b'cookie-session'
...: Key: b'testing'
...: mac1: b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
...: mac2: b'\xd5\x8b\xd5+L\xedT7N\xa5\xba\xca\x0bj\xa5+\x0e\x03\xaft'
...: .-.-.-.-.-.-.
...: SAVES THE SESSION      OK!
...: DERIVE KEY:
...: Salt: b'cookie-session'
...: Key: b'testing'
...: mac1: b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
...: mac2: b'\xd5\x8b\xd5+L\xedT7N\xa5\xba\xca\x0bj\xa5+\x0e\x03\xaft'
...: .-.-.-.-.-.-.
...: OPEN THE SESSION       NOT OKAY!!
...: DERIVE KEY:
...: Salt: b'cookie-session'
...: Key: b'testing'
...: mac1: b'D\xdaR}\xa0\xf2\x9awpP\xa0\x018b\xfcfH}\xcau'
...: mac2: b'\xd5\x8b\xd5+L\xedT7N\xa5\xba\xca\x0bj\xa5+\x0e\x03\xaft'
...: .-.-.-.-.-.-.
...: SAVES THE SESSION      OK!
...: DERIVE KEY:
...: Salt: b'cookie-session'
...: Key: b'testing'
...: mac1: b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
...: mac2: b'\xd5\x8b\xd5+L\xedT7N\xa5\xba\xca\x0bj\xa5+\x0e\x03\xaft'
...: .-.-.-.-.-.-.
...: OPEN THE SESSION       OK!
...: DERIVE KEY:
...: Salt: b'cookie-session'
...: Key: b'testing'
...: mac1: b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
...: mac2: b'\xd5\x8b\xd5+L\xedT7N\xa5\xba\xca\x0bj\xa5+\x0e\x03\xaft'
...: .-.-.-.-.-.-.
...: SAVES THE SESSION      OK!
...: DERIVE KEY:
...: Salt: b'cookie-session'
...: Key: b'testing'
...: mac1: b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
...: mac2: b'\xd5\x8b\xd5+L\xedT7N\xa5\xba\xca\x0bj\xa5+\x0e\x03\xaft'
...: .-.-.-.-.-.-.
...: OPEN THE SESSION       NOT OKAY!!
...: DERIVE KEY:
...: Salt: b'cookie-session'
...: Key: b'testing'
...: mac1: b'=\xcc\x01\xee"\x0ed\xde\xf4z\run\rMm\x98\xcb\x0e\xba'
...: mac2: b'\xd5\x8b\xd5+L\xedT7N\xa5\xba\xca\x0bj\xa5+\x0e\x03\xaft'
...: .-.-.-.-.-.-.
...: SAVES THE SESSION      OK!
...: DERIVE KEY:
...: Salt: b'cookie-session'
...: Key: b'testing'
...: mac1: b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
...: mac2: b'\xd5\x8b\xd5+L\xedT7N\xa5\xba\xca\x0bj\xa5+\x0e\x03\xaft'
...: .-.-.-.-.-.-.
...: OPEN THE SESSION       OK!
...: DERIVE KEY:
...: Salt: b'cookie-session'
...: Key: b'testing'
...: mac1: b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
...: mac2: b'\xd5\x8b\xd5+L\xedT7N\xa5\xba\xca\x0bj\xa5+\x0e\x03\xaft'
...: .-.-.-.-.-.-.
...: SAVES THE SESSION      OK!
...: DERIVE KEY:
...: Salt: b'cookie-session'
...: Key: b'testing'
...: mac1: b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
...: mac2: b'\xd5\x8b\xd5+L\xedT7N\xa5\xba\xca\x0bj\xa5+\x0e\x03\xaft'
...: .-.-.-.-.-.-.

我添加了 "OK!" 和 "NOT OKAY!" 以便您更容易看到哪些执行出错了。

(我也看到,在这个例子中,不是 60% 失败。)

在上面的输出中,只有 "Open session" 事件失败了。所以我又试了一次,看看它是否只在保存会话时发生,但它也发生在保存会话中..

...: SAVES THE SESSION      NOT OK!   
...: DERIVE KEY:
...: Salt: b'cookie-session'
...: Key: b'testing'
...: mac1: b'\xc8D\xf0\x95\xc5R\x9f\xe3n\xc7\xa2 `7\xa9\xdb\xdd\xd8F\x85'
...: mac2: b'\xc8D\xf0\x95\xc5R\x9f\xe3n\xc7\xa2 `7\xa9\xdb\xdd\xd8F\x85'

我终于找到了一个重现错误的小示例程序。

#!/usr/bin/env python

async_mode = "eventlet"

if async_mode is None:
    try:
        import eventlet
        async_mode = 'eventlet'
    except ImportError:
        pass

    if async_mode is None:
        try:
            from gevent import monkey
            async_mode = 'gevent'
        except ImportError:
            pass

    if async_mode is None:
        async_mode = 'threading'

    print('async_mode is ' + async_mode)

if async_mode == 'eventlet':
    import eventlet
    eventlet.monkey_patch()
elif async_mode == 'gevent':
    from gevent import monkey
    monkey.patch_all()

import hmac
import hashlib
import time
from threading import Thread
thread = None


def background_thread():
    time.sleep(0.5)
    error_mac = ""
    while True:
        error_mac = ""
        time.sleep(0.1)
        counter = 0
        for i in range(0, 40):
            time.sleep(0.001)
            mac = hmac.new(b'testing', digestmod=hashlib.sha1).digest() # == b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b':

            if not mac ==  b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b':
                counter = counter + 1
                error_mac = mac
        if error_mac:
            print("Example of the wrong hmacs calculated:")
            print(error_mac)
            print("--------------------------------------")
        print("{} - {}".format(time.time(), counter))

def index():
    global thread
    if thread is None:
        thread = Thread(target=background_thread)
        thread.daemon = True
        thread.start()

    for i in range(0,40):
        print(hmac.new(b'testing', digestmod=hashlib.sha1).digest())
    thread.join()
    return "ok"


if __name__ == '__main__':
    index()

通常如果前 20 个哈希值(由主线程创建)有误,第二个线程也会得到错误的哈希值。如果主线程没有报错,而第二线程只输出timestsamp和zeros,则重启程序。

它 async_mode 设置为 "threading" 一切正常。但是当设置为 "gevent" 或 "eventlet" 时会出现此错误。

错误输出:

b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
b'D\xb4V\r9$gy\xe1 \x13\xd8\xc4f\x93O\x9e\xfa\x02\xff'
Example of the wrong hmacs calculated:
b"\x19\xd2}YU\xfeyX\x87\xee\xf5\x96\x94\xc1'\xa3tP\xb3\x96"
--------------------------------------
1463462121.3955774 - 40
Example of the wrong hmacs calculated:
b"\x19\xd2}YU\xfeyX\x87\xee\xf5\x96\x94\xc1'\xa3tP\xb3\x96"
--------------------------------------
1463462121.6040413 - 40
Example of the wrong hmacs calculated:
b"\x19\xd2}YU\xfeyX\x87\xee\xf5\x96\x94\xc1'\xa3tP\xb3\x96"
--------------------------------------
1463462121.8342147 - 40
Example of the wrong hmacs calculated:
b"\x19\xd2}YU\xfeyX\x87\xee\xf5\x96\x94\xc1'\xa3tP\xb3\x96"

没有错误的输出:

b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
b'j\xb6\xfc\x89\x1e\xef\xd3\xb7\x87C\xea(\xb1\x808\x11V\x1a|\x9b'
1463462453.3856905 - 0
1463462453.5910842 - 0
1463462453.8242626 - 0
1463462454.0677884 - 0
1463462454.2900438 - 0
1463462454.5460255 - 0
1463462454.7883186 - 0

(在我的 ubuntu 机器上,这个例子运行得很好。只有在 ARM7 设备上我们才会遇到这个问题..)

当我将参数更改为 gevetn patch all 时:

...
elif async_mode == 'gevent':
    from gevent import monkey
    monkey.patch_all(ssl=False)
...

这似乎有效。