从多个 uWSGI worker returns 不匹配的行中查询 MySQL

Question

我正在运行从 Flask 应用程序运行与 uWSGI 和多个工作人员一起查询 MySQL 数据库。我注意到有时当我通过 id 查询资源时，返回行的 id 与我查询的不同。

我认为查询隔离意味着这是不可能的。但是，MySQL 似乎混淆了查询。当我不使用 uWSGI 时我无法重现这个，但这可能只是因为它运行在本地主机而不是服务器上测试 Flask 服务器本身。

为什么输入id和结果id不匹配？

from flask import Flask
import pymysql.cursor, random

class Database:
    def __init__(self, user, password, host, database):
        self.connection = pymysql.connect(
            user=user,
            password=password,
            host=host,
            database=database,
            cursorclass=pymysql.cursors.DictCursor
        )

    def query(self, sql, **kwargs):
        with self.connection.cursor() as cursor:
            cursor.execute(sql, kwargs)
            return cursor

app = Flask(__name__)
database = Database('user', 'password', 'localhost', 'database')

@app.route('/resources/<path:id>')
def resource(id):
    item = database.query(
        'SELECT resources.id FROM resources WHERE resources.id = %(id)s',
        id=id
    ).fetchone()

    identifier = random.random()
    print(identifier, 'ID 1:', id)
    print(identifier, 'ID 2:', item['id'])

    if int(item['id']) != int(id):
        print('Error found!!!')

    return 'Done', 200

if __name__ == '__main__':
    app.run()

[pid: 2824|app: 0|req: 1/1] xxx.xxx.xxx.xxx () {44 vars in 737 bytes} [Wed Oct 19 18:38:07 2016] GET /resources/10 => generated 4 bytes in 6 msecs (HTTP/1.1 200) 2 headers in 78 bytes (1 switches on core 0)
0.687535338604848 ID 1: 11
0.687535338604848 ID 2: 11
[pid: 2821|app: 0|req: 1/2] xxx.xxx.xxx.xxx () {44 vars in 737 bytes} [Wed Oct 19 18:38:07 2016] GET /resources/11 => generated 4 bytes in 5 msecs (HTTP/1.1 200) 2 headers in 78 bytes (1 switches on core 0)
0.9216930740141296 ID 1: 13
0.9216930740141296 ID 2: 13
[pid: 2823|app: 0|req: 1/3] xxx.xxx.xxx.xxx () {44 vars in 737 bytes} [Wed Oct 19 18:38:07 2016] GET /resources/13 => generated 4 bytes in 6 msecs (HTTP/1.1 200) 2 headers in 78 bytes (1 switches on core 0)
0.9053128320497649 ID 1: 12
0.9053128320497649 ID 2: 14
Error found!!!
0.794023616025622 ID 1: 15
0.794023616025622 ID 2: 15
[pid: 2824|app: 0|req: 2/4] xxx.xxx.xxx.xxx () {44 vars in 737 bytes} [Wed Oct 19 18:38:07 2016] GET /resources/15 => generated 4 bytes in 1 msecs (HTTP/1.1 200) 2 headers in 78 bytes (1 switches on core 0)
[pid: 2822|app: 0|req: 1/5] xxx.xxx.xxx.xxx () {44 vars in 737 bytes} [Wed Oct 19 18:38:07 2016] GET /resources/12 => generated 4 bytes in 31 msecs (HTTP/1.1 200) 2 headers in 78 bytes (1 switches on core 0)
0.3608322871408709 ID 1: 14
0.3608322871408709 ID 2: 16
Error found!!!
[pid: 2825|app: 0|req: 1/6] xxx.xxx.xxx.xxx () {44 vars in 737 bytes} [Wed Oct 19 18:38:07 2016] GET /resources/14 => generated 4 bytes in 18 msecs (HTTP/1.1 200) 2 headers in 78 bytes (1 switches on core 0)
0.8346421078513786 ID 1: 16
0.8346421078513786 ID 2: 17
Error found!!!

Answer 1

对于遇到此问题的其他人，我找到了以下解决方案。

根据http://uwsgi-docs.readthedocs.io/en/latest/ThingsToKnow.html。

uWSGI tries to (ab)use the Copy On Write semantics of the fork() call whenever possible. By default it will fork after having loaded your applications to share as much of their memory as possible. If this behavior is undesirable for some reason, use the lazy-apps option. This will instruct uWSGI to load the applications after each worker’s fork().

查看 uWSGI, Flask, sqlalchemy, and postgres: SSL error: decryption failed or bad record mac 后，我意识到我的问题与创建多个进程有关。

然而，因为 uWSGI 默认从一个 master worker 加载所有进程（而不是每次运行整个 Flask 应用程序），结果是所有 worker 最终共享一个数据库连接（结局不好！）。

解决方案是包含 lazy-apps 参数，这会在创建每个 worker 时强制所有代码为运行。

从多个 uWSGI worker returns 不匹配的行中查询 MySQL

Querying MySQL from multiple uWSGI workers returns mismatched rows

python

mysql

flask

uwsgi

pymysql