uwsgi 下的 flask 应用程序 - 看不到其他工作人员的 memcached 条目

Question

所以我启动了 20 个 uswgi 进程，我正在使用 Nginx 对它们进行负载平衡，例如 detailed here

我注意到的是，当一名工作人员被击中时，它会在 memcached 中看不到它需要的结果后加载 memcached。此后，工作人员将在 memcached 中找到它们。但是其他工作人员在自己加载之前仍然看不到 memcached 结果。

知道会发生什么吗？这是相关代码：

import memcache
import msgpack
from flask import Flask

app = Flask(__name__, static_url_path = "")
app.config['PROPAGATE_EXCEPTIONS'] = True

class MemCachedWrapper(object):
    """Simple wrapper around memcached to handle translating keys"""
    def make_key(self, key):
        #is this crazy??  memcached only wants strings and won't accept spaces
        return str(key).replace(' ','')
    def __init__(self, servers, debug=0, server_max_value_length=1024*1024*3,
        flush_on_start=False):
        #Set up memcached
        #see README.md for instructions on setting server_max_value_length on memcached (-I option)
        self.cache = memcache.Client(servers, debug=debug,
            server_max_value_length=server_max_value_length)
        if flush_on_start:
            self.cache.flush_all()
    def get(self, key):
        return self.cache.get(self.make_key(key))
    def get_multi(self, keys):
        m = self.make_key
        return self.cache.get_multi([m(k) for k in keys])
    def set(self, key, value):
        return self.cache.set(self.make_key(key), value)
    def delete(self, key):
        return self.cache.delete(self.make_key(key))

cache =MemCachedWrapper([MEMCACHE_URL], debug=0,
        server_max_value_length=MEMCACHE_MAX_VALUE_LENGTH)

def _get_range(key_beg, key_end):
    keys = get_keys(key_beg, key_end)
    def stream_results():
        mc_results = None        
        for scality_key in keys:
            if not mc_results:
                mc_results = cache.get_multi([k[0].encode('utf-8') for k in keys])
            scality_key = scality_key.encode('utf-8')
            obj = mc_results.get(scality_key)
            if obj:
                yield dvid_key, obj
            else:
                print "key miss"
                response = session.get(SCALITY_URL % scality_key)
                cache.set(scality_key, response.content)
                yield dvid_key, response.content
    return stream_results()

@app.route('/api/keyvalue_range/<key_beg>/<key_end>/', methods=['GET'])
def get_range(key_beg, key_end):
    results = _get_range(key_beg, key_end)
    packed = msgpack.packb(list(results), use_bin_type=True)
    return Response(packed, mimetype='application/x-msgpack')


if __name__ == '__main__':
    app.run(host=host, port=PORT, debug=True)

Answer 1

这不是一个很好的答案，但我最终使用 pylibmc 作为我的 memcached 客户端库，它似乎已经解决了这个问题。不知道发生了什么。

至于上面代码的改动，基本上就是这几行：

import pylibmc as memcache
...
self.cache = memcache.Client(servers, binary=True,
   behaviors={"tcp_nodelay": True, "ketama": True, "no_block": True})

uwsgi 下的 flask 应用程序 - 看不到其他工作人员的 memcached 条目

flask app under uwsgi - can't see memcached entries from other workers

python

memcached

nginx

flask

uwsgi