如何在 volttron 中使用 gevent 和请求进行并发 jsonrpc

Question

我正在尝试对 Volttron 中的远程代理执行并发 JSONRPC 调用。我正在使用 Volttron 5.1.0（使用 gevent 1.1.2 和请求 2.11.1）。

代码按预期运行。但是，从日志文件中，我注意到请求不是运行同时发生的。我不确定我错过了什么。

我尝试了 how enable requests async mode? 中提到的建议（即处理请求中的异步部分、猴子补丁等）。但是 none 解决了，要么解决方案已经过时，要么所需的模块在 volttron env 中不可用（我有点担心版本要求）。

任何建议或意见都会有很大帮助。

代理代码相关部分如下：

代理代码

import gevent
import requests

url_roots = ['http://192.168.1.51:8080/', 'http://192.168.1.52:8080/']
jobs = [gevent.spawn(do_rpc, self._agent_id, url_root, 'pricepoint'
                                , get_json_params()
                                ) for url_root in url_roots
                                ]
gevent.joinall(jobs, timeout=11)


def do_rpc(id, url_root, method, params=None):
    result = False
    json_package = {
        'jsonrpc': '2.0',
        'id': id,
        'method':method,
    }
    json_package['params'] = params
    response = requests.post(url_root, data=json.dumps(json_package), timeout=10)
    if response.ok:
        if 'result' in response.json().keys():
            success = response.json()['result']
            if success:
                result = True
    return result

日志

2020-02-19 21:12:15,913 (xyzagent-0.4 28079) xyz.ispace_msg_utils DEBUG: validate_bustopic_msg()
2020-02-19 21:12:15,918 (xyzagent-0.4 28079) xyz.agent DEBUG: New price point (pp) msg on the local-bus, topic: building/pricepoint ...
2020-02-19 21:12:15,919 (xyzagent-0.4 28079) xyz.agent DEBUG: ***** New bid price point from local: 0.20 price_id: 2218566
2020-02-19 21:12:15,931 (xyzagent-0.4 28079) xyz.agent DEBUG: post_ds_new_pp()...
2020-02-19 21:12:15,932 (xyzagent-0.4 28079) xyz.agent DEBUG: us pp messages count: 1...
2020-02-19 21:12:15,933 (xyzagent-0.4 28079) xyz.agent DEBUG: processing pp msg 1/1, price id: 2218566
2020-02-19 21:12:15,938 (xyzagent-0.4 28079) xyz.agent DEBUG: new ttl: 28.
2020-02-19 21:12:15,942 (xyzagent-0.4 28079) xyz.agent DEBUG: _ds_rpc_1_to_m()...
2020-02-19 21:12:15,953 (xyzagent-0.4 28079) requests.packages.urllib3.connectionpool INFO: Starting new HTTP connection (1): 192.168.1.51
2020-02-19 21:12:16,079 () volttron.platform.web DEBUG: {'jsonrpc': '2.0', 'id': '2503402', 'result': True}
2020-02-19 21:12:16,080 () volttron.platform.web DEBUG: res is a dictionary.
2020-02-19 21:12:16,238 (xyzagent-0.4 28079) requests.packages.urllib3.connectionpool DEBUG: "POST /bridge HTTP/1.1" 200 53
2020-02-19 21:12:16,245 (xyzagent-0.4 28079) requests.packages.urllib3.connectionpool INFO: Starting new HTTP connection (1): 192.168.1.52
2020-02-19 21:12:16,526 (xyzagent-0.4 28079) requests.packages.urllib3.connectionpool DEBUG: "POST /bridge HTTP/1.1" 200 53
2020-02-19 21:12:16,529 (xyzagent-0.4 28079) xyz.agent DEBUG: post pp to ds (ZoneController-51), result: success!!!
2020-02-19 21:12:16,529 (xyzagent-0.4 28079) xyz.agent DEBUG: post pp to ds (ZoneController-52), result: success!!!
2020-02-19 21:12:16,530 (xyzagent-0.4 28079) xyz.agent DEBUG: _ds_rpc_1_to_m()...done
2020-02-19 21:12:16,530 (xyzagent-0.4 28079) xyz.agent DEBUG: msg successfully posted to all ds, removing it from the queue
2020-02-19 21:12:16,530 (xyzagent-0.4 28079) xyz.agent DEBUG: reset the retry counter for success ds msg
2020-02-19 21:12:16,530 (xyzagent-0.4 28079) xyz.agent DEBUG: post_ds_new_pp()...done

Answer 1

以下更改起到了作用。请注意，只有 monkey.patch_all() 会破坏 volttron。需要设置标志 thread=False.

代理代码

import gevent
from gevent import monkey
monkey.patch_all(thread=False, select=False)
import requests

日志

2020-02-20 14:36:29,981 (xyzagent-0.4 8657) xyz.agent DEBUG: _ds_rpc_1_to_m()...
2020-02-20 14:36:29,987 (xyzagent-0.4 8657) requests.packages.urllib3.connectionpool INFO: Starting new HTTP connection (1): 192.168.1.51
2020-02-20 14:36:29,992 (xyzagent-0.4 8657) requests.packages.urllib3.connectionpool INFO: Starting new HTTP connection (1): 192.168.1.52
2020-02-20 14:36:30,260 (xyzagent-0.4 8657) requests.packages.urllib3.connectionpool DEBUG: "POST /bridge HTTP/1.1" 200 53
2020-02-20 14:36:30,333 (xyzagent-0.4 8657) requests.packages.urllib3.connectionpool DEBUG: "POST /bridge HTTP/1.1" 200 53
2020-02-20 14:36:30,341 (xyzagent-0.4 8657) xyz.agent DEBUG: post pp to ds (ZoneController-51), result: success!!!
2020-02-20 14:36:30,342 (xyzagent-0.4 8657) xyz.agent DEBUG: post pp to ds (ZoneController-52), result: success!!!
2020-02-20 14:36:30,344 (xyzagent-0.4 8657) xyz.agent DEBUG: _ds_rpc_1_to_m()...done

如何在 volttron 中使用 gevent 和请求进行并发 jsonrpc

How to do concurrent jsonrpc using gevent and requests in volttron

volttron

代理代码

日志

代理代码

日志