Python：等待 requests_futures.sessions 完成后再继续代码流

Question

我目前的代码打印了一个空列表，我该如何等待所有请求和回调完成才能继续代码流？

from requests_futures.sessions import FuturesSession
from time import sleep

session = FuturesSession(max_workers=100)

i = 1884001540 - 100
list = []
def testas(session, resp):
    print(resp)
    resp = resp.json()
    print(resp['participants'][0]['stats']['kills'])
    list.append(resp['participants'][0]['stats']['kills'])
while i < 1884001540:
    url = "https://acs.leagueoflegends.com/v1/stats/game/NA1/" + str(i)
    temp = session.get(url, background_callback=testas)
    i += 1
print(list)

Answer 1

查看 requests-futures-0.9 中的 session.py。5.tar.gz 必须创建一个 future 才能等待其结果，如以下代码所示：

from requests_futures import FuturesSession

session = FuturesSession()
# request is run in the background
future = session.get('http://httpbin.org/get')
# ... do other stuff ...
# wait for the request to complete, if it hasn't already
response = future.result()
print('response status: {0}'.format(response.status_code))
print(response.content)

如 README.rst 所示，未来可以而且应该为每个 session.get() 创建并等待完成。

这可能会在您的代码中应用，如下所示，从 while 循环之前开始：

future = []
while i < 1884001540:
    url = "https://acs.leagueoflegends.com/v1/stats/game/NA1/" + str(i)
    future.append(session.get(url, background_callback=testas)
    i += 1
for f in future:
    response = f.result()
    # the following print statements may be useful for debugging
    # print('response status: {0}'.format(response.status_code))
    # print(response.content, "\n")
print(list)

我不确定您的系统将如何响应大量 (1884001440) 期货，另一种方法是将它们以较小的组进行处理，比如一次处理 100 或 1000 个。在开始时使用相对较少的脚本来测试脚本以了解它们 return 结果的速度可能是明智的。

Answer 2

从这里 https://pypi.python.org/pypi/requests-futures 它说

from requests_futures.sessions import FuturesSession

session = FuturesSession()
# first request is started in background
future_one = session.get('http://httpbin.org/get')
# second requests is started immediately
future_two = session.get('http://httpbin.org/get?foo=bar')
# wait for the first request to complete, if it hasn't already
response_one = future_one.result()

看来 .result() 就是您要找的东西

Python：等待 requests_futures.sessions 完成后再继续代码流

Python: wait for requests_futures.sessions to finish before continuing with the code flow

python

python-requests