为什么 concurrent.futures executor map 在所有期货完成后与 futures.as_completed 一起使用时会抛出错误？

Question

我正在尝试并发发送 HTTP 请求。为此，我使用 concurrent.futures

这里是简单的代码：

import requests
from concurrent import futures

data = range(10)


def send_request(item):
    requests.get("https://httpbin.org/ip")
    print("Request {} complete.".format(item))


executor = futures.ThreadPoolExecutor(max_workers=25)

futures_ = executor.map(send_request, data)

for f in futures.as_completed(futures_):
    f.result()

如果我运行它，我可以看到请求是异步发送的，这正是我想要做的。但是，当所有请求都完成后，出现以下错误：

Request 0 complete.
Request 6 complete.
...
Request 7 complete.
Request 9 complete.
Request 3 complete.
Traceback (most recent call last):
  File "send_thread.py", line 18, in <module>
    for f in futures.as_completed(futures_):
  File "/usr/local/Cellar/python3/3.6.4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/concurrent/futures/_base.py", line 219, in as_completed
    with _AcquireFutures(fs):
  File "/usr/local/Cellar/python3/3.6.4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/concurrent/futures/_base.py", line 146, in __enter__
    future._condition.acquire()
AttributeError: 'NoneType' object has no attribute '_condition'

这是一个很奇怪的错误。这里executor.map好像有问题。如果我用以下行替换地图，它会按预期工作。

futures_ = [executor.submit(send_request, x) for x in data]

我错过了什么？试图找出两者之间的区别，但似乎无法理解导致上述问题的原因。任何输入将不胜感激。

Answer 1

Executor.map 不是 return 你的期货清单，而是结果的生成器，所以不是：

futures_ = executor.map(send_request, data)
for f in futures.as_completed(futures_):
    f.result()

你应该运行:

results = executor.map(send_request, data)
for r in results:
    print(r)

为什么 concurrent.futures executor map 在所有期货完成后与 futures.as_completed 一起使用时会抛出错误？

Why does concurrent.futures executor map throw error when using with futures.as_completed after all the futures are complete?

python

concurrency

python-multithreading

python-3.x