Python3 Concurrent.Futures 有请求

Question

我正在尝试实施并发请求以加快检查 URL 的列表，但它似乎无法使用我的代码，因为它仍在逐一检查它们。

for domain in list:
    try:
        follow_url = requests.head(f'http://{domain}', allow_redirects=True, timeout=60)

        with concurrent.futures.ThreadPoolExecutor(max_workers=20) as executor:
            executor.submit(follow_url)

        with open("alive.txt", "a") as file:
            file.write(f'{domain}\n')

    except Exception as e:
        print(e)

Answer 1

您没有正确应用它。您正在迭代中创建并行流程。正确的方法可能是这样的：

def parallel_req(domain):
    try:
        follow_url = requests.head(f'http://{domain}', allow_redirects=True, timeout=60)
        with open("alive.txt", "a") as file:
            file.write(f'{domain}\n')
    except requests.exceptions.RequestException as e:
        print(e)

with ThreadPoolExecutor() as e:
    e.map(parallel_req, domains)

Python3 Concurrent.Futures 有请求

Python3 Concurrent.Futures with Requests

python-3.x

concurrent.futures