如何用 concurrent.futures 对 python 函数进行基准测试？

Question

我想测量可以并行调用的服务 (API) 的运行时间。基准测试和使用 concurrent.futures (multiprocessing/multithreading) 的正确方法是什么？

我们可以 submit() 任务，然后一旦任务完成，我们可以用 concurrent.futures.as_completed() 检查它 - 我们是否需要 as_completed() 部分，因为那应该 return结果对我们来说，submit() 刚开始任务？

根据我的伪代码，它看起来不需要，但为什么不需要 as_completed()？（我认为 submit() 将任务放入队列中，不等待响应，使用 as_completed() 你会得到完成的任务和结果。

我们来调用服务 my_job():

import concurrent.futures
import time
from typing import List


def _my_job(t: int):
    time.sleep(t)


def measure(N_requ: int, workers: int, wait_time: int, include_as_completed: bool):
    futures = []
    start_time = time.time()
    with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as executor:
        for _ in range(N_requ):
            f = executor.submit(_my_job, wait_time)
            futures.append(f)
        if include_as_completed:
            for f in concurrent.futures.as_completed(futures):
                try:
                    _ = f.result()
                except:
                    print("This should not happen...")
                    pass
    end_time = time.time()
    elapsed_time: float = end_time - start_time
    print(f"Elpsed time: {elapsed_time} | n_workers: {workers}, n_requs: {N_requ}, job's wait_time: {wait_time}")


if __name__ == "__main__":
    N = 20
    n_workers: List[int] = [1, 4, 16, 32]
    print("Without as_completed:")
    for w in n_workers:
        measure(N, workers=w, wait_time=1, include_as_completed=False)
    print("With as_completed:")
    for w in n_workers:
        measure(N, workers=w, wait_time=1, include_as_completed=True)

结果：

Without as_completed:
Elpsed time: 20.05830979347229 | n_workers: 1, n_requs: 20, job's wait_time: 1
Elpsed time: 5.012893199920654 | n_workers: 4, n_requs: 20, job's wait_time: 1
Elpsed time: 2.0075199604034424 | n_workers: 16, n_requs: 20, job's wait_time: 1
Elpsed time: 1.0035851001739502 | n_workers: 32, n_requs: 20, job's wait_time: 1

With as_completed:
Elpsed time: 20.044990062713623 | n_workers: 1, n_requs: 20, job's wait_time: 1
Elpsed time: 5.009936809539795 | n_workers: 4, n_requs: 20, job's wait_time: 1
Elpsed time: 2.0069048404693604 | n_workers: 16, n_requs: 20, job's wait_time: 1
Elpsed time: 1.0069079399108887 | n_workers: 32, n_requs: 20, job's wait_time: 1

Answer 1

您正在将执行程序用作上下文管理器。在 with 块结束时，您自动关闭执行器，等待所有未完成的期货完成。

查看 Executor.shutdown 的文档，尤其是关于 with 行为的部分。

如何用 concurrent.futures 对 python 函数进行基准测试？

How to benchmark python function with concurrent.futures?

python

multithreading

multiprocessing

concurrent.futures