具有异步工作者的多处理池

Question

考虑以下代码：

class Sandbox:
    def __init__(self):
        self.pool = Pool(4)

    def worker(self, x):
        print(x) # Should be printing "testing123"

    def run(self):
        res = self.pool.apply_async(self.worker, ("testing123",))
        print(res.get()) # NotImplementedError
        self.pool.close()
        self.pool.join()


sandbox = Sandbox()
sandbox.run()

当我运行这个时，我得到

NotImplementedError: pool objects cannot be passed between processes or pickled

res.get() 显示异常 - 如果我删除它，则不会发生任何事情。

Answer 1

async_apply获取状态的同时也设置了状态，所以需要修改那些方法，比如：

from multiprocessing import Pool
class Sandbox:
    def __init__(self):
        self.pool = Pool(4)

    def worker(self, x):
        print(x) # Should be printing "testing123"

    def run(self):
        res = self.pool.apply_async(self.worker, ("testing123",))
        print(res.get()) # NotImplementedError
        self.pool.close()
        self.pool.join()

    def __getstate__(self):
        self_dict = self.__dict__.copy()
        del self_dict['pool']
        return self_dict

    def __setstate__(self, state):
        self.__dict__.update(state)


sandbox = Sandbox()
sandbox.run()

具有异步工作者的多处理池

Multiprocessing pool with async workers

python

multithreading

pool