如何使用池多处理调用另一个函数内的函数？

Question

我需要运行相同的函数 10 次，因为数据链接到登录，它需要在另一个函数中：

from multiprocessing import Pool

def main():
    def inside(a):
        print(a)
    Pool.map(inside, 'Ok' * 10)

if __name__ == '__main__':
    main()

from multiprocessing import Pool

def main():
    def inside(a):
        print(a)
    Pool.map(main.inside, 'Ok' * 10)

if __name__ == '__main__':
    main()

两次尝试的结果都是这样的：

AttributeError: 'function' object has no attribute 'map'

如何通过将函数保留在另一个函数中来做到这一点？
有办法吗？

Answer 1

AttributeError: 'function' object has no attribute 'map'

我们需要从 multiprocessing 实例化 Pool 并调用该池对象的 map 方法。

您必须将 inside 方法移动到某些 class 因为 Pool 使用 pickel 来 serialize 和 deserialize方法，如果它在某个方法中，则它不能被 pickel.

导入

Pool needs to pickle (serialize) everything it sends to its worker-processes (IPC). Pickling actually only saves the name of a function and unpickling requires re-importing the function by name. For that to work, the function needs to be defined at the top-level, nested functions won't be importable by the child and already trying to pickle them raises an exception (more).

请访问 SO 的这个。

from multiprocessing import Pool

class Wrap:
    def inside(self, a):
        print(a)

def main():
    pool = Pool() 
    pool.map(Wrap().inside, 'Ok' * 10)

if __name__ == '__main__':
    main()

如果您不想在 class 中包装 inside 方法，请将 inside 方法移动到全局范围，这样它就可以 pickled

from multiprocessing import Pool

def inside(a):
    print(a)

def main():
    with Pool() as pool:
        pool.map(inside, 'Ok'*10)

if __name__ == '__main__':
    main()

如何使用池多处理调用另一个函数内的函数？

How to call a function that is inside another function using pool multiprocessing?

python

python-multiprocessing