pathos.multiprocessing有星图吗？

Question

我在执行下面的代码时出错。问题似乎是 map 不支持接受多个输入的函数，就像在 python 内置 multiprocessing 包中一样。但是在内置包中，有一个 starmap 可以解决这个问题。 pathos.multiprocessing有相同的吗？

import pathos.multiprocessing as mp


class Bar:
    def foo(self, name):
        return len(str(name))

    def boo(self, x, y, z):
        sum = self.foo(x)
        sum += self.foo(y)
        sum += self.foo(z)
        return sum


if __name__ == '__main__':
    b = Bar()
    pool = mp.ProcessingPool()
    results = pool.map(b.boo, [(12, 3, 456), (8, 9, 10), ('a', 'b', 'cde')])
    print(results)

TypeError: boo() missing 2 required positional arguments: 'y' and 'z'

按照建议更新 lambda 表达式（无效）：

if __name__ == '__main__':
    b = Bar()
    pool = mp.ProcessingPool()
    results = pool.map(lambda x: b.boo(*x), [(12, 3, 456), (8, 9, 10), ('a', 'b', 'cde')])
    print(results)

multiprocess.pool.RemoteTraceback:

"""

Traceback (most recent call last):

File "C:\Users\yg451\Anaconda3\lib\site-packages\multiprocess\pool.py", line 121, in worker

result = (True, func(*args, **kwds))

File "C:\Users\yg451\Anaconda3\lib\site-packages\multiprocess\pool.py", line 44, in mapstar

return list(map(*args))

File "C:\Users\yg451\Anaconda3\lib\site-packages\pathos\helpers\mp_helper.py", line 15, in

func = lambda args: f(*args)

File "C:/Users/yg451/Code/foo/Machine Learning/xPype/test/scratch.py", line 18, in

results = pool.map(lambda x: b.boo(*x), [(12, 3, 456), (8, 9, 10), ('a', 'b', 'cde')])

NameError: name 'b' is not defined

"""

Answer 1

我是 pathos 的作者。 pathos 比 starmap 老，并不真的需要它。它以与内置 map 完全相同的方式解决池中的多个参数。

>>> import pathos.multiprocessing as mp
>>> class Bar:
...     def foo(self, name):
...         return len(str(name))
...     def boo(self, x, y, z):
...         sum = self.foo(x)
...         sum += self.foo(y)
...         sum += self.foo(z)
...         return sum
... 
>>> b = Bar()
>>> pool = mp.ProcessingPool()
>>> f = lambda x: b.boo(*x)    
>>> results = pool.map(f, [(12, 3, 456), (8, 9, 10), ('a', 'b', 'cde')])
>>> results 
[6, 4, 5]
>>> results = pool.map(b.boo, [12, 9, 'a'], [3, 9, 'b'], [456, 10, 'cde'])
>>> results
[6, 4, 5]
>>> results = map(b.boo, [12, 9, 'a'], [3, 9, 'b'], [456, 10, 'cde'])
>>> list(results)
[6, 4, 5]
>>>

所以，本质上，starmap 是不必要的。然而，由于它最近被添加到 python 的某些版本的 multiprocessing 中的标准 Pool 界面中，它可能应该在 pathos 中更加突出。请注意，如果您愿意，已经可以从 pathos 获得 "augmented" 版本的 starmap。

>>> import pathos
>>> mp = pathos.helpers.mp
>>> p = mp.Pool()
>>> p.starmap
<bound method Pool.starmap of <multiprocess.pool.Pool object at 0x1038684e0>>
>>>

pathos.multiprocessing有星图吗？

Does pathos.multiprocessing have starmap?

multiprocessing

python-3.x

pathos