cProfiler 奇怪地与多处理一起工作

Question

我收到此代码的错误：

from pathos.multiprocessing import ProcessingPool
def diePlz(im):
    print('Whoopdepoop!')   
def caller():
    im = 1
    pool = ProcessingPool()
    pool.map(diePlz,[im,im,im,im]) 

if __name__=='__main__':
    caller()

当我运行它与 cProfiler: (python3 -m cProfile testProfiler.py)

multiprocess.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/rohit/.local/lib/python3.6/site-packages/multiprocess/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/home/rohit/.local/lib/python3.6/site-packages/multiprocess/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/home/rohit/.local/lib/python3.6/site-packages/pathos/helpers/mp_helper.py", line 15, in <lambda>
    func = lambda args: f(*args)
  File "testProfiler.py", line 3, in diePlz
    print('Whoopdepoop!')
NameError: name 'print' is not defined
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.6/cProfile.py", line 160, in <module>
    main()
  File "/usr/lib/python3.6/cProfile.py", line 153, in main
    runctx(code, globs, None, options.outfile, options.sort)
  File "/usr/lib/python3.6/cProfile.py", line 20, in runctx
    filename, sort)
  File "/usr/lib/python3.6/profile.py", line 64, in runctx
    prof.runctx(statement, globals, locals)
  File "/usr/lib/python3.6/cProfile.py", line 100, in runctx
    exec(cmd, globals, locals)
  File "testProfiler.py", line 11, in <module>
    caller()    
  File "testProfiler.py", line 8, in caller
    pool.map(diePlz,[im,im,im,im]) 
  File "/home/rohit/.local/lib/python3.6/site-packages/pathos/multiprocessing.py", line 137, in map
    return _pool.map(star(f), zip(*args)) # chunksize
  File "/home/rohit/.local/lib/python3.6/site-packages/multiprocess/pool.py", line 266, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/home/rohit/.local/lib/python3.6/site-packages/multiprocess/pool.py", line 644, in get
    raise self._value
NameError: name 'print' is not defined

但是当我运行它没有 cProfiler:

$ python3 testProfiler.py 
Whoopdepoop!
Whoopdepoop!
Whoopdepoop!
Whoopdepoop!

我提供的代码是该问题的最小工作示例。我想调试一个更大的代码，但我无法这样做，因为 cProfiler 不断出现奇怪的错误。

在这种情况下，重点是

NameError: name 'print' is not defined

这意味着 python3 无法识别 print 本身。在我的代码中，它无法识别 range.

Answer 1

所以，我意识到这是在最初 post 之后很长一段时间，但我遇到了完全相同的问题。

在我的例子中，我遇到了与原始 post 完全相同的错误 - python 内置函数（例如 print() 或 len() 导致了如下错误：

NameError: name 'len' is not defined

我目前运行 multiprocess 版本 0.70.11.1 和 dill 版本 0.3.3（使基于进程的并行性工作的 pathos 组件）。

根据我在问题评论中发现的内容：https://github.com/uqfoundation/pathos/issues/129#issuecomment-536081859 其中一位软件包作者建议尝试：

import dill
dill.settings['recurse'] = True

至少在我的情况下，以上修复了错误！

cProfiler 奇怪地与多处理一起工作

cProfiler working weirdly with multiprocessing

python

multiprocessing

profiler

pathos