AttributeError: Can't get attribute 'starfield_solve' on <module '__main__' (built-in)>

Question

我看过类似的帖子有同样的问题。我尝试实施其他建议的有效解决方案，但不知何故它对我不起作用。这是我在 jupyter notebook 中 运行 的代码的简短示例：

def starfield_solve(file,verify):
    do_something

def apply_async_callback(src_path, verify):
    with Pool(processes=PROCESSES) as pool:
        multiple_results = [pool.apply_async(
            starfield_solve, args=(file, verify)) for file in files]

if __name__ == "__main__" :
    apply_async_callback(src_path, verify)

编辑： 我在最小的可重现示例下面。 进一步注意，这个确切的代码适用于 linux，但它 doesn/t 适用于 MacOS Monterey 12.1

import multiprocessing as mp
def starfield_solve(num):
    print(num)
    return num

def apply_async_callback():
    temp_dir = [1,2,3]
    PROCESSES = mp.cpu_count() - 1
    with mp.Pool(processes=PROCESSES) as pool:
        multiple_results = [pool.apply_async(starfield_solve, args=(num,)) for num in temp_dir]
        output = [res.get() for res in multiple_results if res.get()]

if __name__ == "__main__" :
    apply_async_callback()

这是完整的错误：

Process SpawnPoolWorker-20:
Traceback (most recent call last):
  File "/opt/miniconda3/envs/astrometry/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/opt/miniconda3/envs/astrometry/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/miniconda3/envs/astrometry/lib/python3.9/multiprocessing/pool.py", line 114, in worker
    task = get()
  File "/opt/miniconda3/envs/astrometry/lib/python3.9/multiprocessing/queues.py", line 368, in get
    return _ForkingPickler.loads(res)
AttributeError: Can't get attribute 'starfield_solve' on <module '__main__' (built-in)>

Answer 1

这似乎是 IPython（读取为 Jupyter Notebook）的限制，不完全支持多处理（relevant GitHub issue on IPython's repository). Use this solution, based on this GitHub issue：

1. 将您的目标函数 (starfield_solve) 放入 *.py 文件
2. 从文件导入函数
3. 使用该函数 apply_async_callback

因此，第 1 步：

# your_module.py
def starfield_solve(file,verify):
    do_something

然后，在您的 Jupyter Notebook 中，应用第 2 步和第 3 步：

from your_module import starfield_solve


def apply_async_callback(src_path, verify):
    with Pool(processes=PROCESSES) as pool:
        multiple_results = [pool.apply_async(
            starfield_solve, args=(file, verify)) for file in files]