Python Windows 上的 DEAP 和多处理：AttributeError

Question

我有以下情况：

Windows 10
Python 3.7
deap 1.3.1

有一个main.py和

def main():
    ...
    schedule.schedule()
    ...

if __name__== "__main__":
    main()

然后，我还有一个文件schedule.py和

def schedule()
   ...
    toolbox = base.Toolbox()

    creator.create("FitnessMin", base.Fitness, weights=(-1.0,))
    creator.create("Individual", list, fitness=creator.FitnessMin)

    toolbox.register('individual', init_indiv, creator.Individual, bounds=bounds)
    toolbox.register("population", tools.initRepeat, list, toolbox.individual)

    toolbox.register("evaluate", fitness, data=args)
    toolbox.register("mate", tools.cxTwoPoint)
    toolbox.register("mutate", tools.mutFlipBit, indpb=0.05)
    toolbox.register("select", tools.selTournament, tournsize=3)

    # Further parameters
    cxpb = 0.7
    mutpb = 0.2

    # Measure how long it takes to caluclate 1 generation
    MAX_HOURS_GA = parameter._MAX_HOURS_GA
    POPSIZE_GA = parameter._POPSIZE_GA
    pool = multiprocessing.Pool(processes=4)
    toolbox.register("map", pool.map)
    pop = toolbox.population(n=POPSIZE_GA * len(bounds))
    result = algorithms.eaSimple(pop, toolbox, cxpb, mutpb, 1, verbose=False)

现在，执行此操作会出现以下错误：

Process SpawnPoolWorker-1:
Traceback (most recent call last):
  File "C:\Users\...\lib\multiprocessing\process.py", line 297, in _bootstrap
    self.run()
  File "C:\Users\...\lib\multiprocessing\process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\...\lib\multiprocessing\pool.py", line 110, in worker
    task = get()
  File "C:\Users\...\lib\multiprocessing\queues.py", line 354, in get
    return _ForkingPickler.loads(res)
AttributeError: Can't get attribute 'Individual' on <module 'deap.creator' from 'C:\Users...

现在，我确实注意到 DEAP 文档 (https://deap.readthedocs.io/en/master/tutorials/basic/part4.html) 说

Warning As stated in the multiprocessing guidelines, under Windows, a process pool must be protected in a >if __name__ == "__main__" section because of the way processes are initialized.

但这并没有真正帮助我，因为我当然不希望在我的 main 中拥有所有 toolbox.register(...)，甚至可能无法这样做。只是移动池的创建

    pool = multiprocessing.Pool(processes=4)
    toolbox.register("map", pool.map)

对主要没有帮助。

似乎还有其他人也有类似的问题，即使是最近 (https://github.com/rsteca/sklearn-deap/issues/59)。对于其中的大多数，似乎存在某种解决方法，但其中 none 似乎适合我的情况，或者至少我不知道如何让它们工作。我也试过改变注册函数和初始化池的顺序，但没有成功。我也尝试过使用 SCOOP，但结果相似。

有什么想法吗？

Answer 1

解决方案是在全局范围内创建 "FitnessMin" 和 "Individual"，即在 main.py:

import ...

creator.create("FitnessMin", base.Fitness, weights=(-1.0,))
creator.create("Individual", list, fitness=creator.FitnessMin)

def main():
    ...
    schedule.schedule()
    ...

if __name__== "__main__":
    main()

Python Windows 上的 DEAP 和多处理：AttributeError

Python DEAP and multiprocessing on Windows: AttributeError

python

windows

multiprocessing

python-3.x

deap