python scipy 差异进化优化失败，工人不是 1

Question

我正在尝试使用 the scipy differential evolution algo 的 workers 参数。

当我将其设置为 1 时，我的脚本运行没有任何问题。如果我放一些不同的东西，它会失败并出现以下回溯：

Traceback (most recent call last):
  File "/home/ubuntu/.local/lib/python3.6/site-packages/scipy/_lib/_util.py", line 419, in __call__
    return self._mapfunc(func, iterable)
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 266, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 644, in get
    raise self._value
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 424, in _handle_tasks
    put(task)
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/usr/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
TypeError: cannot serialize '_io.TextIOWrapper' object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/.local/lib/python3.6/site-packages/scipy/optimize/_differentialevolution.py", line 878, in _calculate_population_energies
    parameters_pop[0:nfevs]))
  File "/home/ubuntu/.local/lib/python3.6/site-packages/scipy/_lib/_util.py", line 422, in __call__
    raise TypeError("The map-like callable must be of the"
TypeError: The map-like callable must be of the form f(func, iterable)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main_parallel.py", line 323, in <module>
    optimizer(args.asset_class, udl, tenor, args.criteria, params_decay, params, couch_server, args.num_workers)
  File "main_parallel.py", line 269, in optimizer
    maxiter=5, workers=num_workers, mutation=(0.5, 1.5), recombination=0.8)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/scipy/optimize/_differentialevolution.py", line 306, in differential_evolution
    ret = solver.solve()
  File "/home/ubuntu/.local/lib/python3.6/site-packages/scipy/optimize/_differentialevolution.py", line 745, in solve
    self.population[self.feasible]))
  File "/home/ubuntu/.local/lib/python3.6/site-packages/scipy/optimize/_differentialevolution.py", line 883, in _calculate_population_energies
    raise RuntimeError("The map-like callable must be of the"
RuntimeError: The map-like callable must be of the form f(func, iterable), returning a sequence of numbers the same length as 'iterable'

有人可以帮助解释如何使用此参数（特定 objective 函数？其他约束？）？

也欢迎使用多核的简单 python 差分进化优化示例。

Answer 1

differential evolution documentation 关于工人使用的关键部分是警告，当 worker != 1 时，它

[...] Requires that func be pickleable

出现的错误中有几个提示表明 func 不可腌制，即

    self._mapfunc(func, iterable)
    ...
    self._send_bytes(_ForkingPickler.dumps(obj))
    ...
TypeError: cannot serialize '_io.TextIOWrapper' object

显然，曾尝试过酸洗 func，但失败了，大概是因为 func 不可酸洗。看起来 workers 试图被解释为 类映射或可调用 ，如文档中的其他签名所示。可以预见这也会失败，因为 workers 实际上是一个 int。

文档包含一个 complete example，其中 workers != 1 可以正常工作，

from scipy.optimize import differential_evolution, rosen
bounds = [(0,2), (0, 2), (0, 2), (0, 2), (0, 2)]
result = differential_evolution(rosen, bounds, updating='deferred',
                                workers=-1)
result.x, result.fun

如果您将 scipy.optimize.differential_evolution 中使用的 func 重构为可通过 pickling 序列化，您应该能够成功使用 workers != 1。

python scipy 差异进化优化失败，工人不是 1

python scipy differential evolution optimization fails with workers not 1

python

python-3.x

parallel-processing

scipy

differential-evolution