在 OpenAI Gym 上实现多处理时出错，'NoneType' 对象没有属性 'steer'

Question

我正在尝试使用 OpenAI Gym 实现多处理。我知道那里有现成的解决方案，但我这样做是为了在 Gym 和多处理方面进行一些练习。

问题是我运行遇到了一个我无法确定原因的错误。这是代码：

import gym
import numpy as np
import multiprocessing as multip


class Environments:
    gym_environment = 'CarRacing-v0'

    def __init__(self, environments_n):
        self.environments_n = environments_n
        self.cores_count = multip.cpu_count()
        self.envs = []
        self.reset_all_environments()

    def __enter__(self):
        self.pool = multip.Pool(self.cores_count)
        return self

    def __exit__(self, exc_type, exc_value, tb):
        self.pool.close()
        self.pool.join()

    def reset_all_environments(self):
        for env in self.envs:
            env.close()
        self.envs = [gym.make(self.gym_environment) for _ in range(self.environments_n)]
        self.dones = [False for _ in range(self.environments_n)]
        self.last_obs = [env.reset() for env in self.envs]

    @staticmethod
    def step_single(env, action):
        print(action, env)
        return env.step(action)

    def step_all(self):
        actions = [env.action_space.sample() for env in self.envs]
        results = self.pool.starmap(self.step_single, zip(self.envs, actions))


if __name__ == '__main__':
    with Environments(2) as envs:
        print(envs.step_all())

在我的期望中，这应该只是运行创建的 2 个环境中的一个步骤，但是我却得到了这个错误：

Track generation: 1134..1423 -> 289-tiles track
retry to generate track (normal if there are not many of this messages)
Track generation: 1153..1445 -> 292-tiles track
Track generation: 1188..1492 -> 304-tiles track
retry to generate track (normal if there are not many of this messages)
Track generation: 1233..1554 -> 321-tiles track
[0.01238655 0.20499504 0.07908794] <TimeLimit<CarRacing instance>>
[0.8294716 0.5289044 0.7956768] <TimeLimit<CarRacing instance>>
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "C:\Program Files\Python37\lib\multiprocessing\pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "C:\Program Files\Python37\lib\multiprocessing\pool.py", line 47, in starmapstar
    return list(itertools.starmap(args[0], args[1]))
  File "C:\Users\Nadni\PycharmProjects\ActorCritic\test.py", line 33, in step_single
    return env.step(action)
  File "C:\Program Files\Python37\lib\site-packages\gym\wrappers\time_limit.py", line 15, in step
    observation, reward, done, info = self.env.step(action)
  File "C:\Program Files\Python37\lib\site-packages\gym\envs\box2d\car_racing.py", line 315, in step
    self.car.steer(-action[0])
AttributeError: 'NoneType' object has no attribute 'steer'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "test.py", line 42, in <module>
    print(envs.step_all())
  File "test.py", line 37, in step_all
    results = self.pool.starmap(self.step_single, zip(self.envs, actions))
  File "C:\Program Files\Python37\lib\multiprocessing\pool.py", line 276, in starmap
    return self._map_async(func, iterable, starmapstar, chunksize).get()
  File "C:\Program Files\Python37\lib\multiprocessing\pool.py", line 657, in get
    raise self._value
AttributeError: 'NoneType' object has no attribute 'steer'
Exception ignored in: <function Viewer.__del__ at 0x000002DCC71E8A60>
Traceback (most recent call last):
  File "C:\Program Files\Python37\lib\site-packages\gym\envs\classic_control\rendering.py", line 152, in __del__
  File "C:\Program Files\Python37\lib\site-packages\gym\envs\classic_control\rendering.py", line 71, in close
  File "C:\Program Files\Python37\lib\site-packages\pyglet\window\win32\__init__.py", line 305, in close
  File "C:\Program Files\Python37\lib\site-packages\pyglet\window\__init__.py", line 770, in close
ImportError: sys.meta_path is None, Python is likely shutting down

我不知道为什么它说 AttributeError: 'NoneType' object has no attribute 'steer' 即使从 print() 中可以清楚地看出 env 变量正确地是 <TimeLimit<CarRacing instance>>[=17= 的成员]

Answer 1

我不完全确定是什么导致了这个问题，但我在使用 Ray 时遇到了同样的错误。我 "fixed" 通过如下更改它：

引发错误的行：

env = gym.make(env_name).unwrapped
env.step([0,0,0])

不抛出错误的行：

env = gym.make(env_name)
env.reset()
env.step([0,0,0])

希望这可以帮助遇到此问题的人。

在 OpenAI Gym 上实现多处理时出错，'NoneType' 对象没有属性 'steer'

Error when implementing multiprocessing on OpenAI Gym, 'NoneType' object has no attribute 'steer'

python

multiprocessing

python-multiprocessing

openai-gym