TypeError: 'type' object is not iterable when iterating over collections.deque that contains collections.namedtuple

Question

我制作了一个简单的回放缓冲区，当我从中采样时出现错误 TypeError: 'type' object is not iterable

import collections
import numpy as np

Experience = collections.namedtuple("Experience", field_names=["state", "action", "reward", "done", "next_state"])

class ReplayBuffer:
    def __init__(self, capacity):
        self.buffer = collections.deque(maxlen=capacity)

    def __len__(self):
        return len(self.buffer)

    def add_exp(self, exp: Experience):
        self.buffer.append(exp)

    def sample(self, batch_size):
        idxs = np.random.choice(len(self.buffer), batch_size, replace=False)
        states, actions, rewards, dones, next_states = zip(*[self.buffer[idx] for idx in idxs])

        return np.array(states), np.array(actions), \
               np.array(rewards, dtype=np.float32), \
               np.array(dones, dtype=np.uint8), \
               np.array(next_states)

当我打印 self.buffer[0] 的类型时，它给出 'type' 但它不应该是 ReplayBuffer.Experience 吗？

Answer 1

您正在向列表中添加一个类型，而不是该类型的一个实例。您所做的与此基本相同：

class Experience:
    pass

buffer = []

buffer.append(Experience)

希望这能让您更清楚地了解问题所在。您需要先创建 Experience 的实例，然后将该实例添加到列表中。像这样：

exp = Experience(the_state, the_action, the_reward, the_done, the_next_state)
buff.add_exp(exp)

其中所有 the_ 变量都是您要用来实例化对象的数据。

另请注意，更现代的写法 Experience 是 class 和 NamedTuple:

class Experience(NamedTuple):
    state: state_type
    action: action_type
    rewards: reward_type
    done: done_type
    next_state: state_type

其中_type是每个字段的类型。这允许类型检查器帮助您捕获类型错误。

TypeError: 'type' object is not iterable when iterating over collections.deque that contains collections.namedtuple

TypeError: 'type' object is not iterable when iterating over collections.deque that contains collections.namedtuple

python

reinforcement-learning

python-collections