Tensorflow fit_generator - 如何正确使用纪元？

Question

我正在使用 Tensorflow Hub 开发二进制文本分类器。

x_train = np.array(["Some test text", 
                    "Testing this text",
                    "This is relevant to my test",
                    "Cows don't fly",
                    "One two three",
                    "some text"])
y_train = np.array([1,1,1,0,0,0])

model = "https://tfhub.dev/google/tf2-preview/gnews-swivel-20dim/1"

hub_layer = hub.KerasLayer(model, output_shape=[20], input_shape=[], 
                           dtype=tf.string, trainable=True)
model = tf.keras.Sequential()
model.add(hub_layer)
model.add(tf.keras.layers.Dense(16, activation='relu', input_shape=[20]))
model.add(tf.keras.layers.Dense(1, activation='sigmoid'))

model.summary()

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['accuracy'])

当我使用 fit 命令拟合模型时，我可以将我的纪元设置为我想要的最高值。当我使用fit_generator方法时，似乎每个数据点只能使用一次。

# Trains as expected
model.fit(x_train, y_train, epochs=100)

# Errors during training with 
#    'Your dataset iterator ran out of data; interrupting training. 
#     Make sure that your iterator can generate at least `steps_per_epoch * epochs` 
#     batches (in this case, 600 batches).'
model.fit_generator(my_iterator(x_train, y_train), epochs=100, steps_per_epoch=len(x_train))

如何正确设置我的迭代器以便能够使用生成器进行训练？

Answer 1

"The generator is expected to loop over its data indefinitely"。所以需要将我的迭代器定义为：

    def my_iterator(x, y):
    while True:
        for _x, _y in zip(x, y):
            yield np.array([_x]), np.array([_y])

Tensorflow fit_generator - 如何正确使用纪元？

Tensorflow fit_generator - how to use epochs correctly?

python

tensorflow

tensorflow2.0

tf.keras

tensorflow-hub