FailedPreconditionError（回溯见上）：GetNext() 失败，因为迭代器尚未初始化

Question

我在制作predictions.However时为输入数据构建了数据集管道，当我尝试代码时，出现错误

FailedPreconditionError (see above for traceback): GetNext() failed because the iterator has not been initialized. Ensure that you have run the initializer operation for this iterator before getting the next element. [[Node: IteratorGetNext_259 = IteratorGetNextoutput_shapes=[[?,227,227,6]], output_types=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

遍历数据集的迭代器定义如下：

for k in range(num_init_ops):
with tf.device('/cpu:0'):


    pre_data.append(PreDataGenerator(pre_file,
                                mode='predicting',
                                batch_size=batch_size,
                                num_classes=num_classes,
                                shuffle=False,
                                iterator_size=iterator_size,
                                kth_init_op=k))

    # create an reinitializable iterator given the dataset structure
    iterator = Iterator.from_structure(pre_data[k].data.output_types,
                                       pre_data[k].data.output_shapes)
    next_batch = iterator.get_next()

# Ops for initializing the two different iterators
predicting_init_op.append(iterator.make_initializer(pre_data[k].data))

编写for循环是因为我希望创建多个数据集初始化操作来将数据拆分到不同的迭代器中，以防止累积内存调用导致 OOM 错误（我想知道是否这样做会奏效）。

我确定迭代器已初始化（调试时输出正确的结构）。这是我的 Tensorflow 会话代码：

with tf.Session(config=config) as sess:
sess.run(tf.global_variables_initializer())
# sess.run(tf.local_variables_initializer())
saver.restore(sess, './checkpoints_grade1/model_epoch46.ckpt')  # todo:

print("{} Start predicting...".format(datetime.now()))

for j in range(num_init_ops+1):#todo:

    print('{} Initializing {} iterator'.format(datetime.now(),j))

    # Initialize iterator with the predicting dataset
    sess.run(predicting_init_op[j])

    for i in range(iterator_size):

        # get next batch of data
        img_batch = sess.run(next_batch)#todo:?

        # And run the predicting op
        img_batch = tf.reshape(img_batch, (1, 227, 227, 6))
        pred = sess.run(softmax, feed_dict={x: sess.run(img_batch)})
        predicted_label = pred.argmax(axis=1)
        predictions.append(predicted_label[0])
        output_file.write(str(i) + ' , ' + str(predicted_label[0]) + '\n')

Answer 1

您需要初始化迭代器：

sess.run(iterator.initializer)

然后像这样训练：

next_batch = iterator.get_next()
sess.run(iterator.initializer)

for epoch in range(n_epochs):
    while True:
        try:
            batch = sess.run(next_batch)
            # feed data, train
            # ...
        except tf.errors.OutOfRangeError:
            sess.run(iterator.initializer)
            break

或者，在定义 tf.data.Dataset.from_tensor_slices 实例时，您可以指定要训练的轮数：

data = tf.data.Dataset.from_tensor_slices({
            'x':train_data,
            'y':train_labels
            }).repeat(n_epochs).batch(batch_size)
iterator = data.make_initializable_iterator()

有了这个你就不需要 for epoch in range(n_epochs) 循环了：

next_batch = iterator.get_next()
sess.run(iterator.initializer)

while True:
    try:
        batch = sess.run(next_batch)
        # feed data, train
        # ...
    except tf.errors.OutOfRangeError:
        break

Answer 2

我通过a shell script控制预测程序，每次我开始预测一批样本，这样内存就不会运行了。问题解决了。

#!/bin/bash 
for ((i=0;i<9;i++))
do
    python classifier_v4.py --iter_epoch $i
done

FailedPreconditionError（回溯见上）：GetNext() 失败，因为迭代器尚未初始化

FailedPreconditionError (see above for traceback): GetNext() failed because the iterator has not been initialized

python

tensorflow

tensorflow-datasets