Tensorflow session - ValueError: GraphDef cannot be larger than 2GB

Question

我正在尝试迭代数据集批次并运行对预训练模型进行推理。我创建了一个会话并加载了模型：

import numpy as np

sess = tf.Session()

saver = tf.train.import_meta_graph('model_resnet/imagenet.ckpt.meta')
saver.restore(sess, "model_resnet/imagenet.ckpt")

# To view the graph in tensorboard:
summary_writer = tf.summary.FileWriter("/tmp/tensorflow_logdir", graph=tf.get_default_graph())

# To retrieve outputs of layer while inferring
def getActivations(layer,stimuli):
    units = sess.run(layer,feed_dict={"Placeholder_:0": stimuli, keep_prob:1.0})

# Convert to TF Dataset
dataset_train = tf.data.Dataset.from_tensor_slices((X_train, y_train))
dataset_test = tf.data.Dataset.from_tensor_slices((X_test, y_test))

# Create batches
dataset = dataset_train.batch(32)

# Iterator to iterate over images in batch
iterator = dataset.make_one_shot_iterator()

next_element = iterator.get_next()

try:
    getActivations("resnet/pool:0",sess.run(next_element[1]))
except tf.errors.OutOfRangeError:
    print("End of dataset")  # ==> "End of dataset"

我收到这个错误：

ValueError: GraphDef cannot be larger than 2GB.

我可能误解了图表的确切含义。我不明白为什么对 32 张图像的单次迭代会导致图形扩展。我的操作是否添加到预训练模型图上？根据我目前所遇到的，向 TF Graph 添加操作是使用 add 或 tf 完成的。'function_name'，这样正确吗？

任何帮助或指向示例的指针将不胜感激。

谢谢。

Answer 1

我在这里复习了类似的问题并应用了一些方法来消除错误：

使用占位符加载数据：

features_placeholder = tf.placeholder(X_train.dtype, X_train.shape)
labels_placeholder = tf.placeholder(y_train.dtype, y_train.shape)
dataset = tf.data.Dataset.from_tensor_slices((features_placeholder, labels_placeholder))

使用 with 创建会话并使用以下方法加载图表：

with tf.Session() as sess:

    initialize_iterator(sess, iterator, X_train, y_train)
    next_element = iterator.get_next()

    load_graph(sess)

Tensorflow session - ValueError: GraphDef cannot be larger than 2GB

Tensorflow session - ValueError: GraphDef cannot be larger than 2GB

python

tensorflow

transfer-learning