如何在tensorflow中恢复保存的BiRNN模型，使所有输出神经元正确绑定到对应的输出类

Question

我在 tensorflow 中正确恢复保存的模型时遇到了问题。我使用以下代码在 tensorflow 中创建了双向 RNN 模型：

batchX_placeholder = tf.placeholder(tf.float32, [None, timesteps, 1],
                                    name="batchX_placeholder")])
batchY_placeholder = tf.placeholder(tf.float32, [None, num_classes],
                                    name="batchY_placeholder")
weights = tf.Variable(np.random.rand(2*STATE_SIZE, num_classes),
                      dtype=tf.float32, name="weights")
biases = tf.Variable(np.zeros((1, num_classes)), dtype=tf.float32,
                     name="biases")
logits = BiRNN(batchX_placeholder, weights, biases)
with tf.name_scope("prediction"):
    prediction = tf.nn.softmax(logits)
loss_op = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(
logits=logits, labels=batchY_placeholder))
lr = tf.Variable(learning_rate, trainable=False, dtype=tf.float32,
                 name='lr')
optimizer = tf.train.AdamOptimizer(learning_rate=lr)
train_op = optimizer.minimize(loss_op)
init_op = tf.initialize_all_variables()
saver = tf.train.Saver()

使用以下函数创建的 BiRNN 架构：

def BiRNN(x, weights, biases):
    # Unstack to get a list of 'time_steps' tensors of shape (batch_size,
    # num_input)
    x = tf.unstack(x, time_steps, 1)
    # Forward and Backward direction cells
    lstm_fw_cell = rnn.BasicLSTMCell(STATE_SIZE, forget_bias=1.0)
    lstm_bw_cell = rnn.BasicLSTMCell(STATE_SIZE, forget_bias=1.0)
    outputs, _, _ = rnn.static_bidirectional_rnn(lstm_fw_cell,
        lstm_bw_cell, x, dtype=tf.float32)
    # Linear activation, using rnn inner loop last output
    return tf.matmul(outputs[-1], weights) + biases

然后我训练一个模型并在每 200 步后保存它：

with tf.Session() as sess:
    sess.run(init_op)
    current_step = 0
    for batch_x, batch_y in get_minibatch():
        sess.run(train_op, feed_dict={batchX_placeholder: batch_x,
                                      batchY_placeholder: batch_y})
        current_step += 1
        if current_step % 200 == 0:
            saver.save(sess, os.path.join(model_dir, "model")

为了运行推理模式下保存的模型，我使用 "model.meta" 文件中保存的张量流图：

graph = tf.get_default_graph()
saver = tf.train.import_meta_graph(os.path.join(model_dir, "model.meta"))
sess = tf.Session()
saver.restore(sess, tf.train.latest_checkpoint(model_dir)
weights = graph.get_tensor_by_name("weights:0")
biases = graph.get_tensor_by_name("biases:0")
batchX_placeholder = graph.get_tensor_by_name("batchX_placeholder:0")
batchY_placeholder = graph.get_tensor_by_name("batchY_placeholder:0")
logits = BiRNN(batchX_placeholder, weights, biases)
prediction = graph.get_operation_by_name("prediction/Softmax")
argmax_pred = tf.argmax(prediction, 1)
init = tf.global_variables_initializer()
sess.run(init)
for x_seq, y_gt in get_sequence():
    _, y_pred = sess.run([prediction, argmax_pred],
                    feed_dict={batchX_placeholder: [x_seq]],
                               batchY_placeholder: [[0.0, 0.0]]})
    print("Y ground true: " + str(y_gt) + ", Y pred: " + str(y_pred[0]))

当我运行推理模式下的代码时，每次启动它时我都会得到不同的结果。似乎 softmax 层的输出神经元随机捆绑了不同的输出类.

所以，我的问题是：如何在 tensorflow 中保存然后正确恢复模型，以便所有神经元与相应的输出正确捆绑类？

Answer 1

不需要调用tf.global_variables_initializer()，我想那是你的问题。

我删除了一些操作：logits、weights 和 biases 因为你不需要它们，所有这些都已经加载，使用 graph.get_tensor_by_name 来获取它们.

对于prediction，获取tensor而不是operation。（见此）：

这是代码：

graph = tf.get_default_graph()
saver = tf.train.import_meta_graph(os.path.join(model_dir, "model.meta"))
sess = tf.Session()
saver.restore(sess, tf.train.latest_checkpoint(model_dir))

batchX_placeholder = graph.get_tensor_by_name("batchX_placeholder:0")
batchY_placeholder = graph.get_tensor_by_name("batchY_placeholder:0")
prediction = graph.get_tensor_by_name("prediction/Softmax:0")
argmax_pred = tf.argmax(prediction, 1)

编辑 1：我注意到我不清楚为什么你会得到不同的结果。

And when I run the code in inference mode, I get different results each time I launch it.

请注意，虽然您使用了加载模型中的权重，但您正在再次创建 BiRNN，并且 BasicLSTMCell 也有您未从加载模型中设置的权重和其他变量模型，因此需要对其进行初始化（使用新的随机值），从而再次生成未经训练的模型。

如何在tensorflow中恢复保存的BiRNN模型，使所有输出神经元正确绑定到对应的输出类

How to restore saved BiRNN model in tensorflow so that all output neurons correctly bundled to the corresponding output classes

model

restore

tensorflow