如何在重新训练时初始化 tensorflow.contrib.slim fully_connected 层的权重和偏差？

Question

在保存并训练张量流图之后，我将其恢复回来以使用不同的损失函数进行重新训练，如下所示：

 import tensorflow as tf
 import numpy as np
 import pyximport
 pyximport.install()
 import math
 import tensorflow.contrib.slim as slim
 raw_data_train = np.loadtxt('all_data/train_all_raw.csv', skiprows = 1, delimiter=',')
 users = (np.unique(raw_data_train[ :, 0]))
 items = (np.unique(raw_data_train[ :, 1]))
 saver = tf.train.import_meta_graph('all_data/my_test_model.meta')

 with tf.Session() as sess:
     tf.global_variables_initializer().run(session=sess)
     saver.restore(sess, tf.train.latest_checkpoint('all_data/'))

     # placeholders
     user_ids = sess.graph.get_tensor_by_name('user_ids:0')
     left_ids = sess.graph.get_tensor_by_name('left_ids:0')

     # variables
     user_latents = sess.graph.get_tensor_by_name('user_latents:0')
     item_latents = sess.graph.get_tensor_by_name('item_latents:0')

     # network was initiall defined as variable_scope "nn" that is why I am retrieving them as "nn/*" in the following line

     weights_0 = sess.graph.get_tensor_by_name('nn/fully_connected/weights:0')
     biases_0 = sess.graph.get_tensor_by_name('nn/fully_connected/biases:0')
     weights_1 = sess.graph.get_tensor_by_name('nn/fully_connected_1/weights:0')
     biases_1 = sess.graph.get_tensor_by_name('nn/fully_connected_1/biases:0')

     # lookups

     user_embeddings = sess.graph.get_tensor_by_name('embedding_user:0')
     item_left_embeddings = sess.graph.get_tensor_by_name('embedding_left:0')


     # dictionary
     fd = {
       user_ids: users,
       left_ids: items,
     }
     left_emb_val, weights_0_val, biases_0_val, weights_1_val, biases_1_val = sess.run([left_emb, weights_0, biases_0, weights_1, biases_1], feed_dict=fd)

     joined_input = tf.concat( [user_embeddings, item_left_embeddings], 1)
     net = slim.fully_connected(inputs=joined_input, num_outputs=64,           weights_initializer = tf.constant_initializer(weights_0_val), biases_initializer=tf.constant_initializer(biases_0_val), activation_fn=tf.nn.relu)
     left_output = slim.fully_connected(inputs=net, num_outputs=1, weights_initializer = tf.constant_initializer(weights_1_val), biases_initializer=tf.constant_initializer(biases_1_val), activation_fn=None)

# ********* below line gives an error *************

     left_output_val = sess.run([left_output], feed_dict=fd)
     print(left_output_val)

当我试图通过调用 sess.run.

来计算 left_output_val 的值时，上面的代码给出了以下错误

 FailedPreconditionError (see above for traceback): Attempting to use uninitialized value fully_connected_1/biases
 [[Node: fully_connected_1/biases/read = Identity[T=DT_FLOAT, _class=["loc:@fully_connected_1/biases"], _device="/job:localhost/replica:0/task:0/cpu:0"](fully_connected_1/biases)]]

这让我有点惊讶，因为：

我使用以下行初始化了所有变量：
```
 tf.global_variables_initializer().run(session=sess)
```
这可能是因为权重和偏差未按照此处建议的那样使用此行进行初始化：

我正在以下行中初始化权重和偏差：

net = slim.fully_connected(inputs=joined_input, num_outputs=64,           weights_initializer = tf.constant_initializer(weights_0_val), biases_initializer=tf.constant_initializer(biases_0_val), activation_fn=tf.nn.relu)

left_output = slim.fully_connected(inputs=net, num_outputs=1, weights_initializer = tf.constant_initializer(weights_1_val), biases_initializer=tf.constant_initializer(biases_1_val), activation_fn=None)

在运行会话和计算 left_output_val

我很感激在这里解决我的问题的任何想法。

Answer 1

你可以从这个密集层中获取变量并手动初始化它们。

with tf.variable_scope('fully_connected_1', reuse=True):
  weights = tf.get_variable('weights')
  biases = tf.get_variable('biases')

  sess.run([weights.initializer, biases.initializer])

Answer 2

问题出在以下位置：

tf.global_variables_initializer().run(session=sess)

应该在:

之后

left_output = slim.fully_connected(inputs=net, num_outputs=1, weights_initializer = tf.constant_initializer(weights_1_val), biases_initializer=tf.constant_initializer(biases_1_val), activation_fn=None)

如何在重新训练时初始化 tensorflow.contrib.slim fully_connected 层的权重和偏差？

How can I initialize weights and biases of tensorflow.contrib.slim fully_connected layer while re-training?

python

machine-learning

deep-learning

tensorflow

tensorflow-slim