在张量流中堆叠 LSTM layers/cells

Question

我正在尝试在 TF 中堆叠 LSTM 单元，这就是我所拥有的：

for layer in xrange(args.num_layers):
    cell_fw = tf.contrib.rnn.LSTMCell(args.hidden_size,
                                  initializer=tf.orthogonal_initializer())

    cell_bw = tf.contrib.rnn.LSTMCell(args.hidden_size,
                                  initializer=tf.orthogonal_initializer())

    cells_fw.append(cell_fw)
    cells_bw.append(cell_bw)

output = initial_input 
for layer in xrange(args.num_layers):
    ((output_fw, output_bw), (last_state_fw, first_state_bw)) = tf.nn.bidirectional_dynamic_rnn(
        cells_fw[layer], cells_bw[layer], output,
        dtype=tf.float32)

    output = tf.concat([output_fw, output_bw], axis=-1)

这给了我一个错误：

ValueError: Variable bidirectional_rnn/fw/lstm_cell/kernel already exists, disallowed. Did you mean to set reuse=True in VarScope?

当我将其设置为 true 时，我得到

ValueError: Variable bidirectional_rnn/fw/lstm_cell/kernel does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=None in VarScope?

谁能告诉我我做错了什么或者正确的方法是什么。

Answer 1

通常您只需要在不同的范围内创建 RNN，如 this issue:

中所建议的

with tf.variable_scope('lstm1'):
  output, state = tf.nn.rnn_cell.BasicLSTMCell(3)(input, init_state)
with tf.variable_scope('lstm2'):
  output2, state2 = tf.nn.rnn_cell.BasicLSTMCell(3)(input2, init_state2)

请注意，范围应包括 RNN 创建，而不是 cell 创建。

如果你真的需要在同一个范围内使用这些RNNs，用reuse=tf.AUTO_REUSE调用它（在最新版本的tensorflow中引入）。

在张量流中堆叠 LSTM layers/cells

Stacking LSTM layers/cells in tensorflow

machine-learning

tensorflow

neural-network

lstm

rnn