TensorFlow 中堆叠式 LSTM 网络的维数

Question

在查看有关多维输入和堆叠 LSTM RNN 的众多类似问题时，我没有找到一个示例来说明 initial_state 占位符和下面 rnn_tuple_state 之后的维度。尝试的 [lstm_num_layers, 2, None, lstm_num_cells, 2] 是这些示例 (http://monik.in/a-noobs-guide-to-implementing-rnn-lstm-using-tensorflow/, https://medium.com/@erikhallstrm/using-the-tensorflow-multilayered-lstm-api-f6e7da7bbe40) 中代码的扩展，在末尾为特征的每个时间步的多个值添加了额外的维度 feature_dim（这不起作用，而是由于 tensorflow.nn.dynamic_rnn 调用中的尺寸不匹配而产生 ValueError。

time_steps = 10
feature_dim = 2
label_dim = 4
lstm_num_layers = 3
lstm_num_cells = 100
dropout_rate = 0.8

# None is to allow for variable size batches
features = tensorflow.placeholder(tensorflow.float32,
                                  [None, time_steps, feature_dim])
labels = tensorflow.placeholder(tensorflow.float32, [None, label_dim])

cell = tensorflow.contrib.rnn.MultiRNNCell(
    [tensorflow.contrib.rnn.LayerNormBasicLSTMCell(
        lstm_num_cells,
        dropout_keep_prob = dropout_rate)] * lstm_num_layers,
    state_is_tuple = True)

# not sure of the dimensionality for the initial state
initial_state = tensorflow.placeholder(
    tensorflow.float32,
    [lstm_num_layers, 2, None, lstm_num_cells, feature_dim])
# which impacts these two lines as well
state_per_layer_list = tensorflow.unstack(initial_state, axis = 0)
rnn_tuple_state = tuple(
    [tensorflow.contrib.rnn.LSTMStateTuple(
        state_per_layer_list[i][0],
        state_per_layer_list[i][1]) for i in range(lstm_num_layers)])

# also not sure if expanding the feature dimensions is correct here
outputs, state = tensorflow.nn.dynamic_rnn(
    cell, tensorflow.expand_dims(features, -1),
    initial_state = rnn_tuple_state)

最有帮助的是对一般情况的解释，其中：

每个时间步都有 N 个值
每个时间序列有S步
每批次有 B 个序列
每个输出都有 R 个值
网络中有 L 个隐藏的 LSTM 层
每一层有M个节点

因此伪代码版本为：

# B, S, N, and R are undefined values for the purpose of this question
features = tensorflow.placeholder(tensorflow.float32, [B, S, N])
labels = tensorflow.placeholder(tensorflow.float32, [B, R])
...

如果我能说完我一开始就不会在这里问了。提前致谢。欢迎对相关最佳实践提出任何意见。

Answer 1

经过多次试验和错误后，无论特征的维度如何，以下都会产生堆叠的 LSTM dynamic_rnn：

time_steps = 10
feature_dim = 2
label_dim = 4
lstm_num_layers = 3
lstm_num_cells = 100
dropout_rate = 0.8
learning_rate = 0.001

features = tensorflow.placeholder(
    tensorflow.float32, [None, time_steps, feature_dim])
labels = tensorflow.placeholder(
    tensorflow.float32, [None, label_dim])

cell_list = []
for _ in range(lstm_num_layers):
    cell_list.append(
        tensorflow.contrib.rnn.LayerNormBasicLSTMCell(lstm_num_cells,
                                                      dropout_keep_prob=dropout_rate))
cell = tensorflow.contrib.rnn.MultiRNNCell(cell_list, state_is_tuple=True)
initial_state = tensorflow.placeholder(
    tensorflow.float32, [lstm_num_layers, 2, None, lstm_num_cells])
state_per_layer_list = tensorflow.unstack(initial_state, axis=0)
rnn_tuple_state = tuple(
    [tensorflow.contrib.rnn.LSTMStateTuple(
        state_per_layer_list[i][0],
        state_per_layer_list[i][1]) for i in range(lstm_num_layers)])
state_series, last_state = tensorflow.nn.dynamic_rnn(
    cell=cell, inputs=features, initial_state=rnn_tuple_state)

hidden_layer_output = tensorflow.transpose(state_series, [1, 0, 2])
last_output = tensorflow.gather(hidden_layer_output, int(
    hidden_layer_output.get_shape()[0]) - 1)

weights = tensorflow.Variable(tensorflow.random_normal(
    [lstm_num_cells, int(labels.get_shape()[1])]))
biases = tensorflow.Variable(tensorflow.constant(
    0.0, shape=[labels.get_shape()[1]]))
predictions = tensorflow.matmul(last_output, weights) + biases
mean_squared_error = tensorflow.reduce_mean(
    tensorflow.square(predictions - labels))
minimize_error = tensorflow.train.RMSPropOptimizer(
    learning_rate).minimize(mean_squared_error)

开始这段旅程的部分原因是之前引用的例子重塑了输出以适应分类器而不是回归器（这是我试图构建的）。由于这与特征维度无关，因此它用作此用例的通用模板。

TensorFlow 中堆叠式 LSTM 网络的维数

Dimensionality for stacked LSTM network in TensorFlow

python

tensorflow

input

lstm

rnn