Keras 中 LSTM 权重的顺序

Question

我正在尝试对已学习的 LSTM 模型进行简单的评估（即前向传播），但我无法弄清楚 f_t、i_t、o_t 的顺序c_in 从 z 中提取。据我了解，它们是批量计算的。下面是使用 Keras 得到的模型架构：

我的输入顺序是：

input_seq = np.array([[[0.725323664],
                   [0.7671179],
                   [0.805884672]]])

输出应该是：

[ 0.83467698]

使用 Keras，我获得了第一个 LSTM 层的以下参数：

lstm_1_kernel_0 = np.array([[-0.40927699, -0.53539848, 0.40065038, -0.07722378, 0.30405849, 0.54959822, -0.23097005, 0.4720422, 0.05197877, -0.52746099, -0.5856396, -0.43691438]])

lstm_1_recurrent_kernel_0 = np.array([[-0.25504839, -0.0823682, 0.11609183,  0.41123426, 0.03409858, -0.0647027, -0.59183347, -0.15359771,  0.21647622,  0.24863823, 0.46169096, -0.21100986],
                                  [0.29160395,  0.46513283,  0.33996364, -0.31195125, -0.24458826, -0.09762905, 0.16202784, -0.01602131, 0.34460208, 0.39724654, 0.31806156, 0.1102117],
                                  [-0.15919448, -0.33053166, -0.22857222, -0.04912394, -0.21862955,  0.55346996, 0.38505834, 0.18110731, 0.270677, -0.02759281, 0.42814475, -0.13496138]])
lstm_1_bias_0 = np.array([0., 0., 0., 1., 1., 1., 0., 0., 0., 0., 0., 0.])

# LSTM 1
z_1_lstm_1 = np.dot(x_1_lstm_1, lstm_1_kernel_0) + np.dot(h_0_lstm_1, lstm_1_recurrent_kernel_0) + lstm_1_bias_0
i_1_lstm_1 = z_1_lstm_1[0, 0:3]
f_1_lstm_1 = z_1_lstm_1[0, 3:6]
input_to_c_1_lstm_1 = z_1_lstm_1[0, 6:9]
o_1_lstm_1 = z_1_lstm_1[0, 9:12]

所以问题是 i_1_lstm_1、f_1_lstm_1、input_to_c_1_lstm_1、o_1_lstm_1 的正确顺序是什么？

Answer 1

它是 (i, f, c, o)。在 recurrent.py 中，在 LSTMCell 中，权重由以下内容构成：

    self.kernel_i = self.kernel[:, :self.units]
    self.kernel_f = self.kernel[:, self.units: self.units * 2]
    self.kernel_c = self.kernel[:, self.units * 2: self.units * 3]
    self.kernel_o = self.kernel[:, self.units * 3:]

    self.recurrent_kernel_i = self.recurrent_kernel[:, :self.units]
    self.recurrent_kernel_f = self.recurrent_kernel[:, self.units: self.units * 2]
    self.recurrent_kernel_c = self.recurrent_kernel[:, self.units * 2: self.units * 3]
    self.recurrent_kernel_o = self.recurrent_kernel[:, self.units * 3:]

    if self.use_bias:
        self.bias_i = self.bias[:self.units]
        self.bias_f = self.bias[self.units: self.units * 2]
        self.bias_c = self.bias[self.units * 2: self.units * 3]
        self.bias_o = self.bias[self.units * 3:]

Keras 中 LSTM 权重的顺序

Order of LSTM weights in Keras

keras

tensorflow

keras-layer