修改残差 LSTM

Question

我在这里找到了残差 LSTM 的代码：https://gist.github.com/bzamecnik/8ed16e361a0a6e80e2a4a259222f101e

我一直在使用 LSTM 进行具有 3d 输入（样本、时间步长、特征）和单个输出的时间序列分类。我有兴趣在我的数据上尝试残差模型，但我需要的是一个带有 sigmoid 激活的单一输出。任何人都知道如何做到这一点？当前模型似乎 return 10 个输出（输入数据中的特征数）。

def make_residual_lstm_layers(input, rnn_width, rnn_depth, rnn_dropout):
    """
    The intermediate LSTM layers return sequences, while the last returns a single element.
    The input is also a sequence. In order to match the shape of input and output of the LSTM
    to sum them we can do it only for all layers but the last.
    """
    x = input
    for i in range(rnn_depth):
        return_sequences = i < rnn_depth - 1
        x_rnn = LSTM(rnn_width, recurrent_dropout=rnn_dropout, dropout=rnn_dropout, return_sequences=return_sequences)(x)
        if return_sequences:
            # Intermediate layers return sequences, input is also a sequence.
            if i > 0 or input.shape[-1] == rnn_width:
                x = add([x, x_rnn])
            else:
                # Note that the input size and RNN output has to match, due to the sum operation.
                # If we want different rnn_width, we'd have to perform the sum from layer 2 on.
                x = x_rnn
        else:
            # Last layer does not return sequences, just the last element
            # so we select only the last element of the previous output.
            def slice_last(x):
                return x[..., -1, :]

            x = add([Lambda(slice_last)(x), x_rnn])
    return x

input = Input(shape=(32, 10))
output = make_residual_lstm_layers(input, rnn_width=10, rnn_depth=8, rnn_dropout=0.2)
model = Model(inputs=input, outputs=output)
model.summary()

这部分：model.compile(loss='binary_crossentropy', optimizer='adam') 我可以这样添加：

model = Model(inputs=input, outputs=output)
model.compile(loss='binary_crossentropy', optimizer='adam')
model.summary()

但我需要的是这样的：

input = Input(shape=(32, 10))
output = make_residual_lstm_layers(input, rnn_width=10, rnn_depth=8, rnn_dropout=0.2)
newoutput = Dense(1, activation='sigmoid')(output)
model = Model(inputs=input, outputs=newoutput)
model.compile(loss='binary_crossentropy', optimizer='adam')
model.summary()

有人知道如何修改模型来完成此任务吗？

Answer 1

主要问题是特征维度不匹配 (10 != 1) 因此无法在最后一部分应用跳过连接。这是我的建议，我用一个简单的 LSTM 层替换最后一个块，该层具有 1 个输出和一个 sigmoid 激活

def make_residual_lstm_layers(input, rnn_width, rnn_depth, rnn_dropout):

    x = input
    for i in range(rnn_depth):
        
        return_sequences = i < rnn_depth - 1
        x_rnn = LSTM(rnn_width, recurrent_dropout=rnn_dropout, dropout=rnn_dropout, 
                     return_sequences=return_sequences)(x)
        
        if return_sequences:
            
            if i > 0 or input.shape[-1] == rnn_width:
                x = add([x, x_rnn])
            else:
                x = x_rnn
        else:
            
            x = LSTM(1, activation='sigmoid', 
                     recurrent_dropout=rnn_dropout, dropout=rnn_dropout, 
                     return_sequences=return_sequences)(x)            
    return x

input = Input(shape=(32, 10))
output = make_residual_lstm_layers(input, rnn_width=10, rnn_depth=8, rnn_dropout=0.2)
model = Model(inputs=input, outputs=output)
model.summary()

修改残差 LSTM

Modifying residual LSTM

python

keras

lstm

deep-residual-networks