将 softmax 分类器分层到 RNN 自动编码器中

Question

我正在实施的论文是使用带有自动编码器的 RNN 来 class 验证异常网络数据（二进制 class 化）。他们首先在无人监督的情况下训练模型，然后他们描述了这个过程：

Next, fine-tuning training (supervised) is conducted to train the last layer of the network using labeled samples. Implementing the fine-tuning using supervised training criterion can further optimize the whole network. We use softmax regression layer with two channels at the top layer

目前，我已经实现了自动编码器：

class AnomalyDetector(Model):
    def __init__(self):
        super(AnomalyDetector, self).__init__()
        self.encoder = tf.keras.Sequential([
            layers.Dense(64, activation="relu"),
            layers.Dense(32, activation="relu"),
            layers.Dense(16, activation="relu"),
            layers.Dense(8, activation="relu")])

        self.decoder = tf.keras.Sequential([
            layers.Dense(16, activation="relu"),
            layers.Dense(32, activation="relu"),
            layers.Dense(64, activation="relu"),
            layers.Dense(79, activation='relu')
        ])

TensorFlow中的softmax回归层是如何实现的？

我无法理解这个过程，我是否应该向自动编码器添加另一层？我应该向 class 添加另一个功能吗？

Answer 1

以防万一以后有人访问这里 - 您可以通过更改激活来创建 softmax 层。我在我的例子中选择了 sigmoid 激活，因为 sigmoid 等价于 双元素 softmax。根据 documentation.

class AnomalyDetector(Model):
    def __init__(self):
        super(AnomalyDetector, self).__init__()
        self.pretrained = False
        self.finished_training = False
        self.encoder = tf.keras.Sequential([
            layers.SimpleRNN(64, activation="relu", return_sequences=True),
            layers.SimpleRNN(32, activation="relu", return_sequences=True),
            layers.SimpleRNN(16, activation="relu", return_sequences=True),
            layers.SimpleRNN(8, activation="relu", return_sequences=True)])

        self.decoder = tf.keras.Sequential([
            layers.SimpleRNN(16, activation="relu", return_sequences=True),
            layers.SimpleRNN(32, activation="relu", return_sequences=True),
            layers.SimpleRNN(64, activation="relu", return_sequences=True),
            layers.SimpleRNN(79, activation="relu"), return_sequences=True])
            layers.SimpleRNN(1, activation="sigmoid")

将 softmax 分类器分层到 RNN 自动编码器中

Layering softmax classifier into RNN autoencoder

python

tensorflow

neural-network

recurrent-neural-network

autoencoder