用于检测文本异常的 Keras 自动编码器模型

Question

我正在尝试创建一个能够在文本序列中发现异常的自动编码器：

X_train_pada_seq.shape
(28840, 999)

我想使用层嵌入。这是我的模型：

encoder_inputs = Input(shape=(max_len_str, ))
encoder_emb =  Embedding(input_dim=len(word_index)+1, output_dim=20, input_length=laenge_pads)(encoder_inputs)

encoder_LSTM_1 = Bidirectional(LSTM(400, activation='relu', return_sequences=True))(encoder_emb)
encoder_drop = Dropout(0.2)(encoder_LSTM_1)
encoder_LSTM_2 = Bidirectional(GRU(200, activation='relu', return_sequences=False,  name = 'bottleneck'))(encoder_drop)

decoder_repeated = RepeatVector(200)(encoder_LSTM_2)
decoder_LSTM = Bidirectional(LSTM(400, activation='relu', return_sequences=True))(decoder_repeated)
decoder_drop = Dropout(0.2)(decoder_LSTM)
decoder_output = TimeDistributed(Dense(999, activation='softmax'))(decoder_drop)

autoencoder = Model(encoder_inputs, decoder_output)
autoencoder.compile(optimizer=keras.optimizers.Adam(learning_rate=0.0001), loss='categorical_crossentropy', metrics=['accuracy'])
autoencoder.summary()


Model: "model_7"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_10 (InputLayer)        [(None, 999)]             0         
_________________________________________________________________
embedding_19 (Embedding)     (None, 999, 20)           159660    
_________________________________________________________________
bidirectional (Bidirectional (None, 999, 800)          1347200   
_________________________________________________________________
dropout (Dropout)            (None, 999, 800)          0         
_________________________________________________________________
bidirectional_1 (Bidirection (None, 400)               1202400   
_________________________________________________________________
repeat_vector (RepeatVector) (None, 200, 400)          0         
_________________________________________________________________
bidirectional_2 (Bidirection (None, 200, 800)          2563200   
_________________________________________________________________
dropout_1 (Dropout)          (None, 200, 800)          0         
_________________________________________________________________
time_distributed_6 (TimeDist (None, 200, 999)          800199    
=================================================================
Total params: 6,072,659
Trainable params: 6,072,659
Non-trainable params: 0

但是在训练模型时：

history = autoencoder.fit(X_train_pada_seq, X_train_pada_seq, epochs=10, batch_size=64, 
                    validation_data=(X_test_pada_seq, X_test_pada_seq))

我收到一个错误：

ValueError: Shapes (None, 999) and (None, 200, 999) are incompatible

如何重新制作模型来修复错误？

Answer 1

我看过你的代码片段，你的模型输出似乎需要匹配你的目标形状，即 (None, 999)，但你的输出形状是 (None, 200 , 999).

您需要使您的输出模型形状与目标形状匹配。

尝试使用 tf.reduce_mean 和 axis=1（平均所有序列）：

decoder_drop = Dropout(0.2)(decoder_LSTM)
decoder_time = TimeDistributed(Dense(999, activation='softmax'))(decoder_drop)
decoder_output = tf.math.reduce_mean(decoder_time, axis=1)

这应该能让您适合模型。

Answer 2

你的最后一层（输出）应该是这个形状

batchsize x 999 x 200) #999个词，200是每个词的dim

目前您模型的输出是批量大小 x 200 x 999

这是不正确的。

使用稀疏分类交叉熵作为损失函数。然后它会起作用。

用于检测文本异常的 Keras 自动编码器模型

Keras autoencoder model for detect anomaly in text

machine-learning

keras

nlp

lstm

autoencoder