keras 上的准确性没有跨时代增长

Question

我是机器学习和深度学习的新手，我正在尝试使用神经网络对 5 类文本进行分类。为此，我制作了一本字典，以便将单词翻译成索引，最后得到一个包含索引列表的数组。此外，我将标签更改为整数。我也做了填充和那些东西。问题是，当我拟合模型时，准确度保持很低（~0.20）并且不会随着时间的推移而改变。我尝试更改了很多参数，例如词汇表的大小、神经元的数量、丢失概率、优化器参数等。代码的关键部分如下。

# Arrays with indexes (that works fine)
X_train = tokens_to_indexes(tokenized_tr_mrp, vocab, return_vocab=False)
X_test, vocab_dict = tokens_to_indexes(tokenized_te_mrp, vocab)

# Labels to integers
labels_dict = {}
labels_dict['Alzheimer'] = 0
labels_dict['Bladder Cancer'] = 1
labels_dict['Breast Cancer'] = 2
labels_dict['Cervical Cancer'] = 3
labels_dict['Negative'] = 4

y_train = np.array([labels_dict[i] for i in y_tr])
y_test = np.array([labels_dict[i] for i in y_te])

# One-hot encoding of labels
from keras.utils import to_categorical
encoded_train = to_categorical(y_train)
encoded_test = to_categorical(y_test)

# Padding
max_review_length = 235
X_train_pad = sequence.pad_sequences(X_train, maxlen=max_review_length)
X_test_pad = sequence.pad_sequences(X_test, maxlen=max_review_length)

# Model
# Vocab size
top_words = len(list(vocab_dict.keys()))
# Neurone type
rnn = LSTM
# dropout
set_dropout = True
p = 0.2
# embedding size
embedding_vector_length = 64
# regularization strength
L = 0.0005
# Number of neurones
N = 50

# Model
model = Sequential()
# Embedding layer
model.add(Embedding(top_words,
                   embedding_vector_length,
                   embeddings_regularizer=regularizers.l1(l=L),
                   input_length=max_review_length
                    #,embeddings_constraint=UnitNorm(axis=1)
                   ))

# Dropout layer
if set_dropout:
    model.add(Dropout(p))

# Recurrent layer
model.add(rnn(N))

# Output layer
model.add(Dense(5, activation='softmax'))

# Compilation
model.compile(loss='categorical_crossentropy',
             optimizer=Adam(lr=0.001),
             metrics=['Accuracy'])

# Split training set for validation
X_tr, X_va, y_tr_, y_va = train_test_split(X_train_pad, encoded_train, 
                                          test_size=0.3, random_state=2)

# Parameters
batch_size = 50
# N epochs
n_epocas = 20

best_val_acc  = 0
best_val_loss = 1e20
best_i        = 0

best_weights = []
acum_tr_acc = []
acum_tr_loss = []
acum_val_acc = []
acum_val_loss = []

# Training
for e in range(n_epocas):
    h = model.fit(X_tr, y_tr_,
                batch_size=batch_size,
                validation_data=(X_va, y_va),
                epochs=1, verbose=1)
  
    acum_tr_acc  = acum_tr_acc  + h.history['accuracy']
    acum_tr_loss = acum_tr_loss + h.history['loss']
    val_acc  = h.history['val_accuracy'][0]
    val_loss = h.history['val_loss'][0]
    acum_val_acc  = acum_val_acc  + [val_acc]
    acum_val_loss = acum_val_loss + [val_loss]
#   if val_acc  > best_val_acc:
    if val_loss < best_val_loss:
        best_i        = len(acum_val_acc)-1
        best_val_acc  = val_acc
        best_val_loss = val_loss
        best_weights = model.get_weights().copy()
    if len(acum_tr_acc)>1 and (len(acum_tr_acc)+1) % 1 == 0:
        if e>1:
            clear_output()

Answer 1

您发布的代码是非常糟糕的做法。

您可以使用当前方法训练 n_epocas 并添加回调以获得最佳权重（例如 ModelCheckpoint) or use tf.GradientTape 但一次对一个时期使用 model.fit() 会导致奇怪的结果，因为你的优化器不知道它在哪个纪元。

我建议保留您当前的代码，但一次性训练 n_epocas 并在此处报告结果（准确性 + 损失）。

Answer 2

有人给了我解决方案。我只需要更改这一行：

model.compile(loss='categorical_crossentropy',
             optimizer=Adam(lr=0.001),
             metrics=['Accuracy'])

为此：

model.compile(loss='categorical_crossentropy',
                 optimizer=Adam(lr=0.001),
                 metrics=['acc'])

我还更改了与准确性相关的最终循环中的行。 one-hot 编码也是必要的。

keras 上的准确性没有跨时代增长

Accuracy not growing across epochs on keras

python

text-classification

deep-learning

keras

recurrent-neural-network