历史图中的垂直训练损失

Question

为什么我的损失在剧情中是垂直的？我正在 Keras 和 TF 中训练具有单个输出的多头回归模型。

我已经包括了前几个时期的损失：

'history': {'loss': [87756.7265625, 15.339912414550781, 0.4351891875267029, 0.413672536611557, 0.4036506116390228, 0.4017665684223175, 0.4009793996810913, 0.40057373046875, 0.4004470109939575, 0.40035486221313477, 0.4005628526210785, 0.40025919675827026, 0.3998614251613617, 0.4000871181488037, 0.39995384216308594,

数据集形状:

   X_train = [input_1, input_2, input_3] #(543, 318, 13) (543, 176, 13) (543, 60, 13)
   y_train.shape #(543, 1)
   X_val = [input_1_val, input_2_val, input_3_val] #(155, 318, 13) (155, 176, 13) (155, 60, 13)
   y_val.shape #(155, 1)

代码:

# first input model
input_1 = layers.Input(shape=input1_shape) #(543, 318, 13)
cnn1 = layers.Conv1D(filters=64, kernel_size=2, activation='relu')(input_1)
cnn1 = layers.MaxPooling1D(pool_size=2)(cnn1)
cnn1 = layers.Flatten()(cnn1)

# second input model
input_2 = layers.Input(shape=input2_shape) #(543, 176, 13)
cnn2 = layers.Conv1D(filters=64, kernel_size=2, activation='relu')(input_2)
cnn2 = layers.MaxPooling1D(pool_size=2)(cnn2)
cnn2 = layers.Flatten()(cnn2)

# third input model
input_3 = layers.Input(shape=input3_shape) #(543, 60, 13)
cnn3 = layers.Conv1D(filters=64, kernel_size=2, activation='relu')(input_3)
cnn3 = layers.MaxPooling1D(pool_size=2)(cnn3)
cnn3 = layers.Flatten()(cnn3)

# merge input models
merge = layers.concatenate([cnn1, cnn2, cnn3])
dense = layers.Dense(50, activation='relu')(merge)
output = layers.Dense(1)(dense)
model = Model(inputs=[input_1, input_2, input_3], outputs=output) #keras.models
model.compile(optimizer=Adam(learning_rate=lr), loss=MeanSquaredError(), metrics=["mse", "mae"])

history = model.fit(x=X_train, y=y_train, epochs=epochs, batch_size=batch_size, validation_data=(X_val, y_val), verbose=1)

plt.plot(history.history["loss"])
plt.plot(history.history["val_loss"])
plt.title('Model Loss vs Epoch')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['train', 'validation'], loc='upper left')
plt.show()

Answer 1

问题已解决。我没有注意到第一个 epoch 的损失明显高于其余 epochs。

{'loss': [87756.7265625, 15.339912414550781, 0.4351891875267029 ...

在第 2 个时期之后，损失始终小于 1。这扭曲了图形的比例，使其看起来垂直（尤其是 1000 个时期）。缩放还导致 val_loss（包含的值全部 < 1）覆盖训练集损失。

我将 link 添加到 this SO post：“为什么 Keras 损失在第一个 epoch 后急剧下降？”作为遇到此问题的任何人的额外资源。

历史图中的垂直训练损失

Vertical training loss in history plot

keras

tensorflow2.0