对图像序列使用 categorical_crossentropy

Question

我有一个接受图像序列作为输入的模型 (None, n_step, 128, 128)（而不是单个图像），其中 n_step 是固定数字 10。我正在使用 categorical_crossentropy 来 class 化四个 class 问题。但是我有如下所示的错误

ValueError: A target array with shape (1342, 10, 4) was passed for an output of shape (None, 1, 4) while using as loss `categorical_crossentropy`. This loss expects targets to have the same shape as the output.

我从错误中了解到，它一次需要一张图片。有什么方法可以将这种损失用于一系列图像？

模型的输出也将是一组 10 个标签

编辑：

n_steps = 10
feature_count = 4
def create_model():

    trajectory_input = Input(shape=(n_steps, feature_count), name='trajectory_input')
    image_input  = Input(shape=(128, 128, n_steps), name='image_input')

    x_aware = (Conv2D(32, kernel_size=(3, 3), activation='relu'))(image_input)
    x_aware = (BatchNormalization())(x_aware)
    x_aware = (MaxPooling2D(pool_size=(2, 2)))(x_aware)
    x_aware = (Dropout(0.25))(x_aware)

    x_aware = (Conv2D(64, kernel_size=(3, 3), activation='relu'))(x_aware)
    x_aware = (BatchNormalization())(x_aware)
    x_aware = (MaxPooling2D(pool_size=(2, 2)))(x_aware)
    x_aware = (Dropout(0.25))(x_aware)

    x_aware = (Conv2D(64, kernel_size=(3, 3), activation='relu'))(x_aware)
    x_aware = (BatchNormalization())(x_aware)
    x_aware = (MaxPooling2D(pool_size=(2, 2)))(x_aware)
    x_aware = (Dropout(0.25))(x_aware)

    x_aware = (Dense(64, activation='relu', kernel_regularizer=regularizers.l2(0.001)))(x_aware)
    x_aware = (BatchNormalization())(x_aware)
    x_aware = (Dropout(0.25))(x_aware)
    x_aware = Reshape((1, 12544))(x_aware)

    x = (Dense(32, activation='relu', kernel_regularizer=regularizers.l2(0.001)))(trajectory_input)
    x = Reshape((1, 32*n_steps))(x)

    x = concatenate([x, x_aware])
    x = (Dense(64, activation='relu', kernel_regularizer=regularizers.l2(0.001)))(x)
    x = (Dense(32, activation='relu', kernel_regularizer=regularizers.l2(0.001)))(x)

    x_reg = (Dense(8, activation='relu', kernel_regularizer=regularizers.l2(0.001)))(x)
    x_class = (Dense(8, activation='relu', kernel_regularizer=regularizers.l2(0.001)))(x)

    x_reg = Reshape((2, 4))(x_reg)

    output_regression = (Dense(2, name='main_output'))(x_reg)
    output_class = (Dense(4, name='classification_output', activation='softmax'))(x_class)

    adam = Adam(lr=learning_rate)
    model = Model(inputs=[trajectory_input, image_input], outputs=[output_regression, output_class])
    model.compile(optimizer=adam, loss={'main_output': 'mse','classification_output': 'categorical_crossentropy'}, metrics={'main_output': [euc_dist_1, euc_dist_2], 'classification_output': 'accuracy'})
    model.summary()
    return model

输入是图像及其回归任务的相关信息，输出是标签和下一个可预测值。

Answer 1

问题出在分类任务的 Dense 和 Reshape 层的输入上。由于输入在 (10,128,128) 中，单位数应为 10*num_class。所以改变这个

x_reg = (Dense(8, activation='relu', kernel_regularizer=regularizers.l2(0.001)))(x)
x_class = (Dense(8, activation='relu', kernel_regularizer=regularizers.l2(0.001)))(x)

x_reg = Reshape((2, 4))(x_reg)

output_regression = (Dense(2, name='main_output'))(x_reg)
output_class = (Dense(4, name='classification_output', activation='softmax'))(x_class)

至此解决了问题

x_reg = (Dense(8, activation='relu', kernel_regularizer=regularizers.l2(0.001)))(x)
x_class = (Dense(40, activation='relu', kernel_regularizer=regularizers.l2(0.001)))(x)

x_reg = Reshape((2, 4))(x_reg)
x_class = Reshape((10,4))(x_class)

output_regression = (Dense(2, name='main_output'))(x_reg)
output_class = (Dense(4, name='classification_output', activation='softmax'))(x_class)

对图像序列使用 categorical_crossentropy

Using categorical_crossentropy for a sequence of images

python

tensorflow

keras

loss-function

cross-entropy