logits 和标签必须是可广播的：logits_size=[0,2] labels_size=[32,2]

Question

我正在实施一个 CNN 模型，通过使用 Haar 小波分解来检测图像上的莫尔图案。为了生成用于训练的图像数据，我在以下代码中实现了自定义生成：

class WaveletImageGenerator(Sequence):

    def __init__(self, image_data, batch_size=32):
        self.image_paths = [data[0] for data in image_data]
        self.image_labels = [data[1] for data in image_data]
        self.batch_size = batch_size

    def __len__(self):
        return len(self.image_paths) // self.batch_size

    def __getitem__(self, idx):
        i = idx * self.batch_size
        paths = self.image_paths[i:i + self.batch_size]
        X_LL, X_LH, X_HL, X_HH = [], [], [], []
        y = np.array(self.image_labels[i:i + self.batch_size])

        for j, path in enumerate(paths):
            assert len(os.listdir(path)) == 4
            LL_path, LH_path, HL_path, HH_path = [os.path.join(path, d) for d in os.listdir(path)]
            x_LL = read_and_convert_image(LL_path, 0, 1)
            x_LH = read_and_convert_image(LH_path, -1, 1)
            x_HL = read_and_convert_image(HL_path, -1, 1)
            x_HH = read_and_convert_image(HH_path, -1, 1)

            X_LL.append(x_LL)
            X_LH.append(x_LH)
            X_HL.append(x_HL)
            X_HH.append(x_HH)

        return [np.array(X_LL), np.array(X_LH), np.array(X_HL), np.array(X_HH)], to_categorical(y, 2)

这里，类的个数为2（有云纹的图片和没有云纹的图片）。我使用的模型是一个有 4 个输入的 CNN，取自 this GitHub repo。以下是模型代码：

def create_model(img_height=250, img_width=250, img_channels=1, n_classes=2):
    inp_LL = Input(shape=(img_height, img_width, img_channels))
    inp_LH = Input(shape=(img_height, img_width, img_channels))
    inp_HL = Input(shape=(img_height, img_width, img_channels))
    inp_HH = Input(shape=(img_height, img_width, img_channels))

    conv_LL = Conv2D(32, kernel_size=(7, 7), padding='same', activation='relu')(inp_LL)
    conv_LH = Conv2D(32, kernel_size=(7, 7), padding='same', activation='relu')(inp_LH)
    conv_HL = Conv2D(32, kernel_size=(7, 7), padding='same', activation='relu')(inp_HL)
    conv_HH = Conv2D(32, kernel_size=(7, 7), padding='same', activation='relu')(inp_HH)
    pool_LL = MaxPooling2D(pool_size=(2, 2))(conv_LL)
    pool_LH = MaxPooling2D(pool_size=(2, 2))(conv_LH)
    pool_HL = MaxPooling2D(pool_size=(2, 2))(conv_HL)
    pool_HH = MaxPooling2D(pool_size=(2, 2))(conv_HH)

    avg_LH_HL_HH = Maximum()([pool_LH, pool_HL, pool_HH])
    inp_merged = Multiply()([pool_LL, avg_LH_HL_HH])

    x = Conv2D(16, kernel_size=(3, 3), padding='same', activation='relu')(inp_merged)
    x = MaxPooling2D(pool_size=(4, 4))(x)
    x = Dropout(0.25)(x)
    x = Conv2D(32, kernel_size=(3, 3), padding='same', activation='relu')(x)
    x = MaxPooling2D(pool_size=(2, 2))(x)
    x = Conv2D(32, kernel_size=(3, 3), padding='same', activation='relu')(x)
    x = MaxPooling2D(pool_size=(2, 2))(x)
    x = Dropout(0.25)(x)
    x = Flatten()(x)
    x = Dense(32, activation='relu')(x)
    x = Dropout(0.5)(x)
    output = Dense(n_classes, activation='softmax')(x)

    model = Model(inputs=[inp_LL, inp_LH, inp_HL, inp_HH], outputs=output)

    return model

这是我的训练管道：

train_gen, valid_gen = prepare_data_pipeline(args.pos_data_dir, args.neg_data_dir)
model = prepare_model()

if not os.path.exists('weights/'):
    os.makedirs('weights/')

mc = ModelCheckpoint('weights/best_model.h5', monitor='val_accuracy', verbose=1,
                     save_best_only=True, mode='max')
reduce_lr = ReduceLROnPlateau(factor=1e-3, cooldown=0, patience=5, min_lr=5e-6)
es = EarlyStopping(monitor='val_accuracy', mode='max', verbose=1, patience=30)

model.fit(train_gen, validation_data=valid_gen, epochs=100, callbacks=[mc, reduce_lr, es])

然而，当我运行训练时，我遇到了这个错误：

logits and labels must be broadcastable: logits_size=[0,2] labels_size=[32,2]

我非常确定类的数量是 2（因为传递给 Generator 的构造函数的 image_data 参数是一个列表，其中每个元素都是一个元组（path_to_image , label), 并且我已经确保标签是 0 或 1), 从我在这里看到的情况来看, 模型的输出具有形状 (None, 2)。因此我真的不明白为什么我会收到错误。非常感谢任何帮助。

更新：这是我用来准备训练模型的数据管道的函数：

def prepare_data_pipeline(pos_path, neg_path):
    image_data = []
    for subdir in os.listdir(pos_path):
        if os.path.isfile(os.path.join(pos_path, subdir)):
            continue
        image_data.append((os.path.join(pos_path, subdir), 1))
    for subdir in os.listdir(neg_path):
        if os.path.isfile(os.path.join(neg_path, subdir)):
            continue
        image_data.append((os.path.join(neg_path, subdir), 0))

    train_data, valid_data = split_train_valid(image_data)
    train_gen = WaveletImageGenerator(image_data=train_data, batch_size=32)
    valid_gen = WaveletImageGenerator(image_data=valid_data, batch_size=32)

    return train_gen, valid_gen

所以批量大小肯定是 32。

Answer 1

首先，如果真的错过了，请拨打model.compile()

其次，检查x.shape。我制作了模拟数据生成器，它运行良好。

class WaveletImageGenerator(tf.keras.utils.Sequence):
  def __init__(self, batch_size=32):
    self.batch_size = batch_size

  def __len__(self):
    return 4

  def __getitem__(self, idx):
    x = np.random.rand(32, 250, 250, 1)
    y = np.zeros((32,1))
    return [x, x, x, x], tf.keras.utils.to_categorical(y, 2)

train_gen = WaveletImageGenerator()
val_gen = WaveletImageGenerator()
model.compile(optimizer='sgd', 
    loss=tf.keras.losses.CategoricalCrossentropy(), 
    metrics='accuracy')
model.fit(train_gen, validation_data=val_gen, epochs=100)

logits 和标签必须是可广播的：logits_size=[0,2] labels_size=[32,2]

logits and labels must be broadcastable: logits_size=[0,2] labels_size=[32,2]

python

computer-vision

keras

tensorflow