Unet 架构中的最后一个卷积层期望维度错误

Last convoulutional layer in U-net architecure is expecting wrong dimention

我正尝试在 Keras 中实现 u-net,但在训练模型时出现此错误(调用 model.fit()):

ValueError: Error when checking target: expected conv2d_302 to have shape > (None, 1, 128, 640) but got array with shape (360, 1, 128, 128)

model.summary() 的输出是:


Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_19 (InputLayer)           (None, 1, 128, 128)  0                                            
__________________________________________________________________________________________________
conv2d_303 (Conv2D)             (None, 32, 128, 128) 320         input_19[0][0]                   
__________________________________________________________________________________________________
conv2d_304 (Conv2D)             (None, 32, 128, 128) 9248        conv2d_303[0][0]                 
__________________________________________________________________________________________________
max_pooling2d_83 (MaxPooling2D) (None, 32, 64, 64)   0           conv2d_304[0][0]                 
__________________________________________________________________________________________________
conv2d_305 (Conv2D)             (None, 64, 64, 64)   18496       max_pooling2d_83[0][0]           
__________________________________________________________________________________________________
conv2d_306 (Conv2D)             (None, 64, 64, 64)   36928       conv2d_305[0][0]                 
__________________________________________________________________________________________________
max_pooling2d_84 (MaxPooling2D) (None, 64, 32, 32)   0           conv2d_306[0][0]                 
__________________________________________________________________________________________________
conv2d_307 (Conv2D)             (None, 128, 32, 32)  73856       max_pooling2d_84[0][0]           
__________________________________________________________________________________________________
conv2d_308 (Conv2D)             (None, 128, 32, 32)  147584      conv2d_307[0][0]                 
__________________________________________________________________________________________________
max_pooling2d_85 (MaxPooling2D) (None, 128, 16, 16)  0           conv2d_308[0][0]                 
__________________________________________________________________________________________________
conv2d_309 (Conv2D)             (None, 256, 16, 16)  295168      max_pooling2d_85[0][0]           
__________________________________________________________________________________________________
conv2d_310 (Conv2D)             (None, 256, 16, 16)  590080      conv2d_309[0][0]                 
__________________________________________________________________________________________________
max_pooling2d_86 (MaxPooling2D) (None, 256, 8, 8)    0           conv2d_310[0][0]                 
__________________________________________________________________________________________________
conv2d_311 (Conv2D)             (None, 512, 8, 8)    1180160     max_pooling2d_86[0][0]           
__________________________________________________________________________________________________
conv2d_312 (Conv2D)             (None, 512, 8, 8)    2359808     conv2d_311[0][0]                 
__________________________________________________________________________________________________
conv2d_transpose_29 (Conv2DTran (None, 256, 16, 16)  524544      conv2d_312[0][0]                 
__________________________________________________________________________________________________
concatenate_29 (Concatenate)    (None, 256, 16, 32)  0           conv2d_transpose_29[0][0]        
                                                                 conv2d_310[0][0]                 
__________________________________________________________________________________________________
conv2d_313 (Conv2D)             (None, 256, 16, 32)  590080      concatenate_29[0][0]             
__________________________________________________________________________________________________
conv2d_314 (Conv2D)             (None, 256, 16, 32)  590080      conv2d_313[0][0]                 
__________________________________________________________________________________________________
conv2d_transpose_30 (Conv2DTran (None, 128, 32, 64)  131200      conv2d_314[0][0]                 
__________________________________________________________________________________________________
concatenate_30 (Concatenate)    (None, 128, 32, 96)  0           conv2d_transpose_30[0][0]        
                                                                 conv2d_308[0][0]                 
__________________________________________________________________________________________________
conv2d_315 (Conv2D)             (None, 128, 32, 96)  147584      concatenate_30[0][0]             
__________________________________________________________________________________________________
conv2d_316 (Conv2D)             (None, 128, 32, 96)  147584      conv2d_315[0][0]                 
__________________________________________________________________________________________________
conv2d_transpose_31 (Conv2DTran (None, 64, 64, 192)  32832       conv2d_316[0][0]                 
__________________________________________________________________________________________________
concatenate_31 (Concatenate)    (None, 64, 64, 256)  0           conv2d_transpose_31[0][0]        
                                                                 conv2d_306[0][0]                 
__________________________________________________________________________________________________
conv2d_317 (Conv2D)             (None, 64, 64, 256)  36928       concatenate_31[0][0]             
__________________________________________________________________________________________________
conv2d_318 (Conv2D)             (None, 64, 64, 256)  36928       conv2d_317[0][0]                 
__________________________________________________________________________________________________
conv2d_transpose_32 (Conv2DTran (None, 32, 128, 512) 8224        conv2d_318[0][0]                 
__________________________________________________________________________________________________
concatenate_32 (Concatenate)    (None, 32, 128, 640) 0           conv2d_transpose_32[0][0]        
                                                                 conv2d_304[0][0]                 
__________________________________________________________________________________________________
conv2d_319 (Conv2D)             (None, 32, 128, 640) 9248        concatenate_32[0][0]             
__________________________________________________________________________________________________
conv9 (Conv2D)                  (None, 32, 128, 640) 9248        conv2d_319[0][0]                 
__________________________________________________________________________________________________
conv2d_320 (Conv2D)             (None, 1, 128, 640)  33          conv9[0][0]                      
==================================================================================================
Total params: 6,976,161
Trainable params: 6,976,161
Non-trainable params: 0

型号代码如下:

img_rows=128
img_cols= 128
inputs = Input((1, img_rows, img_cols))
conv1 = Conv2D(32, (3, 3), activation='relu', padding='same')(inputs)
conv1 = Conv2D(32, (3, 3), activation='relu', padding='same')(conv1)
pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)

conv2 = Conv2D(64, (3, 3), activation='relu', padding='same')(pool1)
conv2 = Conv2D(64, (3, 3), activation='relu', padding='same')(conv2)
pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)

conv3 = Conv2D(128, (3, 3), activation='relu', padding='same')(pool2)
conv3 = Conv2D(128, (3, 3), activation='relu', padding='same')(conv3)
pool3 = MaxPooling2D(pool_size=(2, 2))(conv3)

conv4 = Conv2D(256, (3, 3), activation='relu', padding='same')(pool3)
conv4 = Conv2D(256, (3, 3), activation='relu', padding='same')(conv4)
pool4 = MaxPooling2D(pool_size=(2, 2))(conv4)

conv5 = Conv2D(512, (3, 3), activation='relu', padding='same')(pool4)
conv5 = Conv2D(512, (3, 3), activation='relu', padding='same')(conv5)

up6 = concatenate([Conv2DTranspose(256, (2, 2), strides=(2, 2), padding='same')(conv5), conv4], axis=3)
conv6 = Conv2D(256, (3, 3), activation='relu', padding='same')(up6)
conv6 = Conv2D(256, (3, 3), activation='relu', padding='same')(conv6)

up7 = concatenate([Conv2DTranspose(128, (2, 2), strides=(2, 2), padding='same')(conv6), conv3], axis=3)
conv7 = Conv2D(128, (3, 3), activation='relu', padding='same')(up7)
conv7 = Conv2D(128, (3, 3), activation='relu', padding='same')(conv7)

up8 = concatenate([Conv2DTranspose(64, (2, 2), strides=(2, 2), padding='same')(conv7), conv2], axis=3)
conv8 = Conv2D(64, (3, 3), activation='relu', padding='same')(up8)
conv8 = Conv2D(64, (3, 3), activation='relu', padding='same')(conv8)

up9 = concatenate([Conv2DTranspose(32, (2, 2), strides=(2, 2), padding='same')(conv8), conv1], axis=3)
conv9 = Conv2D(32, (3, 3), activation='relu', padding='same')(up9)
conv9 = Conv2D(32, (3, 3), activation='relu', padding='same', name='conv9')(conv9)

conv10 = Conv2D(1, (1, 1), activation='sigmoid')(conv9)

model = Model(inputs=[inputs], outputs=[conv10])

model.compile(optimizer=Adam(lr=1e-5), loss="mean_absolute_error")
model.summary()
model.fit(X_train, y_train, batch_size=36, nb_epoch=5)

我不明白为什么倒数第二层(conv9)的输出与最后一层(conv10)的预期不同。

Keras 模型由 https://github.com/jocicmarko/ultrasound-nerve-segmentation/blob/master/train.py 提供。

已更新:添加了完整的 model.summary()。

可以肯定的是,原作者想要在 channels 维度上进行串联,而不是图像维度之一。

卷积网络中的张量可以采用以下两种格式之一:

(batch_size, width, height, channels)

(batch_size, channels, width, height)

在您链接的模型中使用了第一种格式,但您的模型使用了第二种格式。

您可以通过以下两种方式之一修复它:

  1. concatente 层中的 axis=3 更改为 axis=1

  1. 在卷积层中设置data_format="channels_last"data_format 的默认值,如果省略,取自 keras 配置,很可能这个值对于您和您使用的模型的作者是不同的。参见 https://keras.io/layers/convolutional/#conv2d

更新:事实上,原始模型在您链接到的文件的开头更改了data_format

K.set_image_data_format('channels_last')

只需在模型的开头添加这一行,即可解决问题。