Class 使用迁移学习激活地图

Question

我正在使用 Resnet50 构建一个 CNN 模型来识别 classify 5 个对象。这些物体的图像是在我的桌子上拍摄的，所以每个物体都有我桌子的一部分。初始化模型的代码是这样的，

model = Sequential()

pretrained_model= tf.keras.applications.ResNet50(include_top=False,
                   input_shape=(180,180,3),
                   pooling='avg',classes=5,
                   weights='imagenet')
for layer in pretrained_model.layers:
        layer.trainable=False

model.add(pretrained_model)
model.add(Flatten())
model.add(Dense(512, activation='relu'))
model.add(Dense(2, activation='softmax'))

我编译了模型并对其进行了拟合，它按预期工作。

模型运行不佳，预测不是很准确。我怀疑模型是在我办公桌的那部分进行训练的，我想使用 class 激活图来了解这是否属实。

我看过的教程有 class 个从头开始构建的模型的激活图代码。我知道我们需要添加一个全局平均池化层，然后添加一个具有 softmax 激活的致密层以启用 class 激活。

Resnet50 模型以我通过运行、

发现的全局平均池化层结尾

pretrained_model.layers

所以我只需要添加由运行、

添加的致密层

model.add(pretrained_model)
model.add(Dense(2, activation='softmax'))

但是当我打印出这个模型的摘要时，我得到，

Model: "sequential_3"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
resnet50 (Functional)        (None, 2048)              23587712  
_________________________________________________________________
dense_3 (Dense)              (None, 2)                 4098      
=================================================================
Total params: 23,591,810
Trainable params: 4,098
Non-trainable params: 23,587,712

我正在关注 Laurence Moroney 的 example，他说我们必须从全局平均池化层和密集层中提取权重，而我无法使用我刚刚创建的模型。

有没有办法扩展resnet50 (Functional)层来访问全局平均池化层？

编辑

我在这里继续我的查询，因为它是我实际问题的一部分，即启用 class 具有迁移学习的激活图。

如评论中所述，我可以通过提供获得最后一个卷积层，

model.layers[0].layers[-5]

获得密集层和最后一个卷积层的权重后，我尝试创建cam_model，像这样，

cam_model  = Model(inputs=(model.layers[0].layers[0].input), outputs=(model.layers[0].layers[-5].output, model.layers[1].output))

导致此错误，

ValueError: Graph disconnected: cannot obtain value for tensor KerasTensor(type_spec=TensorSpec(shape=(None, 180, 180, 3), dtype=tf.float32, name='resnet50_input'), name='resnet50_input', description="created by layer 'resnet50_input'") at layer "resnet50". The following previous layers were accessed without issue: ['conv1_pad', 'conv1_conv', 'conv1_bn', 'conv1_relu', 'pool1_pad', 'pool1_pool', 'conv2_block1_1_conv', 'conv2_block1_1_bn', 'conv2_block1_1_relu', 'conv2_block1_2_conv', 'conv2_block1_2_bn', 'conv2_block1_2_relu', 'conv2_block1_3_conv', 'conv2_block1_0_conv', 'conv2_block1_0_bn', 'conv2_block1_3_bn', 'conv2_block1_add', 'conv2_block1_out', 'conv2_block2_1_conv', 'conv2_block2_1_bn', 'conv2_block2_1_relu', 'conv2_block2_2_conv', 'conv2_block2_2_bn', 'conv2_block2_2_relu', 'conv2_block2_3_conv', 'conv2_block2_3_bn', 'conv2_block2_add', 'conv2_block2_out', 'conv2_block3_1_conv', 'conv2_block3_1_bn', 'conv2_block3_1_relu', 'conv2_block3_2_conv', 'conv2_block3_2_bn', 'conv2_block3_2_relu', 'conv2_block3_3_conv', 'conv2_block3_3_bn', 'conv2_block3_add', 'conv2_block3_out', 'conv3_block1_1_conv', 'conv3_block1_1_bn', 'conv3_block1_1_relu', 'conv3_block1_2_conv', 'conv3_block1_2_bn', 'conv3_block1_2_relu', 'conv3_block1_3_conv', 'conv3_block1_0_conv', 'conv3_block1_0_bn', 'conv3_block1_3_bn', 'conv3_block1_add', 'conv3_block1_out', 'conv3_block2_1_conv', 'conv3_block2_1_bn', 'conv3_block2_1_relu', 'conv3_block2_2_conv', 'conv3_block2_2_bn', 'conv3_block2_2_r...

我的model.summary是这样的，

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
resnet50 (Functional)        (None, 2048)              23587712  
_________________________________________________________________
dense (Dense)                (None, 5)                 10245     
=================================================================
Total params: 23,597,957
Trainable params: 10,245
Non-trainable params: 23,587,712

我的model.layers[0].summary()的前几层是这样的，

Model: "resnet50"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_2 (InputLayer)            [(None, 180, 180, 3) 0                                            
__________________________________________________________________________________________________
conv1_pad (ZeroPadding2D)       (None, 186, 186, 3)  0           input_2[0][0]                    
__________________________________________________________________________________________________
conv1_conv (Conv2D)             (None, 90, 90, 64)   9472        conv1_pad[0][0]                  
__________________________________________________________________________________________________

我认为图表在 resnet50 层断开连接，但我不知道在哪里可以找到它。有人可以帮忙吗

Answer 1

要打印出 resnet 模型的图层，试试这个：model.layers[0].summary()。如果你想访问 GlobalAveragePooling 层，它恰好是模型的最后一层，那么试试这个： global_max_pooling = model.layers[0].layers[-1]。但是请注意，GlobalAveragePooling 层本身没有任何权重。

Class 使用迁移学习激活地图

Class map activation with transfer learning

python

computer-vision

deep-learning

keras

tensorflow