关于 Tensorflow 2.0 中的输入图像，如何计算中间层过滤器激活的梯度？

Question

我正在尝试可视化激活中间层的特定过滤器的图像类型。为此，我需要计算该过滤器中激活的均值相对于输入图像的梯度，然后使用梯度上升更新图像。

我一直在思考如何在 Tensorflow 2.0 中计算这个梯度。我试过了；在这里，我试图在 block3_conv1 层中获取索引为 0 的过滤器的输出：

input = tf.convert_to_tensor(np.random.random((1, 150, 150, 3))

activation_model = Model(inputs=model.input,
                         outputs=model.get_layer("block3_conv1").output)

with tf.GradientTape() as tape:
    tape.watch(inputs)
    preds = activation_model.predict(inputs)
    loss = np.mean(preds[:,:,:,0]) # defining the mean of all activations as the loss, in the filter with index 0

grads = tape.gradient(tf.convert_to_tensor(loss), inputs)

但这给了我 grads 作为 None。这是模型摘要：

Model: "vgg16"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         [(None, None, None, 3)]   0         
_________________________________________________________________
block1_conv1 (Conv2D)        (None, None, None, 64)    1792      
_________________________________________________________________
block1_conv2 (Conv2D)        (None, None, None, 64)    36928     
_________________________________________________________________
block1_pool (MaxPooling2D)   (None, None, None, 64)    0         
_________________________________________________________________
block2_conv1 (Conv2D)        (None, None, None, 128)   73856     
_________________________________________________________________
block2_conv2 (Conv2D)        (None, None, None, 128)   147584    
_________________________________________________________________
block2_pool (MaxPooling2D)   (None, None, None, 128)   0         
_________________________________________________________________
block3_conv1 (Conv2D)        (None, None, None, 256)   295168    
_________________________________________________________________
block3_conv2 (Conv2D)        (None, None, None, 256)   590080    
_________________________________________________________________
block3_conv3 (Conv2D)        (None, None, None, 256)   590080    
_________________________________________________________________
block3_pool (MaxPooling2D)   (None, None, None, 256)   0         
_________________________________________________________________
block4_conv1 (Conv2D)        (None, None, None, 512)   1180160   
_________________________________________________________________
block4_conv2 (Conv2D)        (None, None, None, 512)   2359808   
_________________________________________________________________
block4_conv3 (Conv2D)        (None, None, None, 512)   2359808   
_________________________________________________________________
block4_pool (MaxPooling2D)   (None, None, None, 512)   0         
_________________________________________________________________
block5_conv1 (Conv2D)        (None, None, None, 512)   2359808   
_________________________________________________________________
block5_conv2 (Conv2D)        (None, None, None, 512)   2359808   
_________________________________________________________________
block5_conv3 (Conv2D)        (None, None, None, 512)   2359808   
_________________________________________________________________
block5_pool (MaxPooling2D)   (None, None, None, 512)   0         
=================================================================
Total params: 14,714,688
Trainable params: 14,714,688
Non-trainable params: 0
_________________________________________________________________

Answer 1

只是不要使用 model.predict。这个 returns numpy 数组，你不能通过 numpy 操作反向传播。下面的代码通过使用模型的 call 函数保留在张量空间中。

with tf.GradientTape() as tape:
    tape.watch(inputs)
    preds = activation_model(inputs)
    loss = tf.reduce_mean(preds[:,:,:,0]) # defining the mean of all activations as the loss, in the filter with index 0

grads = tape.gradient(loss, inputs)

关于 Tensorflow 2.0 中的输入图像，如何计算中间层过滤器激活的梯度？

How to compute gradients of a filter's activation of an intermediate layer, with respect to an input image in Tensorflow 2.0?

python-3.x

tensorflow

tensorflow2.0