TF2.0中如何用GradientTape替换Keras的gradients()函数?

How to replace Keras' gradients() function with GradientTape in TF2.0?

借助 "old" Keras 库,我使用 keras.backend.gradients() 函数为我的 CNN 创建了热图,如下所示:

# load model and image, then predict the class this image belongs to
model = load_model(os.path.join(model_folder, "custom_model.h5"))
image = image.load_img(image_path)
img_tensor = image.img_to_array(image)
img_tensor = np.expand_dims(img_tensor, axis=0)
img_tensor = preprocess_input(img_tensor)

preds = model.predict(img_tensor)
model_prediction = model.output[:, np.argmax(preds[0])]

# Calculate pooled grads for heatmap
conv_layer = model.get_layer("block5_conv3")  # last conv. layer
grads = K.gradients(model_prediction, conv_layer.output)[0]
pooled_grads = K.mean(grads, axis=(0, 1, 2))

# Get values of pooled grads and model conv. layer output as Numpy arrays
input_layer = model.get_layer("model_input")
iterate = K.function([input_layer], [pooled_grads, conv_layer.output[0]])
pooled_grads_value, conv_layer_output_value = iterate([img_tensor])

# Continue with heatmap generation ...

现在我切换到 TF2.0,它是内置的 Keras 实现。一切正常,但是,使用该代码调用 K.gradients():

时出现以下错误
tf.gradients is not supported when eager execution is enabled. Use tf.GradientTape instead.

我做了一些研究并试图了解如何使用 GradientTape,但不幸的是我对 TF 和 TF2.0 知之甚少——我一直使用 Keras。你们能指导我如何使用我的设置再次进行梯度计算吗?

这是您的问题的解决方案。它意味着创建一个同时输出 conv_output 和预测的模型,因此我们可以正确应用 GradientTape

没有你的 model/data 所以我采用了 ResNet50 和随机值。

import numpy as np
import tensorflow as tf

model = tf.keras.applications.resnet50.ResNet50()
img_tensor = np.random.random((1, 224, 224, 3))

conv_layer = model.get_layer('conv5_block3_1_conv')

heatmap_model = tf.keras.models.Model(
    [model.inputs], [model.get_layer('conv5_block3_1_conv').output, model.output]
)

with tf.GradientTape() as tape:
    conv_output, predictions = heatmap_model(img_tensor)
    loss = predictions[:, np.argmax(predictions[0])]

grads = tape.gradient(loss, conv_output)

您不能使用 K.function,因为它在启用即时执行时不起作用,这是默认情况下启用的。 K.function 根据我有限的理解,可以在静态图上使用,当禁用急切执行时会发生这种情况。

tensorflow.compat.v1.disable_eager_execution()

上面的块可以缓解一些问题,但会产生新的问题。更好的解决方法是使用引导梯度方法,您只考虑正梯度和正卷积输出以获得引导梯度。

castConvOutputs = tensorflow.cast(conv_output > 0, "float32")
castGrads = tensorflow.cast(grads > 0, "float32")
guidedGrads = castConvOutputs * castGrads * grads

# My goal was to create the class activation map for single image, so we are skipping axis=0 which is meant to have the batch_size of images at axis=0
convOutputs = conv_output[0]
guidedGrads = guidedGrads[0]

最后,我取了图像宽度和高度的平均值(汇集引导梯度),并对所有通道(深度层)的数学加权激活求和,以创建 class 激活图。

weights = tensorflow.reduce_mean(guidedGrads, axis=(0, 1))
cam = tensorflow.reduce_sum(tensorflow.multiply(weights, convOutputs), axis=-1)

在规范化并缩放到 0-255 整数范围后,稍后可以使用 cv2.applycolormap 选择的颜色图对其进行修改。

致谢:Adrian Rosebrock 博客