为卷积网络的每一层激活层添加可视化 - Keras

Question

我有一个卷积网络 (CNN)，如下所示。我想为每一层激活层添加可视化，如

有几层 CNN 正在执行所需的任务。我只想探测每一层的输出。

def get_model():
    input_shape = (IMG_MODE, img_rows, img_cols)
    model = Sequential()

    model.add(ZeroPadding2D(padding=(1,1), input_shape=input_shape))
    model.add(Conv2D(32, (3, 3), padding = 'valid'))
    model.add(LeakyReLU(alpha=0.01))
    model.add(MaxPooling2D(pool_size=pool_size2))

    ....

    model.add(Dense(nb_classes))
    model.add(Activation('softmax'))

    print(model.summary())
    return model

代码输出：

_________________________________________________________________


   Layer (type)                 Output Shape              Param #   
    =================================================================
    zero_padding2d_1 (ZeroPaddin (None, 1, 114, 94)        0         
    _________________________________________________________________
    conv2d_1 (Conv2D)            (None, 32, 112, 92)       320       
    _________________________________________________________________
    leaky_re_lu_1 (LeakyReLU)    (None, 32, 112, 92)       0         
    _________________________________________________________________
    max_pooling2d_1 (MaxPooling2 (None, 32, 56, 46)        0         
    _________________________________________________________________
    ....        
    _________________________________________________________________
    dense_1 (Dense)              (None, 1024)              8258560   
    _________________________________________________________________
    leaky_re_lu_4 (LeakyReLU)    (None, 1024)              0         
    _________________________________________________________________
    dropout_1 (Dropout)          (None, 1024)              0         
    _________________________________________________________________
    dense_2 (Dense)              (None, 40)                41000     
    _________________________________________________________________
    activation_1 (Activation)    (None, 40)                0         
    =================================================================
    Total params: 8,392,232
    Trainable params: 8,392,232
    Non-trainable params: 0
    _________________________________________________________________
    None
    Train on 320 samples, validate on 80 samples
    Epoch 1/20
     - 18s - loss: 3.7036 - acc: 0.0187 - val_loss: 3.6824 - val_acc: 0.0250
    Epoch 2/20
     - 17s - loss: 3.6903 - acc: 0.0250 - val_loss: 3.6786 - val_acc: 0.0250
    ...
    Epoch 20/20
     - 17s - loss: 0.2067 - acc: 0.9312 - val_loss: 0.9892 - val_acc: 0.7625
    Test score: 0.9891735315322876
    Test accuracy: 0.7625

我尝试使用以下代码来完成我的任务：

 import matplotlib.pyplot as plt
    from keras import models
    layer_outputs = [layer.output for layer in model.layers[:8]]
    activation_model = models.Model(inputs=model.input, outputs=layer_outputs)

    activations = activation_model.predict(img_tensor)

    import matplotlib.pyplot as plt
    plt.matshow(first_layer_activation[0, :, :, 7], cmap='viridis')
    layer_names = []
    for layer in model.layers[:8]:
    layer_names.append(layer.name)
    images_per_row = 16
    for layer_name, layer_activation in zip(layer_names, activations):
    n_features = layer_activation.shape[-1]
    size = layer_activation.shape[1]
    n_cols = n_features // images_per_row
    display_grid = np.zeros((size * n_cols, images_per_row * size))
    for col in range(n_cols):
    for row in range(images_per_row):
    channel_image = layer_activation[0,
    :, :,
    col * images_per_row + row]
    channel_image -= channel_image.mean()
    channel_image /= channel_image.std()
    channel_image *= 64
    channel_image += 128
    channel_image = np.clip(channel_image, 0, 255).astype('uint8')
    display_grid[col * size : (col + 1) * size,
    row * size : (row + 1) * size] = channel_image
    scale = 1. / size
    plt.figure(figsize=(scale * display_grid.shape[1],
    scale * display_grid.shape[0]))
    plt.title(layer_name)
    plt.grid(False)
    plt.imshow(display_grid, aspect='auto', cmap='viridis')

Answer 1

我描述了一种通用方法，如何获取任何模型的卷积层的输出以及如何将它们可视化。我将使用 TensorFlow 的 Keras，其他 Keras 实现的代码可能略有不同。首先需要一个函数来获取模型卷积层的输出

def getConvOutput(model,index=-1):
    # index=-1 means last convolutional layer
    layers = model.layers
    return [layer.output for layer in layers if type(layer) is tf.keras.layers.Conv2D][index]

这将是一个四维张量 (batch_size,height,width,number_of_channels)

接下来我们需要一个函数，它为卷积层的输出构建单个数据元素（单个图像）的网格。它将构造接近正方形的图像，这将是激活图的网格。

def mapsToGrid(output):
    numMaps = int(output.shape[-1])
    #calculate the number of rows and columns which we want to have
    numColumns = math.ceil(numMaps**0.5)
    numRows = math.ceil(numMaps/numColumns)
    # the end of the grid may be filled with zeros
    zerosNum = numRows*numColumns-numMaps
    zerosShape = [int(i) for i in output.shape]
    zerosShape[-1] = zerosNum
    zeros = tf.zeros(
        zerosShape,
        dtype=tf.float32,
        name=None)
    # extend the activation maps with zeros
    concated = tf.concat([output,zeros],-1)
    len,width,depth= [s for s in concated.shape]
    # unstack the activation maps and construct the grid
    mapStack =tf.unstack(concated,axis=2)
    rowStacks = [tf.concat(mapStack[i:i+numColumns],axis=1) for i in range(0,numColumns*numRows,numColumns)]
    result = tf.concat(rowStacks,axis=0)
    return result

一旦你有了这些功能，你就可以得到如下的网格

activation_map_grid_tensor = mapsToGrid(getConvOutput(模型)[0])

索引 0 是必需的，因为 mapsToGrid 适用于单个图像的激活图，因此我们选择批次的第一个元素。现在您可以评估张量，并用例如cv2.imshow()

这种方法取自 https://github.com/cyberneuron/RT-CNN-Vis，这是一个 CNN 可视化平台。人们可能还会发现直接从那里获取代码更容易。

为卷积网络的每一层激活层添加可视化 - Keras

Adding visualization for every layer activation layer of Convolution Network - Keras

python

keras

convolutional-neural-network