DL4J中如何获取CNN网络卷积层的filter数据绘制激活图？

Question

如何从像这样的配置和模型的图层对象中获取过滤器数据？

  ComputationGraphConfiguration config =
        new NeuralNetConfiguration.Builder()
            .seed(seed)
            .gradientNormalization(GradientNormalization.RenormalizeL2PerLayer)
            .l2(1e-3)
            .updater(new Adam(1e-3))
            .weightInit(WeightInit.XAVIER_UNIFORM)
            .graphBuilder()
            .addInputs("trainFeatures")
            .setInputTypes(InputType.convolutional(60, 200, 3))
            .setOutputs("out1", "out2", "out3", "out4", "out5", "out6")
            .addLayer(
                "cnn1",
                new ConvolutionLayer.Builder(new int[] {5, 5}, new int[] {1, 1}, new int[] {0, 0})
                    .nIn(3)
                    .nOut(48)
                    .activation(Activation.RELU)
                    .build(),
                "trainFeatures")
            .addLayer(
                "maxpool1",
                new SubsamplingLayer.Builder(
                        PoolingType.MAX, new int[] {2, 2}, new int[] {2, 2}, new int[] {0, 0})
                    .build(),
                "cnn1")
            .addLayer(
                "cnn2",
                new ConvolutionLayer.Builder(new int[] {5, 5}, new int[] {1, 1}, new int[] {0, 0})
                    .nOut(64)
                    .activation(Activation.RELU)
                    .build(),
                "maxpool1")
            .addLayer(
                "maxpool2",
                new SubsamplingLayer.Builder(
                        PoolingType.MAX, new int[] {2, 1}, new int[] {2, 1}, new int[] {0, 0})
                    .build(),
                "cnn2")
            .addLayer(
                "cnn3",
                new ConvolutionLayer.Builder(new int[] {3, 3}, new int[] {1, 1}, new int[] {0, 0})
                    .nOut(128)
                    .activation(Activation.RELU)
                    .build(),
                "maxpool2")
            .addLayer(
                "maxpool3",
                new SubsamplingLayer.Builder(
                        PoolingType.MAX, new int[] {2, 2}, new int[] {2, 2}, new int[] {0, 0})
                    .build(),
                "cnn3")
            .addLayer(
                "cnn4",
                new ConvolutionLayer.Builder(new int[] {4, 4}, new int[] {1, 1}, new int[] {0, 0})
                    .nOut(256)
                    .activation(Activation.RELU)
                    .build(),
                "maxpool3")
            .addLayer(
                "maxpool4",
                new SubsamplingLayer.Builder(
                        PoolingType.MAX, new int[] {2, 2}, new int[] {2, 2}, new int[] {0, 0})
                    .build(),
                "cnn4")
            .addLayer("ffn0", new DenseLayer.Builder().nOut(3072).build(), "maxpool4")
            .addLayer("ffn1", new DenseLayer.Builder().nOut(3072).build(), "ffn0")
            .addLayer(
                "out1",
                new OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
                    //.nOut(36)
                        .nOut(10)
                    .activation(Activation.SOFTMAX)
                    .build(),
                "ffn1")
            .addLayer(
                "out2",
                new OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
                    //.nOut(36)
                        .nOut(10)
                    .activation(Activation.SOFTMAX)
                    .build(),
                "ffn1")
            .addLayer(
                "out3",
                new OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
                    //.nOut(36)
                        .nOut(10)
                    .activation(Activation.SOFTMAX)
                    .build(),
                "ffn1")
            .addLayer(
                "out4",
                new OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
                    //.nOut(36)
                        .nOut(10)
                    .activation(Activation.SOFTMAX)
                    .build(),
                "ffn1")
            .addLayer(
                "out5",
                new OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
                    //.nOut(36)
                        .nOut(10)
                    .activation(Activation.SOFTMAX)
                    .build(),
                "ffn1").addLayer(
                "out6",
                new OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
                        //.nOut(36)
                        .nOut(10)
                        .activation(Activation.SOFTMAX)
                        .build(),
                "ffn1")

            //.pretrain(false)
            //.backprop(true)
            .build();

我的意思是训练模型后卷积层激活的 NDArray（或什么？），用于绘制这样的激活图：

我不清楚什么样的 Layer API returns 用于构建那个的 2D 数据。

Answer 1

如果您使用的是 DL4J ui 模块，只需将 ConvolutionalIterationListener 添加为模型的另一个侦听器即可获得这些可视化效果。

如果您不想使用侦听器，您至少可以查看其代码以了解如何自行创建这些可视化效果。

Answer 2

您可以遍历图中的每一层和顶点以隔离所有 Type.CONVOLUTIONAL 层，然后打印过滤器偏差和权重。

艰难，正如@PaulDubs 所指出的，这会给你权重而不是激活：你必须申请图像的过滤器以具体查看该图像的激活。

ComputationGraph model = ComputationGraph.load(new File("/data/deepl/moyens11.zip"), false);
GraphVertex[] vertices = model.getVertices();
for (Layer layer : model.getLayers()) {
    if (layer.type() != Type.CONVOLUTIONAL) {
        continue;
    }
    
    String name = Arrays.stream(vertices)
        .filter(v -> v.getLayer() == layer)
        .map(GraphVertex::getVertexName)
        .findAny().orElse("???");
    System.out.println("Layer #"+layer.getIndex()+" "+name+":");
    Map<String,INDArray> params = layer.paramTable();
    INDArray coeffsW = params.get("W"); // Weights
    INDArray coeffsB = params.get("b"); // Bias
    long[] dimsW = coeffsW.shape(); // [<nOut>, <nIn>, <convY>, <convX>] e.g. [32, 1, 3, 3]
    long[] dimsB = coeffsB.shape(); // [1, <nOut>] e.g. [1, 32]
    for (int iOut = 0; iOut < dimsW[0]; iOut++) {
        System.out.println("      Output "+iOut+": "+dimsW[2]+"x"+dimsW[3]);
        for (int iIn = 0; iIn < dimsW[1]; iIn++) {
            System.out.println("         Filter "+iIn+", bias "+coeffsB.getFloat(0, iOut));
            for (int y = 0; y < dimsW[3]; y++) {
                System.out.print("            ");
                for (int x = 0; x < dimsW[2]; x++) {
                    System.out.printf("%15g", coeffsW.getFloat(iOut, iIn, y, x));
                }
                System.out.println();
            }
        }
    }
}

会给出类似的东西：

Layer #2 cnn1:
   Output 0: 3x3
      Filter 0, bias 0.034268413
               -0.181560      -0.107202      -0.143127
               -0.105426      0.0311019      -0.104243
               -0.114821      -0.101115      -0.105325
   Output 1: 3x3
      Filter 0, bias 0.023474146
             -0.00397262      0.0207015       0.253023
               -0.171297     -0.0244920      0.0446173
               -0.103351      -0.107709    -0.00905741
...

DL4J中如何获取CNN网络卷积层的filter数据绘制激活图？

How to obtain data of filters of the convolution layers of the CNN network in DL4J to draw the activation map?

java

deep-learning

deeplearning4j

activation-function

dl4j