从 deeplearning4j 层提取特征

Question

我正在尝试提取层激活以将它们作为特征保存在本地。我对 CNN 还是个新手，所以我想展示一下我所做的，我想知道我所做的是否正确：

public static void main(String[] args) throws IOException {
    ComputationGraph vgg16transfer = getComputationGraph();

    for (File file : new File(ImageClassifier.class.getClassLoader().getResource("mydirectory").getFile()).listFiles()) {
        Map<String, INDArray> stringINDArrayMap = extractTwo(file, vgg16transfer);
        //Extract the features from the last fully connected layers
        saveCompressed(file,stringINDArrayMap.get("fc2"));
    }
}

/**
 * Retrieves the VGG16 computation graph
 * @return ComputationGraph from the pretrained VGG16
 * @throws IOException
 */
public static ComputationGraph getComputationGraph() throws IOException {
    ZooModel zooModel = new VGG16();
    return (ComputationGraph) zooModel.initPretrained(PretrainedType.IMAGENET);
}

/**
 * Compresses the input INDArray and writes it to file
 * @param imageFile the original image file
 * @param array INDArray to be saved (features)
 * @throws IOException
 */
private static void saveCompressed(File imageFile, INDArray array) throws IOException {
    INDArray compress = BasicNDArrayCompressor.getInstance().compress(array);
    Nd4j.write(compress,new DataOutputStream(new FileOutputStream(new File("features/" + imageFile.getName()+ "feat"))));
}

/**
 * Given an input image and a ComputationGraph it calls the feedForward method after rescaling the image.
 * @param imageFile the image whose features need to be extracted 
 * @param vgg16 the ComputationGraph to be used.
 * @return a map of activations for each layer
 * @throws IOException
 */
public static Map<String, INDArray> extractTwo(File imageFile, ComputationGraph vgg16) throws IOException {
    // Convert file to INDArray
    NativeImageLoader loader = new NativeImageLoader(224, 224, 3);
    INDArray image = loader.asMatrix(imageFile);

    // Mean subtraction pre-processing step for VGG
    DataNormalization scaler = new VGG16ImagePreProcessor();
    scaler.transform(image);

    //Call the feedForward method to get a map of activations for each layer
    return vgg16.feedForward(image, false);
}

所以基本上我是在调用 feedForward 方法并从 fc2 层获取激活值。

我对此有几个问题：

1) 我编写的代码是否确实提取了可以保存和存储以供进一步使用的特征？

2) 我将如何对提取的特征进行 PCA/Whitening？

3) 有什么方法可以按照建议将其编码为 VLAD 但是这样的论文：https://arxiv.org/pdf/1707.00058.pdf

4) 然后我想比较保存的特征，我使用简单的欧几里德距离这样做，虽然结果不是最好的，但它似乎有效。我应该做某种预处理还是保存的特征可以直接比较？

谢谢。

Answer 1

对于提取的特征，Nd4j 有一个 PCA，您可以使用： https://github.com/deeplearning4j/nd4j/blob/master/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/linalg/dimensionalityreduction/PCA.java

虽然有点多余。我会考虑直接使用迁移学习。您不必手动执行任何操作。有关更多示例，请参阅此处的文档： https://github.com/deeplearning4j/dl4j-examples/blob/master/dl4j-spark-examples/dl4j-spark/src/main/java/org/deeplearning4j/transferlearning/vgg16/TransferLearning.md

迁移学习 api 将为您提供使用预训练模型所需的内容，同时修改它们仅使用不同的输出层。我们甚至在上面的例子中用 VGG16 覆盖了这一点。

从 deeplearning4j 层提取特征

Extracting features from deeplearning4j layer

java

multidimensional-array

neural-network

deep-learning

deeplearning4j