使用 RecordReader 从文件中读取 CSV 数据时出错

Question

我想用 RecodrReader 和 DataSetIterator 加载训练数据集表单文件，但我在尝试时遇到错误 java.lang.ExceptionInInitializerError，它什么也没告诉我。

这里是主要逻辑和错误发生的地方：

public void trainNN() throws IOException, InterruptedException {

        String setName = "trainingDataCSV";

        createDataSet(TRAINING_SET_SIZE, setName);

        DataSet allData;
        RecordReader recordReader = new CSVRecordReader(0, ',');
        recordReader.initialize(new FileSplit(new File(setName))); // error here

        int numPixels = IMAGE_SIZE * IMAGE_SIZE;
        DataSetIterator iterator = new RecordReaderDataSetIterator(recordReader, TRAINING_SET_SIZE, numPixels, numPixels + 4);
        allData = iterator.next();


        DataNormalization normalizer = new NormalizerStandardize();
        normalizer.fit(allData);
        normalizer.transform(allData);

        SplitTestAndTrain testAndTrain = allData.splitTestAndTrain(0.65);
        DataSet trainingData = testAndTrain.getTrain();
        DataSet testData = testAndTrain.getTest();


        MultiLayerConfiguration configuration = new NeuralNetConfiguration.Builder()
                .activation(Activation.TANH)
                .weightInit(WeightInit.XAVIER)
//                .learningRate(0.1)
                .l2(0.0001)
                .list()
                .layer(0, new DenseLayer.Builder().nIn(numPixels).nOut(50).build())
                .layer(1, new DenseLayer.Builder().nIn(50).nOut(50).build())
                .layer(2, new OutputLayer.Builder(
                        LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
                        .activation(Activation.SOFTMAX)
                        .nIn(50).nOut(4).build())
                .build();

        MultiLayerNetwork neuralNetwork = new MultiLayerNetwork(configuration);

        INDArray output = neuralNetwork.output(testData.getFeatures());
        Evaluation eval = new Evaluation(3);
        eval.eval(testData.getLabels(), output);

        System.out.println(eval.stats());
    }

完整的错误堆栈跟踪

java.lang.ExceptionInInitializerError
    at org.datavec.api.util.ndarray.RecordConverter.toMinibatchArray(RecordConverter.java:198)
    at org.deeplearning4j.datasets.datavec.RecordReaderMultiDataSetIterator.next(RecordReaderMultiDataSetIterator.java:160)
    at org.deeplearning4j.datasets.datavec.RecordReaderDataSetIterator.next(RecordReaderDataSetIterator.java:377)
    at org.deeplearning4j.datasets.datavec.RecordReaderDataSetIterator.next(RecordReaderDataSetIterator.java:452)
    at org.deeplearning4j.datasets.datavec.RecordReaderDataSetIterator.next(RecordReaderDataSetIterator.java:85)
    at App.trainNN(App.java:116)
    at App.setup(App.java:59)
    at processing.core.PApplet.handleDraw(PApplet.java:2412)
    at processing.awt.PSurfaceAWT.callDraw(PSurfaceAWT.java:1557)
    at processing.core.PSurfaceNone$AnimationThread.run(PSurfaceNone.java:316)
Caused by: java.lang.RuntimeException: org.nd4j.linalg.factory.Nd4jBackend$NoAvailableBackendException: Please ensure that you have an nd4j backend on your classpath. Please see: http://nd4j.org/getstarted.html
    at org.nd4j.linalg.factory.Nd4j.initContext(Nd4j.java:5131)
    at org.nd4j.linalg.factory.Nd4j.<clinit>(Nd4j.java:226)
    ... 10 more
Caused by: org.nd4j.linalg.factory.Nd4jBackend$NoAvailableBackendException: Please ensure that you have an nd4j backend on your classpath. Please see: http://nd4j.org/getstarted.html
    at org.nd4j.linalg.factory.Nd4jBackend.load(Nd4jBackend.java:218)
    at org.nd4j.linalg.factory.Nd4j.initContext(Nd4j.java:5128)
    ... 11 more

以下是数据文件的创建方式，以防万一。
它是 400 个值，包含 1.0 或 0.0，用分号分隔，还有 4 个附加值，用于图像上形状的位置和尺寸

    public void createDataSet(int amount, String name) throws IOException {
        BufferedWriter writer = new BufferedWriter(new FileWriter(name));
        writer.write("");


        for (int i = 0; i < amount; i++) {
            Image image = new Image(IMAGE_SIZE, MIN_OBJECT_SIZE, MAX_OBJECT_SIZE, OBJECTS_AMOUNT);
            int positionX = image.getPosition()[0];
            int positionY = image.getPosition()[1];
            int w = image.getW();
            int h = image.getH();

            for (int pixel = 0; pixel < image.getPixelValues().length; pixel++) {
                writer.append(image.getPixelValues()[pixel] + ",");
            }

            writer.append(positionX + ",");
            writer.append(positionY + ",");
            writer.append(w + ",");
            writer.append(String.valueOf(h) + '\n');

        }

        writer.close();

    }

Answer 1

异常的重要部分是：

Caused by: org.nd4j.linalg.factory.Nd4jBackend$NoAvailableBackendException: Please ensure that you have an nd4j backend on your classpath. Please see: http://nd4j.org/getstarted.html

并且您可能会在初始化 ND4J 时遇到相同的错误（例如 Nd4j.zeros(1)）。

确保您的依赖项中有合适的 ND4J 后端，如文档中所示：https://deeplearning4j.konduit.ai/config/maven#add-a-backend

使用 RecordReader 从文件中读取 CSV 数据时出错

Error while reading CSV data from file with RecordReader

java

neural-network

deep-learning

deeplearning4j