神经元网络无法正确训练

Question

我想创建一个能够从 MINST 数据库中猜测数字的神经网络。我正在使用来自 npmjs.org 的库（npm install --save mnist）

我只是无法让我的神经元网络学习数字，我也尝试了一个基本的异或问题，仍然没有用。有人明白为什么吗？

import * as tf from '@tensorflow/tfjs-node';
import * as mnist from 'mnist';

const ACTIVATION = "sigmoid"

const input = tf.input({
  shape: [784],
});

const denseLayer1 = tf.layers.dense({
  units: 30,
  activation: ACTIVATION
});

const denseLayer2 = tf.layers.dense({
  units: 30,
  activation: ACTIVATION
});

const output = tf.layers.dense({
  units: 10,
  activation: ACTIVATION
});

const model = tf.model({
  inputs: input,
  outputs: output.apply(denseLayer2.apply(denseLayer1.apply(input)))
});

model.compile({
  optimizer: tf.train.sgd(0.1),
  loss: tf.losses.meanSquaredError
});


let coolSet = mnist.set(100, 300);
let inputs = [];
let outputs = [];
coolSet.training.forEach((oneTraining, index) => {
  inputs.push(oneTraining.input);
  outputs.push(oneTraining.output)
});
outputs = tf.tensor2d(outputs);
inputs = tf.tensor2d(inputs);

let testInputs = [];
let testOutputs = [];
coolSet.test.forEach(oneTest => {
  testInputs.push(oneTest.input);
  testOutputs.push(oneTest.output)
});

train().then(() => {
  testInputs.forEach((x, index) => {
    const predictedOutput = model.predict(tf.tensor2d([x]));
    console.log(`Excpected Output: ${testOutputs[index]}
  Output: ${predictedOutput.toString()}`)
  });
});

async function train() {
  for (let i = 0; i < 100; i++) {
    const config = {
      shuffle: true,
      epochs: 10
    };
    const response = await model.fit(inputs, outputs, config);
    console.log(response.history.loss[0]);
  }
}

我无法让 NeuronalNetwork 学习。损失会自行降低。

输出：

Excpected Output: 0,0,1,0,0,0,0,0,0,0
  Output: Tensor
     [[0.0957722, 0.1015996, 0.0940665, 0.0999646, 0.1097334, 0.0965195, 0.1045253, 0.0923973, 0.1008056, 0.1046157],]

15ms 150us/step - loss=0.0889 
Epoch 10 / 10

Answer 1

答案很简单：只要多训练它！我认为 10 个 epochs 和 100 次迭代就足够了，但我尝试了 1000 know，现在它成功了！

Answer 2

这个神经网络有几点需要注意。

要解决的问题是分类问题。这意味着给定一个输入，输出是在不同标签中选择一个类别。输出是一个概率（范围从 0 - 1）。输出的总和应为 1。通常，在分类问题中，最后一层是 softmax 激活，它采用输入层并输出一个分数，表示每个可能类别的概率。

至于损失，最好的选择是binaryCrossentropy或categoricalCrossEntropy。人们并没有真正计算预测输出和预期输出之间的欧几里得距离。与回归问题相比，这里的意义不大。

const ACTIVATION = "sigmoid"

const input = tf.input({
  shape: [784],
});

const denseLayer1 = tf.layers.dense({
  units: 30,
  activation: ACTIVATION
});

const denseLayer2 = tf.layers.dense({
  units: 30,
  activation: ACTIVATION
});

const output = tf.layers.dense({
  units: 10,
  activation: 'softmax'
});

const model = tf.model({
  inputs: input,
  outputs: output.apply(denseLayer2.apply(denseLayer1.apply(input)))
});

model.compile({
  optimizer: 'adam',
  loss: 'categoricalCrossentropy'
});


let coolSet = mnist.set(100, 300);
let inputs = [];
let outputs = [];
coolSet.training.forEach((oneTraining, index) => {
  inputs.push(oneTraining.input);
  outputs.push(oneTraining.output)
});
outputs = tf.tensor(outputs);
inputs = tf.tensor(inputs, [100, 784]);

let testInputs = [];
let testOutputs = [];
coolSet.test.forEach(oneTest => {
  testInputs.push(oneTest.input);
  testOutputs.push(oneTest.output)
});

train().then(() => {
  testInputs.slice(0, 10).forEach((x, index) => {
    const predictedOutput = model.predict(tf.tensor([x]));
    console.log(`Excpected Output: ${testOutputs[index]}
  Output: ${predictedOutput.equal(predictedOutput.max(1)).toString()}`)
  });
});

async function train() {
    const config = {
      shuffle: true,
      epochs: 1000, 
      callbacks: {
        onEpochEnd: async (_, l) => {console.log(l.loss)}
      }
    };
    const response = await model.fit(inputs, outputs, config);
}

<script src="https://cdnjs.cloudflare.com/ajax/libs/mnist/1.1.0/mnist.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@0.14.1/dist/tf.min.js"></script>

神经元网络无法正确训练

Neuronal Network just won't train correctly

node.js

tensorflow

tensorflow.js