我的神经网络没有学习成本没有变化

Question

我正在构建一个神经网络来对我从中获取数据的 mnist 数字进行分类我试图只用 tensorflow 构建它我不想使用 keras https://www.kaggle.com/c/digit-recognizer/data 并且通过 epochs 成本没有减少

我遵循了那个教程 https://adventuresinmachinelearning.com/python-tensorflow-tutorial/ 我的代码和他的代码之间的区别在于，我没有从 tensorflow 导入数据，而是替换了已弃用的函数注意：我正在使用 anaconda 和 tensorflow==1.14

这是我的代码

import tensorflow as tf
import pandas as pd
# from tensorflow import keras
# import matplotlib.pyplot as plt
import numpy as np

tr_csv = "file:///D:/code/ml/neuralnet/train.csv"
data = pd.read_csv(tr_csv)

data  = data[:30000]
train = data[:24000]
test = data[24000:]

learning_rate = 1
epochs = 10
batch_size = 100

xtf = tf.compat.v1.placeholder(tf.float32, [None, 784])
ytf = tf.compat.v1.placeholder(tf.float32, [None, 10])

W1 = tf.Variable(tf.random.normal([784, 300], stddev=0.03), name='W1')
b1 = tf.Variable(tf.random.normal([300]), name='b1')
W2 = tf.Variable(tf.random.normal([300, 10], stddev=0.03), name='W2')
b2 = tf.Variable(tf.random.normal([10]), name='b2')

hidden1 = tf.add(tf.matmul(xtf,W1),b1)
hidden1 = tf.nn.relu(hidden1)

y_ = tf.nn.softmax(tf.add(tf.matmul(hidden1,W2),b2))
yclipped =  tf.clip_by_value(y_, 1e-10, 0.9999999)

cross_entropy = -tf.reduce_mean(tf.reduce_sum(ytf * tf.log(yclipped)+ (1 - ytf) * tf.log(1 - yclipped), axis=1))
optimiser = tf.compat.v1.train.GradientDescentOptimizer(learning_rate=learning_rate).minimize(cross_entropy)

correct_prediction = tf.equal(tf.argmax(ytf, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

with tf.compat.v1.Session() as sess :
    init_op = tf.compat.v1.global_variables_initializer()
    sess.run(init_op)
    total_batch = int(len(train.label) / batch_size)
    for epoch in range(10):
        avg_cost = 0
        for chunk in np.array_split(train, 24000/batch_size):
   #     for i in range(total_batch)
            xb = chunk.drop(["label"],axis = 1 )
            yb = chunk.label
            yb = pd.Categorical(yb)
            yb = pd.get_dummies(yb,prefix = "number")
            #(batch_size = batch_size) x

            (_, c) = sess.run([optimiser, cross_entropy],feed_dict={xtf : xb.to_numpy() ,ytf: yb.to_numpy()})

            avg_cost += c / batch_size
        print("Epoch:", (epoch + 1), "cost =", "{:.3f}".format(avg_cost))
        #print(sess.run(accuracy, feed_dict={x: xt., y: mnist.test.labels}))

这是历时的成本



Epoch: 1 cost = 84.110
Epoch: 2 cost = 84.109
Epoch: 3 cost = 84.109
Epoch: 4 cost = 84.109
Epoch: 5 cost = 84.109
Epoch: 6 cost = 84.109
Epoch: 7 cost = 84.109
Epoch: 8 cost = 84.109
Epoch: 9 cost = 84.109
Epoch: 10 cost = 84.109

Answer 1

对于任何神经网络，调参都是必不可少的。您可以尝试不同的组合来得出合适的值。成本函数给出了是否达到收敛的想法。如果不收敛，尝试一组新的参数。

我的神经网络没有学习成本没有变化

my neural network is not learning cost is not changing

python

machine-learning

neural-network

mnist

tensorflow

这是我的代码

这是历时的成本