TensorFlow 示例但有中间层

Question

我正在尝试让这段代码工作。它可能看起来不像，但它主要来自 TensorFlow mnist 示例。不过，我正在尝试获得三层，并且我已经更改了输入和输出大小。输入大小为12，中间大小为6，输出大小为2。这就是我运行这个时候的结果。它不会引发错误，但是当我运行测试选项时，我总是得到 50%。当我回去训练它运行s 时，我确信权重正在改变。有保存模型和权重的代码，所以我非常有信心它不会在我每次重新启动它时清除我的权重。 self.d_y_out 背后的想法是让我可以运行模型并只获得一张图像的输出。我认为问题出在评论 "PROBLEM??".

附近

        self.d_keep = tf.placeholder(tf.float32)
        self.d_W_2 = tf.Variable(tf.random_normal([mid_num, output_num], stddev=0.0001))
        self.d_b_2 = tf.Variable(tf.random_normal([output_num], stddev=0.5))

        self.d_x = tf.placeholder(tf.float32, [None, input_num])
        self.d_W_1 = tf.Variable(tf.random_normal([input_num, mid_num], stddev=0.0001))  # 0.0004
        self.d_b_1 = tf.Variable(tf.zeros([mid_num]))

        self.d_y_ = tf.placeholder(tf.float32, [None, output_num])

        self.d_x_drop = tf.nn.dropout(self.d_x, self.d_keep)

        self.d_y_logits_1 = tf.matmul(self.d_x_drop, self.d_W_1) + self.d_b_1
        self.d_y_mid = tf.nn.relu(self.d_y_logits_1) 
        self.d_y_mid_drop = tf.nn.dropout(self.d_y_mid, self.d_keep)

        self.d_y_logits_2 = tf.matmul(self.d_y_mid_drop, self.d_W_2) + self.d_b_2

        self.d_y_softmax = tf.nn.softmax_cross_entropy_with_logits(logits=self.d_y_logits_2, labels=self.d_y_)

        self.d_cross_entropy = tf.reduce_mean(self.d_y_softmax) ## PROBLEM??

        self.d_train_step = tf.train.GradientDescentOptimizer(0.001).minimize(self.d_cross_entropy)  # 0.0001

        # train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy) #0.5

        #self.d_y_out = tf.argmax(self.d_y, 1)  ## for prediction
        self.d_y_out = tf.argmax(self.d_y_logits_2, 1, name="d_y_out")

    if self.train :

        for i in range(self.start_train,self.cursor_tot): #1000
            batch_xs, batch_ys = self.get_nn_next_train(self.batchsize)
            self.sess.run(self.d_train_step, feed_dict={self.d_x: batch_xs, self.d_y_: batch_ys, self.d_keep: 0.5})
            if True: #mid_num > 0:
                cost = self.sess.run([self.d_cross_entropy, self.d_train_step], 
                    feed_dict={self.d_x: batch_xs, self.d_y_: batch_ys, self.d_keep: 0.5})
                print cost[0], "cost"


    if self.test :
        d_correct_prediction = tf.equal(self.d_y_out, tf.argmax(self.d_y_,1))
        #d_correct_prediction = tf.equal(tf.argmax(self.d_y , 1), tf.argmax(self.d_y_, 1))

        d_accuracy = tf.reduce_mean(tf.cast(d_correct_prediction, tf.float32))

        if self.use_loader : self.get_nn_next_test(self.batchsize)
        print(self.sess.run([d_accuracy, self.d_cross_entropy], 
            feed_dict={self.d_x: self.mnist_test.images, self.d_y_: self.mnist_test.labels, self.d_keep: 1.0}))

    if self.predict_dot :
        for i in range(start, stop ) :
            batch_0, batch_1 = self.get_nn_next_predict(self.batchsize)
            if len(batch_0) > 0 :
                out.extend( self.sess.run([self.d_y_out, self.d_cross_entropy], 
                    feed_dict={self.d_x : batch_0, self.d_y_: batch_1, self.d_keep: 1.0})[0])
                print "out" , len(out) , i, self.cursor_tot, out[:10],"..."

EDIT 我已经对这个问题中的代码进行了大量编辑。非常感谢 vijay m 让我走到这一步。任何帮助，将不胜感激。谢谢

Answer 1

此代码中的问题是您在输入上调用 dropout。你的是单层网络，你不需要 dropout。并使用像 Adam 这样的动量优化器来更快地训练。我所做的更改：

d_y_logits_1 = tf.matmul(d_x, d_W_1) + d_b_1
d_y_mid = tf.nn.relu(d_y_logits_1) 

d_y_logits_2 = tf.matmul(d_y_mid, d_W_2) + d_b_2

d_y_softmax = tf.nn.softmax_cross_entropy_with_logits(logits=d_y_logits_2, labels=d_y_)

d_cross_entropy = tf.reduce_mean(d_y_softmax)

d_train_step = tf.train.AdamOptimizer(0.01).minimize(d_cross_entropy)

TensorFlow 示例但有中间层

TensorFlow example but with middle layer

python

mnist

tensorflow