Tensorflow MomentumOptimizer 问题

Question

当我运行下面的简单代码时，我得到这个错误：

tensorflow.python.framework.errors.FailedPreconditionError: Attempting to use uninitialized value Variable_5/Momentum

此代码适用于 GradientDescentOptimizer，但我遇到 MomentumOptimizer 的错误。请指导我解决它。

这是我的代码：

import tensorflow as tf
import numpy as np
import scipy.io as sio
import h5py
from tensorflow.python.training import queue_runner

maxiter = 200000
display = 1
sess = tf.InteractiveSession()
decay_rate = 0.00005
starter_learning_rate = 0.000009
alpha = 0.00005
init_momentum = 0.9

nnodes1 = 350
nnodes2 = 100
batch_size = 50

train_mat = h5py.File('Basket_train_data_binary.mat')
test_mat = h5py.File('Basket_test_data_binary.mat')

train_mat = train_mat["binary_train"].value
test_mat = test_mat["binary_test"].value

Train = np.transpose(train_mat)
Test = np.transpose(test_mat)

# import the data                                                                                                                                
#from tensorflow.examples.tutorials.mnist import input_data                                                                                      
# placeholders, which are the training data                                                                                                      
x = tf.placeholder(tf.float32, shape=[None,43])
y_ = tf.placeholder(tf.float32, shape=[None])

# define the variables                                                                                                                                                                                                                                                                        
W1 = tf.Variable(tf.zeros([43,nnodes1]))

b1 = tf.Variable(tf.zeros([nnodes1]))

W2 = tf.Variable(tf.zeros([nnodes1,nnodes2]))
b2 = tf.Variable(tf.zeros([nnodes2]))

W3 = tf.Variable(tf.zeros([nnodes2,1]))
b3 = tf.Variable(tf.zeros([1]))

# Passing global_step to minimize() will increment it at each step.                                                                              
global_step = tf.Variable(0, trainable=False)
momentum = tf.Variable(init_momentum, trainable=False)


# initilize the variables                                                                                                                       
sess.run(tf.initialize_all_variables())

# prediction function (just one layer)                                                                                                         

layer1 = tf.nn.sigmoid(tf.matmul(x,W1) + b1)
layer2 = tf.nn.sigmoid(tf.matmul(layer1,W2) + b2)
y = tf.matmul(layer2,W3) + b3

# cost function                                                                                                                                  
cost_function = tf.reduce_sum(tf.square(y_ - y))

l2regularization = tf.reduce_sum(tf.square(W1)) + tf.reduce_sum(tf.square(b1)) + tf.reduce_sum(tf.square(W2)) + tf.reduce_sum(tf.square(b2)) + tf.reduce_sum(tf.square(W3)) + tf.reduce_sum(tf.square(b3))
loss = cost_function + alpha*l2regularization

# define the learning_rate and its decaying procedure.                                                                                           
learning_rate = tf.train.exponential_decay(starter_learning_rate,     global_step,10000, decay_rate, staircase=True)
# define the training paramters and model, gradient model and feeding the function                                                               
#train_step = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss, global_step=global_step)
train_step = tf.train.MomentumOptimizer(learning_rate,0.9).minimize(loss, global_step=global_step)                                              


# evaluation                                                                                                                                     
# it returns 1, if both y and y_ are equal.                                                                                                      
correct_prediction = tf.reduce_sum(tf.square(y_ - y))

# calculate the accuracy                                                                                                                         
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))


# Train the Model for 1000 times. by defining the batch number we     determine that it is sgd                                                       
for i in range(maxiter):
  batch = np.random.randint(0,len(Train),size=batch_size)
  train_step.run(feed_dict={x:Train[batch,0:43], y_:Train[batch,43]})
  if np.mod(i,display) == 0:
    # print tset loss                                                                                                                            
    print "Test", accuracy.eval(feed_dict={x: Test[:,0:43], y_: Test[:,43]})
    # print training loss                                                                                                                        
    print "Train" , sess.run(cost_function,feed_dict={x:     Train[:,0:43], y_: Train[:,43]})

请指导我如何解决这个问题。提前致谢，阿夫欣

Answer 1

在线

# initilize the variables                                                                                                                       
sess.run(tf.initialize_all_variables())

您正在初始化在该行之前声明的所有变量。

您在该行之后声明了优化器（和其他变量），因此优化器使用的变量不受初始化的影响。

在完整的图声明之后（例如：在声明每个变量和操作之后）移动初始化以修复。

TL;DR: 移动

# initilize the variables sess.run(tf.initialize_all_variables())

之后

# calculate the accuracy accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

Tensorflow MomentumOptimizer 问题

Tensorflow MomentumOptimizer issue

momentum

tensorflow