LSTM 模型误差是一个输出的百分比 class

Question

我很难弄清楚我的 LSTM 模型有什么问题。我有 11 个输入和 2 个输出 classes（单热编码）并且非常快，就像在 1 个左右的批次内，错误只是达到输出 classes 之一的百分比并且留在那里。

我尝试打印权重和偏差，但它们似乎全是 NaN。

如果我降低学习率，或者弄乱 layers/units，我可以让它慢慢地达到 % of one class 错误，但它似乎总是达到那个点.

代码如下：

num_units = 30
num_layers = 50
dropout_rate = 0.80
learning_rate=0.0001
batch_size = 180
epoch = 1

input_classes = len(train_input[0])
output_classes = len(train_output[0])

data = tf.placeholder(tf.float32, [None, input_classes, 1]) #Number of examples, number of input, dimension of each input
target = tf.placeholder(tf.float32, [None, output_classes]) #one-hot encoded: [1,0] = bad, [0,1] = good
dropout = tf.placeholder(tf.float32)

cell = tf.contrib.rnn.LSTMCell(num_units, state_is_tuple=True)
cell = tf.contrib.rnn.DropoutWrapper(cell, output_keep_prob=dropout)
cell = tf.contrib.rnn.MultiRNNCell([cell] * num_layers, state_is_tuple=True)

#Input shape [batch_size, max_time, depth], output shape: [batch_size, max_time, cell.output_size]
val, _ = tf.nn.dynamic_rnn(cell, data, dtype=tf.float32) 

val = tf.transpose(val, [1, 0, 2]) #reshapes it to [sequence_size, batch_size, depth]

#get last entry as it includes previous results
last = tf.gather(val, int(val.get_shape()[0]) - 1)

weight = tf.get_variable("W", shape=[num_units, output_classes], initializer=tf.contrib.layers.xavier_initializer())
bias   = tf.get_variable("B", shape=[output_classes], initializer=tf.contrib.layers.xavier_initializer())
logits = tf.matmul(last, weight) + bias

prediction = tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=target)
prediction = tf.clip_by_value(prediction, 1e-10,100.0)

cost = tf.reduce_mean(prediction)

optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)
minimize = optimizer.minimize(cost)

mistakes = tf.not_equal(tf.argmax(target, 1), tf.argmax(logits, 1))
error = tf.reduce_mean(tf.cast(mistakes, tf.float32))

init_op = tf.global_variables_initializer()
saver = tf.train.Saver()
sess = tf.Session()
sess.run(init_op)

no_of_batches = int((len(train_input)) / batch_size)
for i in range(epoch):
    ptr = 0
    for j in range(no_of_batches):
        inp, out = train_input[ptr:ptr+batch_size], train_output[ptr:ptr+batch_size]
        ptr+=batch_size
        sess.run(minimize,{data: inp, target: out, dropout: dropout_rate })

sess.close()

Answer 1

因为你有一个热编码使用 sparse_softmax_cross_entropy_with_logits 而不是 tf.nn.softmax_cross_entropy_with_logits.

参考这个Whosebug的回答，了解两个函数的区别。

LSTM 模型误差是一个输出的百分比 class

LSTM model error is percent of one output class

neural-network

lstm

tensorflow