为什么我的循环神经网络的损失为零？

Question

我有这个 LSTM，我想输入一些财务数据并获得一组输出。将有三个类，正 return 将产生 [1,0,0] 的输出，零 return 将是 [0,1,0]，负 return 将是 [0,0,1]。它运行了，但是我打印出来的损失全为零，而且准确性很差。你能看看我的代码吗，它只有 80 行。干杯。

import tensorflow as tf
import numpy as np
from tensorflow.python.ops import rnn,rnn_cell
import pandas as pd

df=pd.read_csv('SP500.csv')
df = df[['Adj. Open', 'Adj. High', 'Adj. Low', 'Adj. Close', 'Adj. Volume']]
df['Percentage_Change']=df['Adj. Close'].pct_change()
df['HL_PCT'] = (df['Adj. High'] - df['Adj. Close']) / df['Adj. Close'] * 100
df['PCT_change'] = (df['Adj. Close'] - df['Adj. Open']) / df['Adj. Open'] * 100
df['LL_PCT'] = (df['Adj. Close'] - df['Adj. Low']) / df['Adj. Low'] * 100
df = df[['Adj. Close', 'HL_PCT', 'PCT_change', 'Adj. Volume', 'LL_PCT','Percentage_Change']]
df.fillna(0,inplace=True)
ret_ult=[]
for _ in range(len(df.index)):
    ret = np.zeros(3)
    if df['Percentage_Change'][_]>0:
        ret[0]=1
    elif df['Percentage_Change'][_]==0:
        ret[1] = 1
    elif df['Percentage_Change'][_]<0:
        ret[2] = 1
    ret_ult.append(ret)
    ret = np.zeros(3)
classification=np.roll(np.array(ret_ult),-30)
labels=np.array(classification[:-30])


features_normalized=np.array(df[:-30])

train_x=features_normalized[:int(0.3)*len(features_normalized)]
train_y=labels[:int(0.3)*len(features_normalized)]
test_x=features_normalized[-int(0.3)*len(features_normalized):]
test_y=labels[-int(0.3)*len(features_normalized):]


n_classes=3
batch_size=128
chunk_size=6
n_chunks=1
rnn_size=64
hm_epochs = 50

x=tf.placeholder('float',[None, n_chunks,chunk_size])
y=tf.placeholder('float')

def RNN_neural_network_model(x):
    layer={'weights':tf.Variable(tf.random_normal([rnn_size,n_classes])),
                    'biases':tf.Variable(tf.random_normal([n_classes]))}
    x=tf.transpose(x,[1,0,2])
    x=tf.reshape(x,[-1,chunk_size])
    x=tf.split(x,n_chunks,0)

    lstm_cell=rnn_cell.BasicLSTMCell(rnn_size,state_is_tuple=True)
    outputs, states=rnn.static_rnn(lstm_cell,x,dtype=tf.float32)
    output = tf.matmul(outputs[-1], layer['weights'])+ layer['biases']

    return output

def train_neural_network(x):
    prediction=RNN_neural_network_model(x)
    cost=tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=prediction,labels=y))
    optimizer=tf.train.AdamOptimizer(learning_rate=0.001).minimize(cost)
    with tf.Session() as sess:
        sess.run(tf.initialize_all_variables())
        for epoch in range(hm_epochs):
            epoch_loss=0
            for _ in range(int(len(train_x)/batch_size)):
                i = 0
                while i < len(train_x):
                    start = i
                    end = i + batch_size

                    batch_x = train_x[start:end]
                    batch_y = train_y[start:end]
                    i += batch_size
                    batch_x = batch_x.reshape((-1, n_chunks, chunk_size))

                _,c=sess.run([optimizer,cost],feed_dict={x:batch_x,y:batch_y})
                epoch_loss+=c
            print('Epoch',epoch,'completed out of',hm_epochs,'loss',epoch_loss)

        correct=tf.equal(tf.argmax(prediction,1),tf.argmax(y,1))
        accuracy=tf.reduce_mean(tf.cast(correct,'float'))
        print('Accuracy:',accuracy.eval({x:test_x.reshape((-1,n_chunks,chunk_size)),y:test_y}))
train_neural_network(x)

此时输出如下：

Epoch 0 completed out of 50 loss 0
Epoch 1 completed out of 50 loss 0
Epoch 2 completed out of 50 loss 0
Epoch 3 completed out of 50 loss 0
Epoch 4 completed out of 50 loss 0
Epoch 5 completed out of 50 loss 0
Epoch 6 completed out of 50 loss 0
Epoch 7 completed out of 50 loss 0
Epoch 8 completed out of 50 loss 0
Epoch 9 completed out of 50 loss 0
Epoch 10 completed out of 50 loss 0
Epoch 11 completed out of 50 loss 0
Epoch 12 completed out of 50 loss 0
Epoch 13 completed out of 50 loss 0
Epoch 14 completed out of 50 loss 0
Epoch 15 completed out of 50 loss 0
Epoch 16 completed out of 50 loss 0
Epoch 17 completed out of 50 loss 0
Epoch 18 completed out of 50 loss 0
Epoch 19 completed out of 50 loss 0
Epoch 20 completed out of 50 loss 0
Epoch 21 completed out of 50 loss 0
Epoch 22 completed out of 50 loss 0
Epoch 23 completed out of 50 loss 0
Epoch 24 completed out of 50 loss 0
Epoch 25 completed out of 50 loss 0
Epoch 26 completed out of 50 loss 0
Epoch 27 completed out of 50 loss 0
Epoch 28 completed out of 50 loss 0
Epoch 29 completed out of 50 loss 0
Epoch 30 completed out of 50 loss 0
Epoch 31 completed out of 50 loss 0
Epoch 32 completed out of 50 loss 0
Epoch 33 completed out of 50 loss 0
Epoch 34 completed out of 50 loss 0
Epoch 35 completed out of 50 loss 0
Epoch 36 completed out of 50 loss 0
Epoch 37 completed out of 50 loss 0
Epoch 38 completed out of 50 loss 0
Epoch 39 completed out of 50 loss 0
Epoch 40 completed out of 50 loss 0
Epoch 41 completed out of 50 loss 0
Epoch 42 completed out of 50 loss 0
Epoch 43 completed out of 50 loss 0
Epoch 44 completed out of 50 loss 0
Epoch 45 completed out of 50 loss 0
Epoch 46 completed out of 50 loss 0
Epoch 47 completed out of 50 loss 0
Epoch 48 completed out of 50 loss 0
Epoch 49 completed out of 50 loss 0
Accuracy: 0.529333

Answer 1

原因是因为缩进问题，您的模型只能看到一个批次：

for _ in range(int(len(train_x)/batch_size)):
    i = 0
    while i < len(train_x):
        start = i
        end = i + batch_size

        # These batches never get used, only the last one does!
        batch_x = train_x[start:end]
        batch_y = train_y[start:end]
        i += batch_size
        batch_x = batch_x.reshape((-1, n_chunks, chunk_size))

    _,c = sess.run([optimizer,cost],feed_dict={x:batch_x,y:batch_y})
    epoch_loss+=c
print('Epoch',epoch,'completed out of',hm_epochs,'loss',epoch_loss)

正因为如此，它设法将损失最小化为 0，但在整个训练数据集上的表现非常糟糕，因为它根本没有看到它。

为什么我的循环神经网络的损失为零？

Why is the loss of my recurrent neural network zero?

python

tensorflow

neural-network

lstm

rnn