LSTM 输出扁平线

Question

我一直在尝试制作一个简单的 LSTM 网络来预测 S&P500 接下来 5 个值的百分比变化。然而，我的神经网络几乎输出了一条完全平坦的线。

5% 的未来变化，红色是“预测”：

我知道我永远不应该在训练集中检查我的模型，但这只是一个完整性检查，以确定它是否有效完全。

sc = MinMaxScaler(feature_range=(0,1))
dataset = dataset[5:-1]
dataset = dataset.dropna()

close = sc.fit_transform(dataset['Close'].values.reshape(-1,1))
volume = sc.fit_transform(dataset['Volume'].values.reshape(-1,1))
pct = sc.fit_transform(dataset['pct5'].values.reshape(-1,1))

close_train = []
volume_train = []
y = []
pc = []
leng = 60

for i in range(leng, len(close)):
  close_train.append(close[i - 60 : i, 0])
  volume_train.append(volume[i - 60 : i, 0])
  y.append(close[i, 0])
  pc.append(pct[i, 0])


close_train = np.array(close_train, dtype=np.float64)
volume_train = np.array(volume_train, dtype=np.float64)
y = np.array(y)
pc = np.array(pc, dtype=np.float64) #This is just adjusted pct in case you got lost

close_train = np.reshape(close_train, (close_train.shape[0], close_train.shape[1], 1))
volume_train = np.reshape(volume_train, (volume_train.shape[0], volume_train.shape[1], 1))


def buildModel(dataLength, labelLength):
  price = Input(shape=(dataLength, 1), name='price')
  volumen = Input(shape=(dataLength, 1), name='volumen')

  priceLayers1 = LSTM(60, return_sequences=True)(price)
  volumeLayers1 = LSTM(60, return_sequences=True)(volumen)

  priceLayers2 = LSTM(60, return_sequences=True)(price)
  volumeLayers2 = LSTM(60, return_sequences=True)(volumen)

  priceLayers3 = LSTM(60, return_sequences=False)(price)
  volumeLayers3 = LSTM(60, return_sequences=False)(volumen)

  output = concatenate(
      [
       price,
       volumen
      ]
  )

  output = Dense(1, activation='linear', name='dense')(output)

  model = Model(
      [
       price,
       volumen
      ],
      [
       output
      ]
  )
  opt = tf.keras.optimizers.Adam(learning_rate=0.001)
  model.compile(optimizer=opt, loss='mse')
  print(output)
  return model


rnn = buildModel(60, 4)
hist = rnn.fit(
    [
     close_train,
     volume_train
    ],
    [
     pc
    ],
    epochs = 100,
    batch_size=50
)

nsamples, nx, ny = close_train.shape
test_close = close_train.reshape((nsamples,nx*ny))

test_vol = volume_train.reshape((nsamples,nx*ny))

pred = rnn.predict([test_close[0, :60], test_vol[0, :60]])
print(pred_dim)
pred1 = sc.inverse_transform(pred_dim)


final = []
for i in range(0, len(pred1)+60):
  if i <60:
    final.append(None)
    continue
  final.append(pred1[i-60, 0])

plt.figure(figsize=(30,20))
plt.plot(dataset['pct5'])
plt.plot(final, c='r')
plt.axvline(60, c='r')

print(final)

PS：我不希望它能准确工作，因为这实际上是不可能的，但我希望它能正常工作，这样我就可以继续前进了:)

Answer 1

您目前在问题中显示的模型是输入的线性回归。

即

output = Dense(1, activation='linear', name='dense')(concatenate[price, volume])

我怀疑是否有可能让模型更好地预测平均值...它可能将权重设置为尽可能接近 0 并将偏差设置为信号的平均值。

你可能想写这样的东西：

def buildModel(dataLength, labelLength):
  price = Input(shape=(dataLength, 1), name='price')
  volumen = Input(shape=(dataLength, 1), name='volumen')

  priceLayers1 = LSTM(60, return_sequences=True)(price)
  volumeLayers1 = LSTM(60, return_sequences=True)(volumen)

  priceLayers2 = LSTM(60, return_sequences=True)(priceLayers1)
  volumeLayers2 = LSTM(60, return_sequences=True)(volumeLayers1)

  priceLayers3 = LSTM(60, return_sequences=False)(priceLayers2)
  volumeLayers3 = LSTM(60, return_sequences=False)(volumeLayers2)

  x = Concatenate()([priceLayers3, volumeLayers3])
  output = Dense(1, activation='linear', name='dense')(x)
  model = tf.keras.Model([price, volumen], output)
  model.compile(optimizer='adam', loss='mse')
  return model

LSTM 输出扁平线

LSTM outputs flat line

python

machine-learning

lstm

keras