如何将先前的时间戳预测作为下一个时间戳的附加输入？

Question

这个问题可能有人问过，但我很困惑。

我正在尝试应用一种 RNN 类型，例如用于时间序列预测的 LSTM。我有输入，y（股票 returns）。对于每个时间戳，我想得到预测。 Q1 - 我选择 seq2seq 方法是否正确？

我还想使用来自先前时间戳的预测（使用一些常数初始化初始值）作为附加（仍然使用我现有的输入）平方残差形式的输入，即使用 eps_{t-1} = (y_{t-1} - y^_{t-1})^2 作为 t 处的附加输入（以及之前的输入）。

那么，我如何在 tensorflow 或 pytorch 中执行此操作？

我试图在附图上描绘出我想要的东西。 The graph

p.s。抱歉，问题表述不当

Answer 1

假设您输入的维度 (32,10,1) 为 batch_size 32，时间 steps 长度为 10，维度为 1。与您的目标相同（股票 return).此代码使用 tf.scan 函数，这在实现自定义循环网络时很有用（它将遍历时间 ps）。如您所愿，仍然可以在某处使用 t 中的 t-1 的残差。

ps：它是从零开始的一个非常基本的lstm实现，没有任何偏差或输出激活。

import tensorflow as tf
import numpy as np
tf.reset_default_graph()

BS = 32
TS = 10
inputs_dim = 1
target_dim = 1

inputs = tf.placeholder(shape=[BS, TS, inputs_dim], dtype=tf.float32)
stock_returns = tf.placeholder(shape=[BS, TS, target_dim], dtype=tf.float32)

state_size = 16

# initial hidden state
init_state = tf.placeholder(shape=[2, BS, state_size], 
                        dtype=tf.float32, name='initial_state')
# initializer
xav_init = tf.contrib.layers.xavier_initializer
# params
W = tf.get_variable('W', shape=[4, state_size, state_size], 
                initializer=xav_init())
U = tf.get_variable('U', shape=[4, inputs_dim, state_size], 
                initializer=xav_init())
W_out = tf.get_variable('W_out', shape=[state_size, target_dim], 
                   initializer=xav_init())

#the function to feed tf.scan with
def step(prev, inputs_):

   #unpack all inputs and previous outputs
   st_1, ct_1 = prev[0][0], prev[0][1]
   x = inputs_[0]
   target = inputs_[1]

   #get previous squared residual
   eps = prev[1]


   """
   here do whatever you want with eps_t-1
   like x += eps if x if of the same dimension
   or include it somewhere in your graph
   """


   # lstm gates (add bias if needed)
   #
   #  input gate
   i = tf.sigmoid(tf.matmul(x,U[0]) + tf.matmul(st_1,W[0]))
   #  forget gate
   f = tf.sigmoid(tf.matmul(x,U[1]) + tf.matmul(st_1,W[1]))
   #  output gate
   o = tf.sigmoid(tf.matmul(x,U[2]) + tf.matmul(st_1,W[2]))
   #  gate weights
   g = tf.tanh(tf.matmul(x,U[3]) + tf.matmul(st_1,W[3]))

   ct = ct_1*f + g*i
   st = tf.tanh(ct)*o

   """
   make prediction, compute residual in t
   and pass it to t+1
   Normaly, we would compute prediction outside the scan function, 
   but as we need it here, we could just keep it and return it back
   as an output of the scan function 
   """

   prediction_t = tf.matmul(st, W_out) # + bias
   eps = (target - prediction_t)**2 

   return [tf.stack((st, ct), axis=0), eps, prediction_t] 

states, eps, preds = tf.scan(step, [tf.transpose(inputs, [1,0,2]), 
       tf.transpose(stock_returns, [1,0,2])], initializer=[init_state, 
       tf.zeros((32,1), dtype=tf.float32), 
       tf.zeros((32,1),dtype=tf.float32)])


with tf.Session() as sess:
  sess.run(tf.global_variables_initializer())
  out = sess.run(preds, feed_dict= 
               {inputs:np.random.rand(BS,TS,inputs_dim),                                      
                stock_returns:np.random.rand(BS,TS,target_dim),                                    
                init_state:np.zeros((2,BS,state_size))})
  out = tf.transpose(out,[1,0,2])
  print(out)

输出：

Tensor("transpose_2:0", shape=(32, 10, 1), dtype=float32)

来自 here

的基本代码

如何将先前的时间戳预测作为下一个时间戳的附加输入？

How to feed previous time-stamp prediction as additional input to the next time-stamp?

time-series

lstm

recurrent-neural-network

forecasting