为什么 "Self forcast" 比 "Forecast from input" 差？
Why "Self forcast" is worse than "Forecast from input"?
我尝试将@Daniel Möller 提供的代码实施到我的数据中。这是使用 LSTM 学习的时间序列预测问题。
https://github.com/danmoller/TestRepo/blob/master/TestBookLSTM.ipynb
import numpy as np, pandas as pd, matplotlib.pyplot as plt
from sklearn.preprocessing import MinMaxScaler
from keras.models import Sequential
from keras.layers import LSTM, Dense, TimeDistributed, Bidirectional
from sklearn.metrics import mean_squared_error, accuracy_score
from scipy.stats import linregress
from keras.callbacks import EarlyStopping
fi = 'pollution.csv'
raw = pd.read_csv(fi, delimiter=',')
raw = raw.drop('Dates', axis=1)
print (raw.shape)
scaler = MinMaxScaler(feature_range=(-1, 1))
raw = scaler.fit_transform(raw)
n_rows = raw.shape[0]
n_feats = raw.shape[1]
time_shift = 7
train_size = int(n_rows * 0.8)
train_data = raw[:train_size, :]
test_data = raw[train_size:, :]
x_train = train_data[:-time_shift, :]
x_test = test_data[:-time_shift,:]
x_predict = raw[:-time_shift,:]
y_train = train_data[time_shift:, :]
y_test = test_data[time_shift:,:]
y_predict_true = raw[time_shift:,:]
x_train = x_train.reshape(1, x_train.shape[0], x_train.shape[1])
y_train = y_train.reshape(1, y_train.shape[0], y_train.shape[1])
x_test = x_test.reshape(1, x_test.shape[0], x_test.shape[1])
y_test = y_test.reshape(1, y_test.shape[0], y_test.shape[1])
x_predict = x_predict.reshape(1, x_predict.shape[0], x_predict.shape[1])
y_predict_true = y_predict_true.reshape(1, y_predict_true.shape[0], y_predict_true.shape[1])
print (x_train.shape)
print (y_train.shape)
print (x_test.shape)
print (y_test.shape)
model = Sequential()
model.add(LSTM(64,return_sequences=True,input_shape=(None, n_feats)))
model.add(LSTM(32,return_sequences=True))
model.add(LSTM(n_feats,return_sequences=True))
stop = EarlyStopping(monitor='loss',min_delta=0.000000000001,patience=30)
model.compile(loss='mse', optimizer='Adam')
model.fit(x_train,y_train,epochs=10,callbacks=[stop],verbose=2,validation_data=(x_test,y_test))
newModel = Sequential()
newModel.add(LSTM(64,return_sequences=True,stateful=True,batch_input_shape=(1,None,n_feats)))
newModel.add(LSTM(32,return_sequences=True,stateful=True))
newModel.add(LSTM(n_feats,return_sequences=False,stateful=True))
newModel.set_weights(model.get_weights())
newModel.reset_states()
lastSteps = np.empty((1, n_rows, n_feats))
lastSteps[:,:time_shift] = x_predict[:,-time_shift:]
newModel.predict(x_predict).reshape(1,1,n_feats)
rangeLen = n_rows - time_shift
for i in range(rangeLen):
lastSteps[:,i+time_shift] = newModel.predict(lastSteps[:,i:i+1,:]).reshape(1,1,n_feats)
forecastFromSelf = lastSteps[:,time_shift:,:]
print (forecastFromSelf.shape)
forecastFromSelf = scaler.inverse_transform(forecastFromSelf.reshape(forecastFromSelf.shape[1],forecastFromSelf.shape[2]))
y_predict_true = scaler.inverse_transform(y_predict_true.reshape(y_predict_true.shape[1],y_predict_true.shape[2]))
plt.plot(y_predict_true[:,0], color='b', label='True')
plt.plot(forecastFromSelf[:,0],color='r', label='Predict')
plt.legend()
plt.title("Self forcast (Feat 1)")
plt.show()
newModel.reset_states()
newModel.predict(x_predict)
newSteps = []
for i in range(x_predict.shape[1]):
newSteps.append(newModel.predict(x_predict[:,i:i+1,:]))
forecastFromInput = np.asarray(newSteps).reshape(1,x_predict.shape[1],n_feats)
print (forecastFromInput.shape)
forecastFromInput = scaler.inverse_transform(forecastFromInput.reshape(forecastFromInput.shape[1],forecastFromInput.shape[2]))
plt.plot(y_predict_true[:,0], color='b', label='True')
plt.plot(forecastFromInput[:,0], color='r', label='Predict')
plt.legend()
plt.title("Forecast from input (Feat 1)")
plt.show()
可以通过增加模型层数和 epoch 数来增加预测。
但是，这里的问题是，为什么 "Self forcast" 比 "Forecast from input" 差？
污染数据在这里：https://github.com/sirjanrocky/some-neural-tests-for-study/blob/master/pollution.csv
此代码运行无误。你也可以试试
假设您的数据在第 1000 步结束，并且您没有更多数据。
但是您甚至希望在第 1100 步之前进行预测。如果您没有输入数据，您将不得不依赖预测的输出。
在“自我预测”中，您将无中生有地预测 100 步，没有任何基础
但是每个预测都有一个相关的错误（它并不完美）。当您根据预测进行预测时，您提供了一个有错误的输入并获得了一个有更多错误的输出。这是不可避免的。
根据预测进行预测，根据预测进行预测，根据预测进行预测....这会累积很多错误。
如果你的模型可以很好地进行这种预测，那么你可以肯定地说它已经学到了很多关于序列的知识。
当您根据已知输入进行预测时，您正在做一件更安全的事情。你有真实的数据要输入，真实的数据是没有错误的。
所以你的预测虽然肯定有错误，但并不是“累积”错误。您不是根据错误数据进行预测，而是根据真实数据进行预测。
图片中：
自我预测（误差累积）
true input --> model --> predicted output step 1001 (a little error)
(steps 1 to 1000) |
V
model
|
V
predicted output step 1002 (more error)
|
V
model
|
V
predicted output step 1003 (even more error)
来自输入的预测（误差不累积）
true input --> model --> predicted output step 1001 (a little error)
(steps 1 to 1000)
true input --> model --> predicted output step 1002 (a little error)
(step 1001)
true input --> model --> predicted output step 1003 (a little error)
(step 1002)
预测部分发现错误（自我预测）
如果要预测测试数据比较：
（评论中的数字只是为了我自己的方向，他们假设训练数据长度为1000，测试数据长度为200，总共1200个元素）
lastSteps = np.empty((1,n_rows-train_size,n_feats)) #test size = 200
lastSteps[:,:time_shift] = y_train[:,-time_shift:] #0 to 6 = 993 to 999
newModel.predict(x_train) #predict 999
rangeLen = n_rows - train_size - time_shift
for i in range(rangeLen):
lastSteps[:,i+time_shift] = newModel.predict(lastSteps[:,i:i+1,:]).reshape(1,1,n_feats)
#el 7 (1000) <- pred from el 0 (993)
forecastFromSelf = lastSteps[:,time_shift:,:] #1000 forward
如果要预测结束后的未知数据：
You should train the entire data (train with x_predict, y_predict_true)
lastSteps = np.empty((1,new_predictions + time_shift,n_feats))
lastSteps[:,:time_shift] = y_predict_true[:,-time_shift:] #0 to 6 = 1193 to 1199
newModel.predict(x_predict) #predict 1199
rangeLen = new_predictions
for i in range(rangeLen):
lastSteps[:,i+time_shift] = newModel.predict(lastSteps[:,i:i+1,:]).reshape(1,1,n_feats)
#el 7 (1200) <- pred from el 0 (1193)
forecastFromSelf = lastSteps[:,time_shift:,:] #1200 forward
我尝试将@Daniel Möller 提供的代码实施到我的数据中。这是使用 LSTM 学习的时间序列预测问题。 https://github.com/danmoller/TestRepo/blob/master/TestBookLSTM.ipynb
import numpy as np, pandas as pd, matplotlib.pyplot as plt
from sklearn.preprocessing import MinMaxScaler
from keras.models import Sequential
from keras.layers import LSTM, Dense, TimeDistributed, Bidirectional
from sklearn.metrics import mean_squared_error, accuracy_score
from scipy.stats import linregress
from keras.callbacks import EarlyStopping
fi = 'pollution.csv'
raw = pd.read_csv(fi, delimiter=',')
raw = raw.drop('Dates', axis=1)
print (raw.shape)
scaler = MinMaxScaler(feature_range=(-1, 1))
raw = scaler.fit_transform(raw)
n_rows = raw.shape[0]
n_feats = raw.shape[1]
time_shift = 7
train_size = int(n_rows * 0.8)
train_data = raw[:train_size, :]
test_data = raw[train_size:, :]
x_train = train_data[:-time_shift, :]
x_test = test_data[:-time_shift,:]
x_predict = raw[:-time_shift,:]
y_train = train_data[time_shift:, :]
y_test = test_data[time_shift:,:]
y_predict_true = raw[time_shift:,:]
x_train = x_train.reshape(1, x_train.shape[0], x_train.shape[1])
y_train = y_train.reshape(1, y_train.shape[0], y_train.shape[1])
x_test = x_test.reshape(1, x_test.shape[0], x_test.shape[1])
y_test = y_test.reshape(1, y_test.shape[0], y_test.shape[1])
x_predict = x_predict.reshape(1, x_predict.shape[0], x_predict.shape[1])
y_predict_true = y_predict_true.reshape(1, y_predict_true.shape[0], y_predict_true.shape[1])
print (x_train.shape)
print (y_train.shape)
print (x_test.shape)
print (y_test.shape)
model = Sequential()
model.add(LSTM(64,return_sequences=True,input_shape=(None, n_feats)))
model.add(LSTM(32,return_sequences=True))
model.add(LSTM(n_feats,return_sequences=True))
stop = EarlyStopping(monitor='loss',min_delta=0.000000000001,patience=30)
model.compile(loss='mse', optimizer='Adam')
model.fit(x_train,y_train,epochs=10,callbacks=[stop],verbose=2,validation_data=(x_test,y_test))
newModel = Sequential()
newModel.add(LSTM(64,return_sequences=True,stateful=True,batch_input_shape=(1,None,n_feats)))
newModel.add(LSTM(32,return_sequences=True,stateful=True))
newModel.add(LSTM(n_feats,return_sequences=False,stateful=True))
newModel.set_weights(model.get_weights())
newModel.reset_states()
lastSteps = np.empty((1, n_rows, n_feats))
lastSteps[:,:time_shift] = x_predict[:,-time_shift:]
newModel.predict(x_predict).reshape(1,1,n_feats)
rangeLen = n_rows - time_shift
for i in range(rangeLen):
lastSteps[:,i+time_shift] = newModel.predict(lastSteps[:,i:i+1,:]).reshape(1,1,n_feats)
forecastFromSelf = lastSteps[:,time_shift:,:]
print (forecastFromSelf.shape)
forecastFromSelf = scaler.inverse_transform(forecastFromSelf.reshape(forecastFromSelf.shape[1],forecastFromSelf.shape[2]))
y_predict_true = scaler.inverse_transform(y_predict_true.reshape(y_predict_true.shape[1],y_predict_true.shape[2]))
plt.plot(y_predict_true[:,0], color='b', label='True')
plt.plot(forecastFromSelf[:,0],color='r', label='Predict')
plt.legend()
plt.title("Self forcast (Feat 1)")
plt.show()
newModel.reset_states()
newModel.predict(x_predict)
newSteps = []
for i in range(x_predict.shape[1]):
newSteps.append(newModel.predict(x_predict[:,i:i+1,:]))
forecastFromInput = np.asarray(newSteps).reshape(1,x_predict.shape[1],n_feats)
print (forecastFromInput.shape)
forecastFromInput = scaler.inverse_transform(forecastFromInput.reshape(forecastFromInput.shape[1],forecastFromInput.shape[2]))
plt.plot(y_predict_true[:,0], color='b', label='True')
plt.plot(forecastFromInput[:,0], color='r', label='Predict')
plt.legend()
plt.title("Forecast from input (Feat 1)")
plt.show()
可以通过增加模型层数和 epoch 数来增加预测。 但是，这里的问题是，为什么 "Self forcast" 比 "Forecast from input" 差？
污染数据在这里：https://github.com/sirjanrocky/some-neural-tests-for-study/blob/master/pollution.csv 此代码运行无误。你也可以试试
假设您的数据在第 1000 步结束，并且您没有更多数据。
但是您甚至希望在第 1100 步之前进行预测。如果您没有输入数据，您将不得不依赖预测的输出。
在“自我预测”中，您将无中生有地预测 100 步，没有任何基础
但是每个预测都有一个相关的错误（它并不完美）。当您根据预测进行预测时，您提供了一个有错误的输入并获得了一个有更多错误的输出。这是不可避免的。
根据预测进行预测，根据预测进行预测，根据预测进行预测....这会累积很多错误。
如果你的模型可以很好地进行这种预测，那么你可以肯定地说它已经学到了很多关于序列的知识。
当您根据已知输入进行预测时，您正在做一件更安全的事情。你有真实的数据要输入，真实的数据是没有错误的。
所以你的预测虽然肯定有错误，但并不是“累积”错误。您不是根据错误数据进行预测，而是根据真实数据进行预测。
图片中：
自我预测（误差累积）
true input --> model --> predicted output step 1001 (a little error)
(steps 1 to 1000) |
V
model
|
V
predicted output step 1002 (more error)
|
V
model
|
V
predicted output step 1003 (even more error)
来自输入的预测（误差不累积）
true input --> model --> predicted output step 1001 (a little error)
(steps 1 to 1000)
true input --> model --> predicted output step 1002 (a little error)
(step 1001)
true input --> model --> predicted output step 1003 (a little error)
(step 1002)
预测部分发现错误（自我预测）
如果要预测测试数据比较：
（评论中的数字只是为了我自己的方向，他们假设训练数据长度为1000，测试数据长度为200，总共1200个元素）
lastSteps = np.empty((1,n_rows-train_size,n_feats)) #test size = 200
lastSteps[:,:time_shift] = y_train[:,-time_shift:] #0 to 6 = 993 to 999
newModel.predict(x_train) #predict 999
rangeLen = n_rows - train_size - time_shift
for i in range(rangeLen):
lastSteps[:,i+time_shift] = newModel.predict(lastSteps[:,i:i+1,:]).reshape(1,1,n_feats)
#el 7 (1000) <- pred from el 0 (993)
forecastFromSelf = lastSteps[:,time_shift:,:] #1000 forward
如果要预测结束后的未知数据：
You should train the entire data (train with
x_predict, y_predict_true)
lastSteps = np.empty((1,new_predictions + time_shift,n_feats))
lastSteps[:,:time_shift] = y_predict_true[:,-time_shift:] #0 to 6 = 1193 to 1199
newModel.predict(x_predict) #predict 1199
rangeLen = new_predictions
for i in range(rangeLen):
lastSteps[:,i+time_shift] = newModel.predict(lastSteps[:,i:i+1,:]).reshape(1,1,n_feats)
#el 7 (1200) <- pred from el 0 (1193)
forecastFromSelf = lastSteps[:,time_shift:,:] #1200 forward