时间序列预测的正则化 LSTM
Regularization LSTM for a Time Series forecast
我有一个包含 220 行和 1410 列的数据框。行代表以天为单位的日期时间，列代表三个时间间隔内的 470 个不同特征 (470 * 3 = 1410)，如下所示：
CEE_SECO_01_lag1 CEE_SECO_02_lag1 ... BM_lag3 PME_lag3
2017-07-31 30553.75 28373.42 ... 266325.00 217874.92
2017-08-01 30656.70 28715.36 ... 266325.00 217874.92
2017-08-02 30600.47 28341.33 ... 266325.00 217874.92
2017-08-03 28468.36 26797.55 ... 266325.00 217874.92
2017-08-04 29081.35 27373.21 ... 266325.00 217874.92
... ... ... ... ...
2018-03-03 33249.36 31572.65 ... 262770.31 218720.93
2018-03-04 36189.80 34308.52 ... 262770.31 218720.93
2018-03-05 36082.87 33824.52 ... 262770.31 218720.93
2018-03-06 35227.69 32910.53 ... 262770.31 218720.93
2018-03-07 35891.20 33809.99 ... 262770.31 218720.93
[220 rows x 1410 columns]
我正在尝试 运行 LSTM 模型来解决预测问题。
首先我重塑了我的数据：
import pandas as pd
import numpy as np
n_samples = len(X_train)
n_steps = 3
n_features = int(len(X_train.columns)/n_steps)
X_train_lstm = X_train.to_numpy().reshape((n_samples,n_steps,n_features))
X_teste_lstm = X_test.to_numpy().reshape((1,n_steps,n_features))
y_treino_lstm = y_train.to_numpy()
y_teste_lstm = y_test.to_numpy()
然后，我创建了一个 Vanilla LSTM 模型（像这样 https://machinelearningmastery.com/how-to-develop-lstm-models-for-time-series-forecasting/）：
from keras.models import Sequential
from keras.layers import LSTM
from keras.layers import Dense
model = Sequential()
model.add(LSTM(50, activation='relu', input_shape=(n_steps, n_features)))
model.add(Dense(1))
model.compile(optimizer='adam', loss='mse')
model.fit(X_train_lstm, y_train_lstm, epochs=1000)
而且效果很好。但是当我试图在这个模型中进行正则化时（像这样：https://machinelearningmastery.com/use-weight-regularization-lstm-networks-time-series-forecasting/），我被卡住了。
如果我试试这个：
from keras.regularizers import L1L2
model = Sequential()
model.add(LSTM(50, activation='relu', kernel_regularizer = L1L2(l1=0.01, l2=0.0) , input_shape=(n_steps, n_features)))
model.add(Dense(1))
model.compile(optimizer='adam', loss='mse')
model.fit(X_train_lstm, y_train_lstm, epochs=1000)
发生此错误：
ValueError: If a RNN is stateful, it needs to know its batch size. Specify the batch size of your input tensors:
- If using a Sequential model, specify the batch size by passing a `batch_input_shape` argument to your first layer.
- If using the functional API, specify the batch size by passing a `batch_shape` argument to your Input layer.
如果我尝试将 input_shape 更改为 batch_input_shape，如下所示：
n_batchs = 1
model = Sequential()
model.add(LSTM(50, activation='relu', kernel_regularizer = L1L2(l1=0.01, l2=0.0) , batch_input_shape=(n_batchs, n_steps, n_features)))
model.add(Dense(1))
model.compile(optimizer='adam', loss='mse')
model.fit(X_train_lstm, y_train_lstm, epochs=1000)
我收到这个错误：
InvalidArgumentError: Specified a list with shape [1,470] from a tensor with shape [32,470]
[[node sequential_3795/lstm_3791/TensorArrayUnstack/TensorListFromTensor (defined at C:\Users\Pichau\anaconda3\lib\site-packages\keras\backend.py:4330) ]] [Op:__inference_train_function_48158087]
Errors may have originated from an input operation.
Input Source operations connected to node sequential_3795/lstm_3791/TensorArrayUnstack/TensorListFromTensor:
sequential_3795/lstm_3791/transpose (defined at C:\Users\Pichau\anaconda3\lib\site-packages\keras\backend.py:4199)
Function call stack:
train_function
我已经尝试了 n_batchs 的许多不同值，例如 32、28、特征数、观察值、列或时间步长以及一些随机值。
如何使用 LSTM 模型进行正则化？
我已经为你整理了一个例子
import pandas as pd
import numpy as np
import tensorflow as tf
from sklearn.datasets import make_regression
X, y = make_regression()
print(X.shape, y.shape)
lstm_train = []
for i in range(0,100,10):
for j in range(0,100,10):
lstm_train.append(X[j:j+10])
lstm_train = np.stack(lstm_train, axis=0)
print(lstm_train.shape)
from tensorflow.keras.regularizers import L1L2
model = tf.keras.Sequential()
model.add(tf.keras.layers.InputLayer(input_shape=(10, 100)))
model.add(tf.keras.layers.LSTM(50, activation='relu', kernel_regularizer = L1L2(l1=0.01, l2=0.0)))
model.add(tf.keras.layers.Dense(1))
model.compile(optimizer='adam', loss='mse')
model.fit(lstm_train, y, epochs=10)
症结就在这一行。我们可以在模型中添加一个独占的 InputLayer 并在那里指定维度，而不是在 LSTM 层中进行。
model.add(tf.keras.layers.InputLayer(input_shape=(10, 100)))
我有一个包含 220 行和 1410 列的数据框。行代表以天为单位的日期时间，列代表三个时间间隔内的 470 个不同特征 (470 * 3 = 1410)，如下所示：
CEE_SECO_01_lag1 CEE_SECO_02_lag1 ... BM_lag3 PME_lag3
2017-07-31 30553.75 28373.42 ... 266325.00 217874.92
2017-08-01 30656.70 28715.36 ... 266325.00 217874.92
2017-08-02 30600.47 28341.33 ... 266325.00 217874.92
2017-08-03 28468.36 26797.55 ... 266325.00 217874.92
2017-08-04 29081.35 27373.21 ... 266325.00 217874.92
... ... ... ... ...
2018-03-03 33249.36 31572.65 ... 262770.31 218720.93
2018-03-04 36189.80 34308.52 ... 262770.31 218720.93
2018-03-05 36082.87 33824.52 ... 262770.31 218720.93
2018-03-06 35227.69 32910.53 ... 262770.31 218720.93
2018-03-07 35891.20 33809.99 ... 262770.31 218720.93
[220 rows x 1410 columns]
我正在尝试 运行 LSTM 模型来解决预测问题。
首先我重塑了我的数据：
import pandas as pd
import numpy as np
n_samples = len(X_train)
n_steps = 3
n_features = int(len(X_train.columns)/n_steps)
X_train_lstm = X_train.to_numpy().reshape((n_samples,n_steps,n_features))
X_teste_lstm = X_test.to_numpy().reshape((1,n_steps,n_features))
y_treino_lstm = y_train.to_numpy()
y_teste_lstm = y_test.to_numpy()
然后，我创建了一个 Vanilla LSTM 模型（像这样 https://machinelearningmastery.com/how-to-develop-lstm-models-for-time-series-forecasting/）：
from keras.models import Sequential
from keras.layers import LSTM
from keras.layers import Dense
model = Sequential()
model.add(LSTM(50, activation='relu', input_shape=(n_steps, n_features)))
model.add(Dense(1))
model.compile(optimizer='adam', loss='mse')
model.fit(X_train_lstm, y_train_lstm, epochs=1000)
而且效果很好。但是当我试图在这个模型中进行正则化时（像这样：https://machinelearningmastery.com/use-weight-regularization-lstm-networks-time-series-forecasting/），我被卡住了。
如果我试试这个：
from keras.regularizers import L1L2
model = Sequential()
model.add(LSTM(50, activation='relu', kernel_regularizer = L1L2(l1=0.01, l2=0.0) , input_shape=(n_steps, n_features)))
model.add(Dense(1))
model.compile(optimizer='adam', loss='mse')
model.fit(X_train_lstm, y_train_lstm, epochs=1000)
发生此错误：
ValueError: If a RNN is stateful, it needs to know its batch size. Specify the batch size of your input tensors:
- If using a Sequential model, specify the batch size by passing a `batch_input_shape` argument to your first layer.
- If using the functional API, specify the batch size by passing a `batch_shape` argument to your Input layer.
如果我尝试将 input_shape 更改为 batch_input_shape，如下所示：
n_batchs = 1
model = Sequential()
model.add(LSTM(50, activation='relu', kernel_regularizer = L1L2(l1=0.01, l2=0.0) , batch_input_shape=(n_batchs, n_steps, n_features)))
model.add(Dense(1))
model.compile(optimizer='adam', loss='mse')
model.fit(X_train_lstm, y_train_lstm, epochs=1000)
我收到这个错误：
InvalidArgumentError: Specified a list with shape [1,470] from a tensor with shape [32,470]
[[node sequential_3795/lstm_3791/TensorArrayUnstack/TensorListFromTensor (defined at C:\Users\Pichau\anaconda3\lib\site-packages\keras\backend.py:4330) ]] [Op:__inference_train_function_48158087]
Errors may have originated from an input operation.
Input Source operations connected to node sequential_3795/lstm_3791/TensorArrayUnstack/TensorListFromTensor:
sequential_3795/lstm_3791/transpose (defined at C:\Users\Pichau\anaconda3\lib\site-packages\keras\backend.py:4199)
Function call stack:
train_function
我已经尝试了 n_batchs 的许多不同值，例如 32、28、特征数、观察值、列或时间步长以及一些随机值。
如何使用 LSTM 模型进行正则化？
我已经为你整理了一个例子
import pandas as pd
import numpy as np
import tensorflow as tf
from sklearn.datasets import make_regression
X, y = make_regression()
print(X.shape, y.shape)
lstm_train = []
for i in range(0,100,10):
for j in range(0,100,10):
lstm_train.append(X[j:j+10])
lstm_train = np.stack(lstm_train, axis=0)
print(lstm_train.shape)
from tensorflow.keras.regularizers import L1L2
model = tf.keras.Sequential()
model.add(tf.keras.layers.InputLayer(input_shape=(10, 100)))
model.add(tf.keras.layers.LSTM(50, activation='relu', kernel_regularizer = L1L2(l1=0.01, l2=0.0)))
model.add(tf.keras.layers.Dense(1))
model.compile(optimizer='adam', loss='mse')
model.fit(lstm_train, y, epochs=10)
症结就在这一行。我们可以在模型中添加一个独占的 InputLayer 并在那里指定维度，而不是在 LSTM 层中进行。
model.add(tf.keras.layers.InputLayer(input_shape=(10, 100)))