在 Python 中将时间序列数据拆分为训练测试集和有效集

Question

我正在做一个项目，在这个项目中我组合了 2 个时间序列数据集（例如 D1、D2）。 D1 是 5-minutes 间隔，D2 是 1-minute 间隔，所以我将 D1 转换为 1 分钟间隔并与 D2。现在我想根据这些条件将这个新数据集 D1D2 分成训练集、测试集和有效集：

Note: I have searched a lot and try to find a solution for my problem but couldn't any answer fit to my question, so don't mark this as duplicate, please!

有效集应该是数据集末尾的 60 个值。
然后，测试集应该是最近的值，直到 valid set
然后，我将用剩余的数据设置火车。

下面是我现在进行拆分的方式：

def split_train_test(dataset, train_size, test_size):
    train = dataset[:train_size, :]
    test = dataset[test_size:, :]
    # split into input and outputs
    train_X, train_y = train[:, :-1], train[:, -1]
    test_X, test_y = test[:, :-1], test[:, -1]
    # reshape input to be 3D [samples, timesteps, features]
    train_X = train_X.reshape((train_X.shape[0], 1, train_X.shape[1]))
    test_X = test_X.reshape((test_X.shape[0], 1, test_X.shape[1]))
    print(train_X.shape, train_y.shape, test_X.shape)
    return train, test, train_X, train_y, test_X, test_y

但是现在需要根据以上条件转成train、test、split？

我该怎么做？以及拆分时间序列数据集的正确方法吗？

Answer 1

试试这个：

valid_set = dataset.iloc[-60:, :]
test_set = dataset.iloc[-120:-60]
train_set = dataset.iloc[:-120]

归纳：

def split_train_test(dataset, validation_size):
    valid = dataset.iloc[-validation_size:, :]
    train_test = dataset.iloc[:-validation_size)]

    train_length = int(0.63 * len(train_test))

    # split into input and outputs
    train_X, train_y = train_test.iloc[:train_length, :-1], train_test.iloc[:train_length, -1]
    test_X, test_y = train_test.iloc[train_length:, :-1], train_test.iloc[train_length:, -1]
    valid_X, valid_y = valid.iloc[:, :-1], valid.iloc[:, -1]

    return train_test, valid, train_X, train_y, test_X, test_y, valid_X, valid_y

您可以将 % split rati 作为参数传递到函数中，而不是像我那样将其硬编码到函数中。

在 Python 中将时间序列数据拆分为训练测试集和有效集

Split time series data into Train Test and Valid sets in Python

python

pandas

machine-learning

scikit-learn

train-test-split