在 GridSearchCV 之后进行预测时是否遵循管道步骤
Are the pipeline steps followed when Predicting after GridSearch CV
我将 GridSearchCV 与包含标准化作为第一步的管道结合使用。我发现当使用 GridSearchCV 的 .predict 方法预测测试数据集时,结果与手动实施管道步骤时的结果不同。我在下面创建了我的脚本的简化版本,以表明发现的错误不同。为简单起见,搜索 space 每个参数仅包含 1 个值。
我知道这里的差别很小。但是在我的原始代码中,这个差异明显更大。因此,我试图了解是什么导致了这两种方法之间的差异
正在初始化数据
import random
import numpy as np
from sklearn.metrics import mean_absolute_error
from sklearn.model_selection import GridSearchCV, KFold
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVR
random.seed(5)
np.random.seed(5)
x_train = np.random.rand(1000,4)
y_train = np.random.rand(1000,1)
x_test = np.random.rand(100,4)
y_test = np.random.rand(100,1)
C_space = 1
epsilon_space = 0.04
gamma_space = 0.001
实施管道
cv_folds = KFold(n_splits=5, shuffle=True)
steps = [
('scaler', StandardScaler()),
('svr', SVR(kernel='rbf', gamma=gamma_space, C=C_space, epsilon=epsilon_space))
]
pipe = Pipeline(steps, verbose=0)
search_space = [{
'svr__gamma':[gamma_space],
'svr__C':[C_space],
'svr__epsilon':[epsilon_space]
}]
mod = GridSearchCV(pipe, search_space,
scoring='neg_mean_absolute_error', cv=5, verbose=0, return_train_score=True, refit=True)
svr = mod.fit(x_train, y_train)
y_pred = svr.predict(x_test)
error = mean_absolute_error(y_pred, y_test)
手动执行步骤
scaler = StandardScaler()
x_train_scaled = scaler.fit_transform(x_train)
x_test_scaled = scaler.fit_transform(x_test)
manual_svr = SVR(kernel='rbf', gamma=gamma_space, C=C_space,
epsilon=epsilon_space).fit(x_train_scaled, y_train)
y_pred_manual = manual_svr.predict(x_test_scaled)
error_manual = mean_absolute_error(y_pred_manual, y_test)
结果是:
Pipeline error is: 0.23495746730222067
Manual error is: 0.23487379770774958
您正在将 GridSearchCV
内的 StandardScaler
拟合到 训练数据 ,而您 再 -将您的“手册”scaler
与 测试数据 相匹配。有
scaler = StandardScaler()
x_train_scaled = scaler.fit_transform(x_train)
x_test_scaled = scaler.fit_transform(x_test)
您正在覆盖适合训练数据的scaler
!
这是不是使用定标器的方式。将定标器与 训练数据相匹配 ,然后使用该定标器对测试数据进行标准化。
让我们将您的输出与其应有的样子进行比较。首先让我们提取适合 GridSearchCV
的缩放器并用它标准化测试数据:
gscv_sclr = mod.best_estimator_.named_steps['scaler']
gscv_test_scld = gscv_sclr.transform(x_test)
如您所见,这是不等于您手动标准化的测试数据:
np.allclose(gscv_test_scld, x_test_scaled)
# Out: False
现在让我们只用训练数据拟合“手动”标准器,并使用此标准器转换您的测试数据:
scaler_new = StandardScaler()
x_train_scaled = scaler_new.fit_transform(x_train)
x_test_scaled_new = scaler_new.transform(x_test)
# and compare it to the gridsearchcv scaler:
np.allclose(gscv_test_scld, x_test_scaled_new)
# Out: True
等于!
现在使用这个正确标准化的测试集进行预测:
# this refitting is actually not needed. it is simply here for having separate models...
manual_svr_new = SVR(kernel='rbf', gamma=gamma_space, C=C_space,
epsilon=epsilon_space).fit(x_train_scaled, y_train)
y_pred_manual_new = manual_svr_new.predict(x_test_scaled_new)
error_manual_new = mean_absolute_error(y_pred_manual_new, y_test)
# And test it:
error_manual_new == error
# Out: True
现在您已获得管道的结果。
我将 GridSearchCV 与包含标准化作为第一步的管道结合使用。我发现当使用 GridSearchCV 的 .predict 方法预测测试数据集时,结果与手动实施管道步骤时的结果不同。我在下面创建了我的脚本的简化版本,以表明发现的错误不同。为简单起见,搜索 space 每个参数仅包含 1 个值。
我知道这里的差别很小。但是在我的原始代码中,这个差异明显更大。因此,我试图了解是什么导致了这两种方法之间的差异
正在初始化数据
import random
import numpy as np
from sklearn.metrics import mean_absolute_error
from sklearn.model_selection import GridSearchCV, KFold
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVR
random.seed(5)
np.random.seed(5)
x_train = np.random.rand(1000,4)
y_train = np.random.rand(1000,1)
x_test = np.random.rand(100,4)
y_test = np.random.rand(100,1)
C_space = 1
epsilon_space = 0.04
gamma_space = 0.001
实施管道
cv_folds = KFold(n_splits=5, shuffle=True)
steps = [
('scaler', StandardScaler()),
('svr', SVR(kernel='rbf', gamma=gamma_space, C=C_space, epsilon=epsilon_space))
]
pipe = Pipeline(steps, verbose=0)
search_space = [{
'svr__gamma':[gamma_space],
'svr__C':[C_space],
'svr__epsilon':[epsilon_space]
}]
mod = GridSearchCV(pipe, search_space,
scoring='neg_mean_absolute_error', cv=5, verbose=0, return_train_score=True, refit=True)
svr = mod.fit(x_train, y_train)
y_pred = svr.predict(x_test)
error = mean_absolute_error(y_pred, y_test)
手动执行步骤
scaler = StandardScaler()
x_train_scaled = scaler.fit_transform(x_train)
x_test_scaled = scaler.fit_transform(x_test)
manual_svr = SVR(kernel='rbf', gamma=gamma_space, C=C_space,
epsilon=epsilon_space).fit(x_train_scaled, y_train)
y_pred_manual = manual_svr.predict(x_test_scaled)
error_manual = mean_absolute_error(y_pred_manual, y_test)
结果是:
Pipeline error is: 0.23495746730222067
Manual error is: 0.23487379770774958
您正在将 GridSearchCV
内的 StandardScaler
拟合到 训练数据 ,而您 再 -将您的“手册”scaler
与 测试数据 相匹配。有
scaler = StandardScaler()
x_train_scaled = scaler.fit_transform(x_train)
x_test_scaled = scaler.fit_transform(x_test)
您正在覆盖适合训练数据的scaler
!
这是不是使用定标器的方式。将定标器与 训练数据相匹配 ,然后使用该定标器对测试数据进行标准化。
让我们将您的输出与其应有的样子进行比较。首先让我们提取适合 GridSearchCV
的缩放器并用它标准化测试数据:
gscv_sclr = mod.best_estimator_.named_steps['scaler']
gscv_test_scld = gscv_sclr.transform(x_test)
如您所见,这是不等于您手动标准化的测试数据:
np.allclose(gscv_test_scld, x_test_scaled)
# Out: False
现在让我们只用训练数据拟合“手动”标准器,并使用此标准器转换您的测试数据:
scaler_new = StandardScaler()
x_train_scaled = scaler_new.fit_transform(x_train)
x_test_scaled_new = scaler_new.transform(x_test)
# and compare it to the gridsearchcv scaler:
np.allclose(gscv_test_scld, x_test_scaled_new)
# Out: True
等于!
现在使用这个正确标准化的测试集进行预测:
# this refitting is actually not needed. it is simply here for having separate models...
manual_svr_new = SVR(kernel='rbf', gamma=gamma_space, C=C_space,
epsilon=epsilon_space).fit(x_train_scaled, y_train)
y_pred_manual_new = manual_svr_new.predict(x_test_scaled_new)
error_manual_new = mean_absolute_error(y_pred_manual_new, y_test)
# And test it:
error_manual_new == error
# Out: True
现在您已获得管道的结果。