如何在从模型中选择特征时执行随机网格搜索？

Question

我正在寻找 select 基于随机森林、梯度提升和极端梯度提升的特征重要性的特征。我正在尝试使用随机网格搜索来拟合我的模型以获得最佳模型的特征重要性，但它给了我一个我不明白的错误，这是我的代码：

gbr = GradientBoostingRegressor(random_state=seed)
gbr_params = {
    "learning_rate": [0.001, 0.01, 0.1],
    "min_samples_split": [50, 100],
    "min_samples_leaf": [50, 100],
    "max_depth":[5, 10, 20]}

xgbr = xgboost.XGBRegressor(random_state=seed) 
xgbr_params = {  
    "learning_rate": [0.001, 0.01, 0.1],
    "min_samples_leaf": [50, 100],
    "max_depth":[5, 10, 20],
    'reg_alpha': [1.1, 1.2, 1.3],
    'reg_lambda': [1.1, 1.2, 1.3]}

rfr = RandomForestRegressor(random_state=seed)
rfr_params={'n_estimators':[100, 500, 1000], 
             'max_features':[10,14,18],
             'min_samples_split': [50, 100],
             'min_samples_leaf': [50, 100],} 

fs_xgbr = dcv.RandomizedSearchCV(xgbr, xgbr_params, cv=5, iid=False, n_jobs=-1)
fs_gbr = dcv.RandomizedSearchCV(gbr, gbr_params, cv=5,iid=False, n_jobs=-1)
fs_rfr = dcv.RandomizedSearchCV(rfr, rfr_params, cv=5,iid=False, n_jobs=-1)

fs_rfr.fit(X, Y)
model = SelectFromModel(fs_rfr, prefit=True)
X_rfr = model.transform(X)
print('rfr', X_rfr.shape)

在 X_rfr = model.transform(X) 行给出了这个错误：

ValueError: The underlying estimator RandomizedSearchCV has no `coef_` or `feature_importances_` attribute. Either pass a fitted estimator to SelectFromModel or call fit before calling transform.

我不是程序员，也没有在其他地方找到任何解决方案来解决这个问题，是否无法采用 feature_importances_ 模型的最佳参数由随机搜索决定？

Answer 1

不是传递给 SelectFromModel fs_rfr，它是 RandomizedSearchCV 类型的对象，而是传递最佳估计器，例如 fs_rfr.best_estimator_

证明

import xgboost
from sklearn.ensemble import GradientBoostingRegressor, RandomForestRegressor
from sklearn.datasets import make_regression
from sklearn.model_selection import RandomizedSearchCV
from sklearn.feature_selection import SelectFromModel

seed=42

gbr = GradientBoostingRegressor(random_state=seed)
gbr_params = {
    "learning_rate": [0.001, 0.01, 0.1],
    "min_samples_split": [50, 100],
    "min_samples_leaf": [50, 100],
    "max_depth":[5, 10, 20]}

xgbr = xgboost.XGBRegressor(random_state=seed) 
xgbr_params = {  
    "learning_rate": [0.001, 0.01, 0.1],
    "min_samples_leaf": [50, 100],
    "max_depth":[5, 10, 20],
    'reg_alpha': [1.1, 1.2, 1.3],
    'reg_lambda': [1.1, 1.2, 1.3]}

rfr = RandomForestRegressor(random_state=seed)
rfr_params={'n_estimators':[100, 500, 1000], 
             'max_features':[10,14,18],
             'min_samples_split': [50, 100],
             'min_samples_leaf': [50, 100],} 

fs_xgbr = RandomizedSearchCV(xgbr, xgbr_params, cv=5, iid=False, n_jobs=-1)
fs_gbr = RandomizedSearchCV(gbr, gbr_params, cv=5,iid=False, n_jobs=-1)
fs_rfr = RandomizedSearchCV(rfr, rfr_params, cv=5,iid=False, n_jobs=-1)

X, y = make_regression(1000,10)

fs_xgbr.fit(X, y)
fs_gbr.fit(X, y)
fs_rfr.fit(X, y)

model = SelectFromModel(fs_rfr.best_estimator_, prefit=True)
X_rfr = model.transform(X)
print('rfr', X_rfr.shape)

model = SelectFromModel(fs_xgbr.best_estimator_, prefit=True)
X_xgbr = model.transform(X)
print('xgbr', X_xgbr.shape)

model = SelectFromModel(fs_gbr.best_estimator_, prefit=True)
X_gbr = model.transform(X)
print('gbr', X_gbr.shape)

rfr (1000, 3)
xgbr (1000, 3)
gbr (1000, 4)

如何在从模型中选择特征时执行随机网格搜索？

How to perform a random gridsearch within selecting features from a model?

python

machine-learning

scikit-learn

grid-search

gridsearchcv