使用 GridSearchCV 优化 MLP 学习率

Question

我正在尝试使用 GridSearchCV 调整 MLP 分类器的超参数，但遇到以下问题：

/usr/local/lib/python3.7/dist-packages/sklearn/model_selection/_validation.py:536: FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan.


Details: 

ValueError: learning rate 0.01 is not supported. 

  FitFailedWarning)

/usr/local/lib/python3.7/dist-packages/sklearn/model_selection/_validation.py:536: FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan.

Details: 

ValueError: learning rate 0.02 is not supported

........

代码：

clf = MLPClassifier()
params= {
   'hidden_layer_sizes': hidden_layers_generator(X,np.arange(1,17,1)),
   'solver': ['sgd'],
   'momentum': np.arange(0.1,1.1,0.1),
   'learning_rate': np.arange(0.01,1.01,0.01),
   'max_iter': np.arange(100,2100,100)}
grid = GridSearchCV(clf, params, cv=10, scoring='accuracy')
grid.fit(X, y)
grid_mean_scores = grid.cv_results_['mean_test_score']
pd.DataFrame(grid.cv_results_)[['mean_test_score', 'std_test_score', 'params']]

hidden_layers_generator的代码如下：

from itertools import combinations_with_replacement
def hidden_layers_generator(df,hidden_layers):
  hd_sizes = []
  for l in range(1, len(hidden_layers)):
    comb = combinations_with_replacement(np.arange(1,len(df.columns),10), l)
    hd_sizes.append(list(comb))
  return hd_sizes

这是 X 和 y 数据帧的一小段：

X.head()
    sl      sw      pl      pw
0   5.1     3.5     1.4     0.2
1   4.9     3.0     1.4     0.2
2   4.7     3.2     1.3     0.2
3   4.6     3.1     1.5     0.2
4   5.0     3.6     1.4     0.2

Answer 1

如果你看一下MLPClassifier的documentation，你会发现learning_rate参数不是你想的那样，而是一种调度器。你想要的是 learning_rate_init 参数。所以在配置中更改这一行：

   'learning_rate': np.arange(0.01,1.01,0.01),

到

   'learning_rate_init': np.arange(0.01,1.01,0.01),

使用 GridSearchCV 优化 MLP 学习率

MLP learning rate optimization with GridSearchCV

python

scikit-learn

grid-search

mlp