神经网络 - 超参数调整

Question

我正在为我的 MLPClassifier 模型进行超参数调整。我正在尝试使用网格搜索，但出了点问题，我无法得到什么。

这是型号：

data = pd.read_csv("Xy_train.csv")
data = data.loc[(data.age <= 90) & (data.thal != 0)]
x = data.drop(columns=['y', 'id'])
y = data.y


x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.3, random_state=986)
scaler = StandardScaler()
x_train_s = scaler.fit_transform(x_train)
x_test_s = scaler.fit_transform(x_test)

model = MLPClassifier(random_state=986,
                      hidden_layer_sizes=(15),
                      max_iter=200,
                      activation='relu',
                      verbose=True,
                      learning_rate_init=0.01)
model.fit(x_train_s, y_train)

x_train = np.linspace(-5, 5, 50)
y_train = np.linspace(-5, 5, 50)
predictions = pd.DataFrame()

print(f"Accuracy: {accuracy_score(y_true=y_test, y_pred=model.predict(x_test_s)):.3f}")
print(confusion_matrix(y_true=y_test, y_pred=model.predict(x_test_s)))

这是我的网格搜索：

epochs = [5, 10]
batches = [5, 10, 100]
optimizers = ['lbfgs', 'sgd', 'adam']
hyperparameters = dict(optimizer=optimizers, epochs=epochs, batch_size=batches)
grid = GridSearchCV(estimator=model, cv=3, param_grid=hyperparameters)
grid.fit(x_train, y_train)

我得到了一堆错误，但并没有说明太多，我认为可能最相关的 is:Check 可用参数列表 estimator.get_params().keys()。错误是：

line 49, in <module>
    grid.fit(x_train, y_train)
line 754, in _dispatch
    job = self._backend.apply_async(batch, callback=cb)
line 209, in apply_async
    result = ImmediateResult(func)
 line 233, in set_params
    raise ValueError('Invalid parameter %s for estimator %s. '
ValueError: Invalid parameter epochs for estimator MLPClassifier(activation='relu', alpha=0.0001, batch_size=5, beta_1=0.9,
              beta_2=0.999, early_stopping=False, epsilon=1e-08,
              hidden_layer_sizes=15, learning_rate='constant',
              learning_rate_init=0.01, max_fun=15000, max_iter=200,
              momentum=0.9, n_iter_no_change=10, nesterovs_momentum=True,
              power_t=0.5, random_state=986, shuffle=True, solver='adam',
              tol=0.0001, validation_fraction=0.1, verbose=True,
              warm_start=False). Check the list of available parameters with `estimator.get_params().keys()'

但我确实检查了参数键，似乎没问题。

Answer 1

在您的 hyperparameters 网格中：

hyperparameters = dict(optimizer=optimizers, epochs=epochs, batch_size=batches)

您要求的超参数不存在，例如 optimizer 和 epochs，因此出现错误。

根据documentation，你应该分别要求solver和max_iter，即：

hyperparameters = dict(solver=optimizers, max_iter=epochs, batch_size=batches)

神经网络 - 超参数调整

Neural Networks - hyper parameters tuning

machine-learning

scikit-learn

grid-search

mlp