如何使用 optuna trial 在 sklearn MLPRegressor 中设置 hidden_layer_sizes

How to set hidden_layer_sizes in sklearn MLPRegressor using optuna trial

我想将 [OPTUNA][1]sklearn [MLPRegressor][1] 模型一起使用。

对于几乎所有超参数,如何为它们设置 OPTUNA 都非常简单。 例如,要设置学习率: learning_rate_init = trial.suggest_float('learning_rate_init ',0.0001, 0.1001, step=0.005)

我的问题是如何为 hidden_layer_sizes 设置它,因为它是一个元组。假设我想要两个隐藏层,第一个有 100 个神经元,第二个有 50 个神经元。如果没有 OPTUNA,我会这样做:

MLPRegressor( hidden_layer_sizes =(100,50))

但是如果我想让 OPTUNA 在每一层尝试不同的神经元呢?例如,从 100 到 500,如何设置? MLPRegressor 需要一个元组

您可以按如下方式设置 objective 函数:

import optuna
import warnings
from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.neural_network import MLPRegressor
from sklearn.metrics import mean_squared_error
warnings.filterwarnings('ignore')

X, y = make_regression(random_state=1)

X_train, X_valid, y_train, y_valid = train_test_split(X, y, random_state=1)

def objective(trial):

    params = {
        'learning_rate_init': trial.suggest_float('learning_rate_init ', 0.0001, 0.1, step=0.005),
        'first_layer_neurons': trial.suggest_int('first_layer_neurons', 10, 100, step=10),
        'second_layer_neurons': trial.suggest_int('second_layer_neurons', 10, 100, step=10),
        'activation': trial.suggest_categorical('activation', ['identity', 'tanh', 'relu']),
    }

    model = MLPRegressor(
        hidden_layer_sizes=(params['first_layer_neurons'], params['second_layer_neurons']),
        learning_rate_init=params['learning_rate_init'],
        activation=params['activation'],
        random_state=1,
        max_iter=100
    )

    model.fit(X_train, y_train)

    return mean_squared_error(y_valid, model.predict(X_valid), squared=False)

study = optuna.create_study(direction='minimize')
study.optimize(objective, n_trials=3)
# [I 2021-11-11 18:04:02,216] A new study created in memory with name: no-name-14c92e38-b8cd-4b8d-8a95-77158d996f20
# [I 2021-11-11 18:04:02,283] Trial 0 finished with value: 161.8347337123744 and parameters: {'learning_rate_init ': 0.0651, 'first_layer_neurons': 20, 'second_layer_neurons': 40, 'activation': 'tanh'}. Best is trial 0 with value: 161.8347337123744.
# [I 2021-11-11 18:04:02,368] Trial 1 finished with value: 159.55535852658082 and parameters: {'learning_rate_init ': 0.0551, 'first_layer_neurons': 90, 'second_layer_neurons': 70, 'activation': 'relu'}. Best is trial 1 with value: 159.55535852658082.
# [I 2021-11-11 18:04:02,440] Trial 2 finished with value: 161.73980822730888 and parameters: {'learning_rate_init ': 0.0051, 'first_layer_neurons': 100, 'second_layer_neurons': 30, 'activation': 'identity'}. Best is trial 1 with value: 159.55535852658082.