如何使用 optuna trial 在 sklearn MLPRegressor 中设置 hidden_layer_sizes
How to set hidden_layer_sizes in sklearn MLPRegressor using optuna trial
我想将 [OPTUNA][1]
与 sklearn
[MLPRegressor][1]
模型一起使用。
对于几乎所有超参数,如何为它们设置 OPTUNA 都非常简单。
例如,要设置学习率:
learning_rate_init = trial.suggest_float('learning_rate_init ',0.0001, 0.1001, step=0.005)
我的问题是如何为 hidden_layer_sizes
设置它,因为它是一个元组。假设我想要两个隐藏层,第一个有 100 个神经元,第二个有 50 个神经元。如果没有 OPTUNA,我会这样做:
MLPRegressor( hidden_layer_sizes =(100,50))
但是如果我想让 OPTUNA 在每一层尝试不同的神经元呢?例如,从 100 到 500,如何设置? MLPRegressor
需要一个元组
您可以按如下方式设置 objective 函数:
import optuna
import warnings
from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.neural_network import MLPRegressor
from sklearn.metrics import mean_squared_error
warnings.filterwarnings('ignore')
X, y = make_regression(random_state=1)
X_train, X_valid, y_train, y_valid = train_test_split(X, y, random_state=1)
def objective(trial):
params = {
'learning_rate_init': trial.suggest_float('learning_rate_init ', 0.0001, 0.1, step=0.005),
'first_layer_neurons': trial.suggest_int('first_layer_neurons', 10, 100, step=10),
'second_layer_neurons': trial.suggest_int('second_layer_neurons', 10, 100, step=10),
'activation': trial.suggest_categorical('activation', ['identity', 'tanh', 'relu']),
}
model = MLPRegressor(
hidden_layer_sizes=(params['first_layer_neurons'], params['second_layer_neurons']),
learning_rate_init=params['learning_rate_init'],
activation=params['activation'],
random_state=1,
max_iter=100
)
model.fit(X_train, y_train)
return mean_squared_error(y_valid, model.predict(X_valid), squared=False)
study = optuna.create_study(direction='minimize')
study.optimize(objective, n_trials=3)
# [I 2021-11-11 18:04:02,216] A new study created in memory with name: no-name-14c92e38-b8cd-4b8d-8a95-77158d996f20
# [I 2021-11-11 18:04:02,283] Trial 0 finished with value: 161.8347337123744 and parameters: {'learning_rate_init ': 0.0651, 'first_layer_neurons': 20, 'second_layer_neurons': 40, 'activation': 'tanh'}. Best is trial 0 with value: 161.8347337123744.
# [I 2021-11-11 18:04:02,368] Trial 1 finished with value: 159.55535852658082 and parameters: {'learning_rate_init ': 0.0551, 'first_layer_neurons': 90, 'second_layer_neurons': 70, 'activation': 'relu'}. Best is trial 1 with value: 159.55535852658082.
# [I 2021-11-11 18:04:02,440] Trial 2 finished with value: 161.73980822730888 and parameters: {'learning_rate_init ': 0.0051, 'first_layer_neurons': 100, 'second_layer_neurons': 30, 'activation': 'identity'}. Best is trial 1 with value: 159.55535852658082.
我想将 [OPTUNA][1]
与 sklearn
[MLPRegressor][1]
模型一起使用。
对于几乎所有超参数,如何为它们设置 OPTUNA 都非常简单。
例如,要设置学习率:
learning_rate_init = trial.suggest_float('learning_rate_init ',0.0001, 0.1001, step=0.005)
我的问题是如何为 hidden_layer_sizes
设置它,因为它是一个元组。假设我想要两个隐藏层,第一个有 100 个神经元,第二个有 50 个神经元。如果没有 OPTUNA,我会这样做:
MLPRegressor( hidden_layer_sizes =(100,50))
但是如果我想让 OPTUNA 在每一层尝试不同的神经元呢?例如,从 100 到 500,如何设置? MLPRegressor
需要一个元组
您可以按如下方式设置 objective 函数:
import optuna
import warnings
from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.neural_network import MLPRegressor
from sklearn.metrics import mean_squared_error
warnings.filterwarnings('ignore')
X, y = make_regression(random_state=1)
X_train, X_valid, y_train, y_valid = train_test_split(X, y, random_state=1)
def objective(trial):
params = {
'learning_rate_init': trial.suggest_float('learning_rate_init ', 0.0001, 0.1, step=0.005),
'first_layer_neurons': trial.suggest_int('first_layer_neurons', 10, 100, step=10),
'second_layer_neurons': trial.suggest_int('second_layer_neurons', 10, 100, step=10),
'activation': trial.suggest_categorical('activation', ['identity', 'tanh', 'relu']),
}
model = MLPRegressor(
hidden_layer_sizes=(params['first_layer_neurons'], params['second_layer_neurons']),
learning_rate_init=params['learning_rate_init'],
activation=params['activation'],
random_state=1,
max_iter=100
)
model.fit(X_train, y_train)
return mean_squared_error(y_valid, model.predict(X_valid), squared=False)
study = optuna.create_study(direction='minimize')
study.optimize(objective, n_trials=3)
# [I 2021-11-11 18:04:02,216] A new study created in memory with name: no-name-14c92e38-b8cd-4b8d-8a95-77158d996f20
# [I 2021-11-11 18:04:02,283] Trial 0 finished with value: 161.8347337123744 and parameters: {'learning_rate_init ': 0.0651, 'first_layer_neurons': 20, 'second_layer_neurons': 40, 'activation': 'tanh'}. Best is trial 0 with value: 161.8347337123744.
# [I 2021-11-11 18:04:02,368] Trial 1 finished with value: 159.55535852658082 and parameters: {'learning_rate_init ': 0.0551, 'first_layer_neurons': 90, 'second_layer_neurons': 70, 'activation': 'relu'}. Best is trial 1 with value: 159.55535852658082.
# [I 2021-11-11 18:04:02,440] Trial 2 finished with value: 161.73980822730888 and parameters: {'learning_rate_init ': 0.0051, 'first_layer_neurons': 100, 'second_layer_neurons': 30, 'activation': 'identity'}. Best is trial 1 with value: 159.55535852658082.