如何针对 Optuna 中的多个指标进行优化
How to optimize for multiple metrics in Optuna
如何在 Optuna 的 objective
函数中同时优化多个指标。例如,我正在训练一个 LGBM 分类器,并希望为所有常见的分类指标(如 F1、精度、召回率、准确性、AUC 等)找到最佳超参数集。
def objective(trial):
# Train
gbm = lgb.train(param, dtrain)
preds = gbm.predict(X_test)
pred_labels = np.rint(preds)
# Calculate metrics
accuracy = sklearn.metrics.accuracy_score(y_test, pred_labels)
recall = metrics.recall_score(pred_labels, y_test)
precision = metrics.precision_score(pred_labels, y_test)
f1 = metrics.f1_score(pred_labels, y_test, pos_label=1)
...
我该怎么做?
定义网格并使用这些参数拟合模型并生成预测后,计算要优化的所有指标:
def objective(trial):
param_grid = {"n_estimators": trial.suggest_int("n_estimators", 2000, 10000, step=200}
clf = lgbm.LGBMClassifier(objective='binary', **param_grid)
clf.fit(X_train, y_train)
preds = clf.predict(X_valid)
probs = clf.predict_proba(X_valid)
# Metrics
f1 = sklearn.metrics.f1_score(y_valid, press)
accuracy = ...
precision = ...
recall = ...
logloss =
和return他们按你想要的顺序排列:
def objective(trial):
...
return f1, logloss, accuracy, precision, recall
然后,在研究对象中,指定是要将每个指标最小化还是最大化到 directions
,如下所示:
study = optuna.create_study(directions=['maximize', 'minimize', 'maximize', 'maximize', 'maximize'])
study.optimize(objective, n_trials=100)
有关详细信息,请参阅文档中的 Multi-objective Optimization with Optuna。
如何在 Optuna 的 objective
函数中同时优化多个指标。例如,我正在训练一个 LGBM 分类器,并希望为所有常见的分类指标(如 F1、精度、召回率、准确性、AUC 等)找到最佳超参数集。
def objective(trial):
# Train
gbm = lgb.train(param, dtrain)
preds = gbm.predict(X_test)
pred_labels = np.rint(preds)
# Calculate metrics
accuracy = sklearn.metrics.accuracy_score(y_test, pred_labels)
recall = metrics.recall_score(pred_labels, y_test)
precision = metrics.precision_score(pred_labels, y_test)
f1 = metrics.f1_score(pred_labels, y_test, pos_label=1)
...
我该怎么做?
定义网格并使用这些参数拟合模型并生成预测后,计算要优化的所有指标:
def objective(trial):
param_grid = {"n_estimators": trial.suggest_int("n_estimators", 2000, 10000, step=200}
clf = lgbm.LGBMClassifier(objective='binary', **param_grid)
clf.fit(X_train, y_train)
preds = clf.predict(X_valid)
probs = clf.predict_proba(X_valid)
# Metrics
f1 = sklearn.metrics.f1_score(y_valid, press)
accuracy = ...
precision = ...
recall = ...
logloss =
和return他们按你想要的顺序排列:
def objective(trial):
...
return f1, logloss, accuracy, precision, recall
然后,在研究对象中,指定是要将每个指标最小化还是最大化到 directions
,如下所示:
study = optuna.create_study(directions=['maximize', 'minimize', 'maximize', 'maximize', 'maximize'])
study.optimize(objective, n_trials=100)
有关详细信息,请参阅文档中的 Multi-objective Optimization with Optuna。