mlr3中benchmarkresult如何获取模型结果？

Question

最近在学习使用mlr3包。我正在做一个特征选择，想比较模型 results.However，我想知道如何从基准结果中获得模型结果，比如在特征选择学习模型中使用了哪些特征。

这是我的代码：

#setup task
task = TaskClassif$new(id = "SPI", backend = test, target = "recidivism")

#learner of non-FS
learner_ranger = lrn("classif.ranger",
                     predict_type = "prob",
                     predict_sets = c("train", "test")) %>% print()

# set up for FS
terminator = trm("evals", n_evals = 20)
fselector = fs("random_search")

# create a new learner of FS
at = AutoFSelector$new(
  learner = learner_ranger,
  resampling = rsmp("holdout"),
  measure = msr("classif.auc"),
  terminator = terminator,
  fselector = fselector
)
at


# To compare the optimized feature subset with the complete feature set, we can use benchmark() :
grid = benchmark_grid(task = task,
                      learner = list(at,learner_ranger),
                      resampling = rsmp("holdout"))


# 
set.seed(1)
bmr_Wra = benchmark(grid, store_models = TRUE)

# 
measures = list(
  msr("classif.auc", id = "auc_train", predict_sets = "train"),
  msr("classif.auc", id = "auc_test"),
  msr("classif.acc", id = "acc_test"),
  msr("classif.ce",id = "ce_test")
)

bmr_Wra$aggregate(measures)


# ce
autoplot(bmr_Wra) + theme(axis.text.x = element_text(angle = 45, hjust = 1))

# roc
autoplot(bmr_Wra, type = "roc")

# prc
autoplot(bmr_Wra, type = "prc")`

我注意到了：

Note that it is not feasible to access learned models via this field, as the training task would be ambiguous. For this reason the returned learner are reseted before they are returned. Instead, select a row from the table returned by $score().

所以我使用了bmr_Wra$score()，但我仍然需要进一步分析模型并提取变量重要性等信息。任何帮助表示赞赏。非常感谢！

Answer 1

从 gh 安装最新的 mlr3fselect 版本，它提供了新的辅助功能。

extract_inner_fselect_results(bmr_Wra)

Returns 内部重采样循环的最佳特征集。由于您使用的是保留验证，因此它只是一组。

extract_inner_fselect_archives(bmr_Wra)

Returns 内部重采样循环的所有评估特征集。如果要访问存储在 ResampleResult 中的相应模型，请使用 store_models = TRUE 创建 AutoTuner。

您可能想阅读有关 nested resampling 的书籍章节。

mlr3中benchmarkresult如何获取模型结果？

How to get the model results from the benchmarkresult in mlr3?

r

mlr3