在 mlr3 中使用 "classif.fbeta" 作为性能指标时如何更改 Beta 值？

Question

library(mlr3verse)
preformace_msr <- msr("classif.fbeta", beta = 1.5)

我正在尝试在 fbeta 度量中使用 BETA 的自定义值来调整分类模型。

但是上面尝试给出 beta 值的方法会在 mlr3 中引发错误。

在 mlr3 中正确的做法是什么？

Answer 1

所以报错如下：

library(mlr3verse)
preformace_msr <- msr("classif.fbeta", beta = 1.5)
#> Error: Cannot set argument 'beta' for 'MeasureBinaryimple' (not a constructor argument, not a parameter, not a field.

我认为这与 mlr3 中仅实施了 F1-Measure（而非 F1.5-measure 等）这一事实有关。参见 beta=1 in the source file。

在 mlr3 book 中，您将在 6.3 下找到以下内容：

In this section we showcase how to implement a custom performance measure.

根据本节，您可以通过以下方式实现 Fx-Measure（我认为我的语法不是 best/safest，但它应该有效）：

library(mlr3verse)
library(mlr3measures)
library(R6)

# make custom measure:
MeasureCustomFbeta = R6::R6Class("classif.custom_fbeta",
                             inherit = mlr3::MeasureClassif,
                             public = list(
                               #declase field
                               beta=NULL, #delcare field
                               initialize = function(beta) {
                                 self$beta <- beta
                                 super$initialize(
                                   # custom id for the measure
                                   id = "custom_fbeta",
                                  
                                   # required predict type of the learner
                                   predict_type = "response",
                                   
                                   # feasible range of values
                                   range = c(0, Inf),
                                   
                                   # minimize during tuning?
                                   minimize = TRUE
                                 )
                               }
                             ),
  
                             
                             private = list(
                               # customized scoring function operating on the prediction object
                               .score = function(prediction, ...) {
                                 fbeta_cm = function(m, beta) {
                                   pred_pos = sum(m[1L, ])
                                   cond_pos = sum(m[, 1L])
                                   if (m[1L, 1L] == 0L || pred_pos == 0L || cond_pos == 0L)
                                     return(na_value)
                                   
                                   P = m[1L, 1L] / pred_pos
                                   R = m[1L, 1L] / cond_pos
                                   ((1 + beta^2) * P * R) / ((beta^2 * P) + R)
                                 }
                                 fbeta_cm(confusion_matrix(prediction$truth, prediction$response, prediction$positive)$matrix, self$beta)
                                 
                               })
                               
)

# add it to the dictionary
mlr3::mlr_measures$add("classif.custom_fbeta", MeasureCustomFbeta)

测试：

# get data
data("Sonar", package = "mlbench")

# make task
task = TaskClassif$new(id = "Sonar", Sonar, target = "Class", positive = "R")

# make learner
learner = lrn("classif.rpart", predict_type = "response")

# predict
pred = learner$train(task)$predict(task)
pred$confusion
#>         truth
#> response  R  M
#>        R 87 16
#>        M 10 95

# measure "classif.beta
measure_old <- msr("classif.fbeta")
pred$score(measure_old)
#> classif.fbeta 
#>          0.87

# customized measure
measure_new <- msr("classif.custom_fbeta", beta=1.5)
pred$score(measure_new)
#> classif.custom_fbeta 
#>            0.8801556

Answer 2

直到现在，措施还没有参数。将在下一个版本中通过 https://github.com/mlr-org/mlr3/pull/623.

解决

在 mlr3 中使用 "classif.fbeta" 作为性能指标时如何更改 Beta 值？

how to change Beta value when using "classif.fbeta" as a performance measure in mlr3?

mlr3