为“hidden_dropout_ratios”超参数指定“makeNumericVectorParam”，这取决于隐藏层的数量

Question

我想通过 mlr 调整 "classif.h2o.deeplearning" 学习者。在调优过程中，我有几个想要探索的架构。对于这些架构中的每一个，我想指定一个 dropout space。但是我正在为此苦苦挣扎。

示例：

library(mlr)
library(h2o)

ctrl <- makeTuneControlRandom(maxit = 10) 

lrn <- makeLearner("classif.h2o.deeplearning", predict.type = "prob")

我通过 "hidden" DiscreteParam 定义了两个架构 "a" 和 "b"，我想为每个架构创建一个 "hidden_dropout_ratios"[=20= 的 NumericVectorParam ]

par_set <- makeParamSet(
  makeDiscreteParam("hidden", values = list(a = c(16L, 16L),
                                            b = c(16L, 16L, 16L))),
  makeDiscreteParam("activation", values = "RectifierWithDropout", tunable = FALSE),
  makeNumericParam("input_dropout_ratio", lower = 0, upper = 0.4, default = 0.1),
  makeNumericVectorParam("hidden_dropout_ratios", len = 2, lower = 0, upper = 0.6, default = rep(0.3, 2),
                         requires = quote(length(hidden) == 2)),
  makeNumericVectorParam("hidden_dropout_ratios", len = 3, lower = 0, upper = 0.6, default = rep(0.3, 3),
                         requires = quote(length(hidden) == 3)))

这会产生错误：

Error in makeParamSet(makeDiscreteParam("hidden", values = list(a = c(16L,  : 
All parameters must have unique names!

仅设置其中之一会导致 dropout 仅应用于具有适当隐藏层数的架构。

当我尝试对所有隐藏层使用相同的丢弃时：

par_set <- makeParamSet(
  makeDiscreteParam("hidden", values = list(a = c(16L, 16L),
                                            b = c(16L, 16L, 16L))),
  makeDiscreteParam("activation", values = "RectifierWithDropout", tunable = FALSE),
  makeNumericParam("input_dropout_ratio", lower = 0, upper = 0.4, default = 0.1),
  makeNumericParam("hidden_dropout_ratios", lower = 0, upper = 0.6, default = 0.3))

tw <- makeTuneWrapper(lrn,
                      resampling = cv3,
                      control = ctrl,
                      par.set = par_set,
                      show.info = TRUE,
                      measures = list(auc,
                                      bac))

perf_tw <- resample(tw, 
                     task = sonar.task,
                     resampling = cv5,
                     extract = getTuneResult,
                     models = TRUE,
                     show.info = TRUE,
                     measures = list(auc,
                                     bac))

我收到错误：

Error in .h2o.doSafeREST(h2oRestApiVersion = h2oRestApiVersion, urlSuffix = page,  : 
ERROR MESSAGE:

Illegal argument(s) for DeepLearning model: DeepLearning_model_R_1566289564965_2.  Details: ERRR on field: _hidden_dropout_ratios: Must have 3 hidden layer dropout ratios.

也许我可以通过为每个架构创建一个单独的学习器然后结合 makeModelMultiplexer?

来克服这个问题

我需要你的帮助来克服这个问题。谢谢。

编辑：我能够使用 makeModelMultiplexer 并通过为每个架构（隐藏层数）创建一个学习器来克服这个问题。

base_lrn <- list(
  makeLearner("classif.h2o.deeplearning",
              id = "h20_2",
              predict.type = "prob"),
  makeLearner("classif.h2o.deeplearning",
              id = "h20_3",
              predict.type = "prob"))

mm_lrn <- makeModelMultiplexer(base_lrn)

par_set <- makeParamSet(
  makeDiscreteParam("selected.learner", values = extractSubList(base_lrn, "id")),
  makeDiscreteParam("h20_2.hidden", values = list(a = c(16L, 16L),
                                                  b = c(32L, 32L)),
                    requires = quote(selected.learner == "h20_2")),
  makeDiscreteParam("h20_3.hidden", values = list(a = c(16L, 16L, 16L),
                                                  b = c(32L, 32L, 32L)),
                    requires = quote(selected.learner == "h20_3")),
  makeDiscreteParam("h20_2.activation", values = "RectifierWithDropout", tunable = FALSE,
                    requires = quote(selected.learner == "h20_2")),
  makeDiscreteParam("h20_3.activation", values = "RectifierWithDropout", tunable = FALSE,
                    requires = quote(selected.learner == "h20_3")),
  makeNumericParam("h20_2.input_dropout_ratio", lower = 0, upper = 0.4, default = 0.1,
                   requires = quote(selected.learner == "h20_2")),
  makeNumericParam("h20_3.input_dropout_ratio", lower = 0, upper = 0.4, default = 0.1,
                   requires = quote(selected.learner == "h20_3")),
  makeNumericVectorParam("h20_2.hidden_dropout_ratios", len = 2, lower = 0, upper = 0.6, default = rep(0.3, 2),
                         requires = quote(selected.learner == "h20_2")),
  makeNumericVectorParam("h20_3.hidden_dropout_ratios", len = 3, lower = 0, upper = 0.6, default = rep(0.3, 3),
                         requires = quote(selected.learner == "h20_3")))

tw <- makeTuneWrapper(mm_lrn,
                      resampling = cv3,
                      control = ctrl,
                      par.set = par_set,
                      show.info = TRUE,
                      measures = list(auc,
                                      bac))

perf_tw <- resample(tw, 
                    task = sonar.task,
                    resampling = cv5,
                    extract = getTuneResult,
                    models = TRUE,
                    show.info = TRUE,
                    measures = list(auc,
                                    bac))

有没有更优雅的方案？

Answer 1

我没有使用 h2o 学习者或他们的深度学习方法的经验。

但是，在单个 ParamSet 中指定相同的参数两次（作为您的第一次尝试）将不起作用。所以无论如何你总是需要使用两个ParamSets。

关于您遇到的第二个错误，我无话可说。这看起来像是一个与 h2o 相关的问题。

使用 makeModelMultiplexer() 是一种选择。您还可以使用单个 benchmark() 调用并在之后聚合它们。

为“hidden_dropout_ratios”超参数指定“makeNumericVectorParam”，这取决于隐藏层的数量

specify `makeNumericVectorParam` for `hidden_dropout_ratios` hyper parameter which would depend on the number of hidden layers

r

h2o

mlr