TidyModels 中的工作流程如何将配方步骤应用于 CV 折叠？

Question

为了控制我正在进行的项目的少数积极 class，我在我的食谱中实施 step_downsample()。我还使用 10 折交叉验证来减轻偏差。当我使用工作流来包装学习者、食谱、网格搜索和 CV 折叠时，工作流是否会在模型训练之前将食谱步骤应用于每个单独的折叠？操作顺序对我来说很模糊，我无法在文档中找到任何满意的答案。谢谢！

Answer 1

我想你可能会发现 this chapter helpful, especially the section "Where does the model begin and end?"。

是，在 tidymodels 中，预处理方法（即特征工程过程）被认为是建模过程的一部分，并且像学习者一样在每个折叠上进行训练。

如果您在调整期间设置 verbose = TRUE，您可以在日志记录中看到这种情况：

library(tidymodels)
library(themis)
#> 
#> Attaching package: 'themis'
#> The following objects are masked from 'package:recipes':
#> 
#>     step_downsample, step_upsample
data(Ionosphere, package = "mlbench")

svm_mod <-
  svm_rbf(cost = tune(), rbf_sigma = tune()) %>%
  set_mode("classification") %>%
  set_engine("kernlab")

iono_rec <-
  recipe(Class ~ ., data = Ionosphere)  %>%
  # remove any zero variance predictors
  step_zv(all_predictors()) %>% 
  # remove any linear combinations
  step_lincomb(all_numeric()) %>%
  step_downsample(Class)

set.seed(123)
iono_rs <- bootstraps(Ionosphere, times = 5)

set.seed(325)
svm_mod %>% 
  tune_grid(
    iono_rec,
    resamples = iono_rs,
    control = control_grid(verbose = TRUE)
  )
#> i Bootstrap1: preprocessor 1/1
#> ✓ Bootstrap1: preprocessor 1/1
#> i Bootstrap1: preprocessor 1/1, model 1/10
#> ✓ Bootstrap1: preprocessor 1/1, model 1/10
#> i Bootstrap1: preprocessor 1/1, model 1/10 (predictions)
#> i Bootstrap1: preprocessor 1/1, model 2/10
#> ✓ Bootstrap1: preprocessor 1/1, model 2/10
#> i Bootstrap1: preprocessor 1/1, model 2/10 (predictions)
#> i Bootstrap1: preprocessor 1/1, model 3/10
#> ✓ Bootstrap1: preprocessor 1/1, model 3/10
#> i Bootstrap1: preprocessor 1/1, model 3/10 (predictions)
#> i Bootstrap1: preprocessor 1/1, model 4/10
#> ✓ Bootstrap1: preprocessor 1/1, model 4/10
#> i Bootstrap1: preprocessor 1/1, model 4/10 (predictions)
#> i Bootstrap1: preprocessor 1/1, model 5/10
#> ✓ Bootstrap1: preprocessor 1/1, model 5/10
#> i Bootstrap1: preprocessor 1/1, model 5/10 (predictions)
#> i Bootstrap1: preprocessor 1/1, model 6/10
#> ✓ Bootstrap1: preprocessor 1/1, model 6/10
#> i Bootstrap1: preprocessor 1/1, model 6/10 (predictions)
#> i Bootstrap1: preprocessor 1/1, model 7/10
#> ✓ Bootstrap1: preprocessor 1/1, model 7/10
#> i Bootstrap1: preprocessor 1/1, model 7/10 (predictions)
#> i Bootstrap1: preprocessor 1/1, model 8/10
#> ✓ Bootstrap1: preprocessor 1/1, model 8/10
#> i Bootstrap1: preprocessor 1/1, model 8/10 (predictions)
#> i Bootstrap1: preprocessor 1/1, model 9/10
#> ✓ Bootstrap1: preprocessor 1/1, model 9/10
#> i Bootstrap1: preprocessor 1/1, model 9/10 (predictions)
#> i Bootstrap1: preprocessor 1/1, model 10/10
#> ✓ Bootstrap1: preprocessor 1/1, model 10/10
#> i Bootstrap1: preprocessor 1/1, model 10/10 (predictions)
#> i Bootstrap2: preprocessor 1/1
#> ✓ Bootstrap2: preprocessor 1/1
#> i Bootstrap2: preprocessor 1/1, model 1/10
#> ✓ Bootstrap2: preprocessor 1/1, model 1/10
#> i Bootstrap2: preprocessor 1/1, model 1/10 (predictions)
#> i Bootstrap2: preprocessor 1/1, model 2/10
#> ✓ Bootstrap2: preprocessor 1/1, model 2/10
#> i Bootstrap2: preprocessor 1/1, model 2/10 (predictions)
#> i Bootstrap2: preprocessor 1/1, model 3/10
#> ✓ Bootstrap2: preprocessor 1/1, model 3/10
#> i Bootstrap2: preprocessor 1/1, model 3/10 (predictions)
#> i Bootstrap2: preprocessor 1/1, model 4/10
#> ✓ Bootstrap2: preprocessor 1/1, model 4/10
#> i Bootstrap2: preprocessor 1/1, model 4/10 (predictions)
#> i Bootstrap2: preprocessor 1/1, model 5/10
#> ✓ Bootstrap2: preprocessor 1/1, model 5/10
#> i Bootstrap2: preprocessor 1/1, model 5/10 (predictions)
#> i Bootstrap2: preprocessor 1/1, model 6/10
#> ✓ Bootstrap2: preprocessor 1/1, model 6/10
#> i Bootstrap2: preprocessor 1/1, model 6/10 (predictions)
#> i Bootstrap2: preprocessor 1/1, model 7/10
#> ✓ Bootstrap2: preprocessor 1/1, model 7/10
#> i Bootstrap2: preprocessor 1/1, model 7/10 (predictions)
#> i Bootstrap2: preprocessor 1/1, model 8/10
#> ✓ Bootstrap2: preprocessor 1/1, model 8/10
#> i Bootstrap2: preprocessor 1/1, model 8/10 (predictions)
#> i Bootstrap2: preprocessor 1/1, model 9/10
#> ✓ Bootstrap2: preprocessor 1/1, model 9/10
#> i Bootstrap2: preprocessor 1/1, model 9/10 (predictions)
#> i Bootstrap2: preprocessor 1/1, model 10/10
#> ✓ Bootstrap2: preprocessor 1/1, model 10/10
#> i Bootstrap2: preprocessor 1/1, model 10/10 (predictions)
#> i Bootstrap3: preprocessor 1/1
#> ✓ Bootstrap3: preprocessor 1/1
#> i Bootstrap3: preprocessor 1/1, model 1/10
#> ✓ Bootstrap3: preprocessor 1/1, model 1/10
#> i Bootstrap3: preprocessor 1/1, model 1/10 (predictions)
#> i Bootstrap3: preprocessor 1/1, model 2/10
#> ✓ Bootstrap3: preprocessor 1/1, model 2/10
#> i Bootstrap3: preprocessor 1/1, model 2/10 (predictions)
#> i Bootstrap3: preprocessor 1/1, model 3/10
#> ✓ Bootstrap3: preprocessor 1/1, model 3/10
#> i Bootstrap3: preprocessor 1/1, model 3/10 (predictions)
#> i Bootstrap3: preprocessor 1/1, model 4/10
#> ✓ Bootstrap3: preprocessor 1/1, model 4/10
#> i Bootstrap3: preprocessor 1/1, model 4/10 (predictions)
#> i Bootstrap3: preprocessor 1/1, model 5/10
#> ✓ Bootstrap3: preprocessor 1/1, model 5/10
#> i Bootstrap3: preprocessor 1/1, model 5/10 (predictions)
#> i Bootstrap3: preprocessor 1/1, model 6/10
#> ✓ Bootstrap3: preprocessor 1/1, model 6/10
#> i Bootstrap3: preprocessor 1/1, model 6/10 (predictions)
#> i Bootstrap3: preprocessor 1/1, model 7/10
#> ✓ Bootstrap3: preprocessor 1/1, model 7/10
#> i Bootstrap3: preprocessor 1/1, model 7/10 (predictions)
#> i Bootstrap3: preprocessor 1/1, model 8/10
#> ✓ Bootstrap3: preprocessor 1/1, model 8/10
#> i Bootstrap3: preprocessor 1/1, model 8/10 (predictions)
#> i Bootstrap3: preprocessor 1/1, model 9/10
#> ✓ Bootstrap3: preprocessor 1/1, model 9/10
#> i Bootstrap3: preprocessor 1/1, model 9/10 (predictions)
#> i Bootstrap3: preprocessor 1/1, model 10/10
#> ✓ Bootstrap3: preprocessor 1/1, model 10/10
#> i Bootstrap3: preprocessor 1/1, model 10/10 (predictions)
#> i Bootstrap4: preprocessor 1/1
#> ✓ Bootstrap4: preprocessor 1/1
#> i Bootstrap4: preprocessor 1/1, model 1/10
#> ✓ Bootstrap4: preprocessor 1/1, model 1/10
#> i Bootstrap4: preprocessor 1/1, model 1/10 (predictions)
#> i Bootstrap4: preprocessor 1/1, model 2/10
#> ✓ Bootstrap4: preprocessor 1/1, model 2/10
#> i Bootstrap4: preprocessor 1/1, model 2/10 (predictions)
#> i Bootstrap4: preprocessor 1/1, model 3/10
#> ✓ Bootstrap4: preprocessor 1/1, model 3/10
#> i Bootstrap4: preprocessor 1/1, model 3/10 (predictions)
#> i Bootstrap4: preprocessor 1/1, model 4/10
#> ✓ Bootstrap4: preprocessor 1/1, model 4/10
#> i Bootstrap4: preprocessor 1/1, model 4/10 (predictions)
#> i Bootstrap4: preprocessor 1/1, model 5/10
#> ✓ Bootstrap4: preprocessor 1/1, model 5/10
#> i Bootstrap4: preprocessor 1/1, model 5/10 (predictions)
#> i Bootstrap4: preprocessor 1/1, model 6/10
#> ✓ Bootstrap4: preprocessor 1/1, model 6/10
#> i Bootstrap4: preprocessor 1/1, model 6/10 (predictions)
#> i Bootstrap4: preprocessor 1/1, model 7/10
#> ✓ Bootstrap4: preprocessor 1/1, model 7/10
#> i Bootstrap4: preprocessor 1/1, model 7/10 (predictions)
#> i Bootstrap4: preprocessor 1/1, model 8/10
#> ✓ Bootstrap4: preprocessor 1/1, model 8/10
#> i Bootstrap4: preprocessor 1/1, model 8/10 (predictions)
#> i Bootstrap4: preprocessor 1/1, model 9/10
#> ✓ Bootstrap4: preprocessor 1/1, model 9/10
#> i Bootstrap4: preprocessor 1/1, model 9/10 (predictions)
#> i Bootstrap4: preprocessor 1/1, model 10/10
#> ✓ Bootstrap4: preprocessor 1/1, model 10/10
#> i Bootstrap4: preprocessor 1/1, model 10/10 (predictions)
#> i Bootstrap5: preprocessor 1/1
#> ✓ Bootstrap5: preprocessor 1/1
#> i Bootstrap5: preprocessor 1/1, model 1/10
#> ✓ Bootstrap5: preprocessor 1/1, model 1/10
#> i Bootstrap5: preprocessor 1/1, model 1/10 (predictions)
#> i Bootstrap5: preprocessor 1/1, model 2/10
#> ✓ Bootstrap5: preprocessor 1/1, model 2/10
#> i Bootstrap5: preprocessor 1/1, model 2/10 (predictions)
#> i Bootstrap5: preprocessor 1/1, model 3/10
#> ✓ Bootstrap5: preprocessor 1/1, model 3/10
#> i Bootstrap5: preprocessor 1/1, model 3/10 (predictions)
#> i Bootstrap5: preprocessor 1/1, model 4/10
#> ✓ Bootstrap5: preprocessor 1/1, model 4/10
#> i Bootstrap5: preprocessor 1/1, model 4/10 (predictions)
#> i Bootstrap5: preprocessor 1/1, model 5/10
#> ✓ Bootstrap5: preprocessor 1/1, model 5/10
#> i Bootstrap5: preprocessor 1/1, model 5/10 (predictions)
#> i Bootstrap5: preprocessor 1/1, model 6/10
#> ✓ Bootstrap5: preprocessor 1/1, model 6/10
#> i Bootstrap5: preprocessor 1/1, model 6/10 (predictions)
#> i Bootstrap5: preprocessor 1/1, model 7/10
#> ✓ Bootstrap5: preprocessor 1/1, model 7/10
#> i Bootstrap5: preprocessor 1/1, model 7/10 (predictions)
#> i Bootstrap5: preprocessor 1/1, model 8/10
#> ✓ Bootstrap5: preprocessor 1/1, model 8/10
#> i Bootstrap5: preprocessor 1/1, model 8/10 (predictions)
#> i Bootstrap5: preprocessor 1/1, model 9/10
#> ✓ Bootstrap5: preprocessor 1/1, model 9/10
#> i Bootstrap5: preprocessor 1/1, model 9/10 (predictions)
#> i Bootstrap5: preprocessor 1/1, model 10/10
#> ✓ Bootstrap5: preprocessor 1/1, model 10/10
#> i Bootstrap5: preprocessor 1/1, model 10/10 (predictions)
#> # Tuning results
#> # Bootstrap sampling 
#> # A tibble: 5 x 4
#>   splits            id         .metrics          .notes          
#>   <list>            <chr>      <list>            <list>          
#> 1 <split [351/136]> Bootstrap1 <tibble [20 × 6]> <tibble [0 × 1]>
#> 2 <split [351/124]> Bootstrap2 <tibble [20 × 6]> <tibble [0 × 1]>
#> 3 <split [351/134]> Bootstrap3 <tibble [20 × 6]> <tibble [0 × 1]>
#> 4 <split [351/130]> Bootstrap4 <tibble [20 × 6]> <tibble [0 × 1]>
#> 5 <split [351/138]> Bootstrap5 <tibble [20 × 6]> <tibble [0 × 1]>

^{由 reprex package (v1.0.0)}

于 2021 年 3 月 10 日创建

在这种特殊情况下，它遍历每个重采样并首先训练配方，然后适合第一个 learner/parameter 选项，然后评估该重采样和 learner/parameter 选项的保留集的预测，然后转到下一个学习者选项。

TidyModels 中的工作流程如何将配方步骤应用于 CV 折叠？

How do workflows in TidyModels apply recipe steps to CV folds?

tidymodels