分类变量的值上限为 53
Categorical Variable has a Limit of 53 Values
我正在使用 R 编程语言。我正在尝试将“随机森林”（一种统计模型）拟合到我的数据中，但问题是：我的一个分类变量有超过 53 个类别——显然 R 中的“随机森林”包不允许用户有超过 53 个类别，这使我无法在我的模型中使用此变量。理想情况下，我想使用这个变量。
为了说明这个例子，我创建了一个数据集（称为“数据”），其中一个变量有超过 53 个类别：
#load libraries
library(caret)
library(randomforest)
library(ranger)
#first data set
cat_var <- c("a","b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r",
"s", "t", "u", "v", "w", "x", "y", "z", "aa", "bb", "cc", "dd", "ee", "ff",
"gg", "hh", "ii", "jj", "kk", "ll", "mm", "nn", "oo", "pp", "qq", "rr", "ss", "tt", "uu", "vv", "ww", "xx", "yy", "zz", "aaa", "bbb")
var_1 <- rnorm(54,10,10)
var_2 <- rnorm(54, 5, 5)
var_3 <- rnorm(54, 6,18)
response <- c("a","b")
response <- sample(response, 54, replace=TRUE, prob=c(0.3, 0.7))
data_1 = data.frame(cat_var, var_1, var_2, var_3, response)
data_1$response = as.factor(data_1$response)
data_1$cat_var = as.factor(data_1$cat_var)
#second data set
cat_var <- c("a","b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r",
"s", "t", "u", "v", "w", "x", "y", "z", "aa", "bb", "cc", "dd", "ee", "ff",
"gg", "hh", "ii", "jj", "kk", "ll", "mm", "nn", "oo", "pp", "qq", "rr", "ss", "tt", "uu", "vv", "ww", "xx", "yy", "zz", "aaa", "bbb")
var_1 <- rnorm(54,10,10)
var_2 <- rnorm(54, 5, 5)
var_3 <- rnorm(54, 6,18)
response <- c("a","b")
response <- sample(response, 54, replace=TRUE, prob=c(0.3, 0.7))
data_2 = data.frame(cat_var, var_1, var_2, var_3, response)
data_2$response = as.factor(data_2$response)
data_2$cat_var = as.factor(data_2$cat_var)
# third data set
cat_var <- c("a","b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r",
"s", "t", "u", "v", "w", "x", "y", "z", "aa", "bb", "cc", "dd", "ee", "ff",
"gg", "hh", "ii", "jj", "kk", "ll", "mm", "nn", "oo", "pp", "qq", "rr", "ss", "tt", "uu", "vv", "ww", "xx", "yy", "zz", "aaa", "bbb")
var_1 <- rnorm(54,10,10)
var_2 <- rnorm(54, 5, 5)
var_3 <- rnorm(54, 6,18)
response <- c("a","b")
response <- sample(response, 54, replace=TRUE, prob=c(0.3, 0.7))
data_3 = data.frame(cat_var, var_1, var_2, var_3, response)
data_3$response = as.factor(data_3$response)
data_3$cat_var = as.factor(data_3$cat_var)
#combine data sets
data = rbind(data_1, data_2, data_3)
从这里开始，我对拟合随机森林模型很感兴趣。我查看了不同的 Whosebug 帖子（例如 R randomForest too many categories error even with fewer than 53 categories , R - Random Forest and more than 53 categories），这是我注意到的。
如果您尝试按原样拟合随机森林模型，会发生以下情况：
#random forest using the "Randomforest" library
rf = randomForest(response ~ var_1 + var_2 + var_3 + cat_var, data=data, ntree=50, mtry=2)
Error in randomForest.default(m, y, ...) :
Can not handle categorical predictors with more than 53 categories.
在其中一篇帖子中，一位用户建议使用“插入符号”库来拟合模型——显然插入符号模型没有 53 个类别的限制。这行得通，但我不确定这是否正确：
#random forest using the "caret" and "ranger" libraries: (are these correct?)
random_forest <- train(response ~.,
data = data,
method = 'ranger')
random_forest <- train(response ~.,
data = data,
method = 'rf')
最后，另一个用户建议使用“模型矩阵”方法，但我不确定我是否完全理解这种方法：
#model matrix method
dummyMat <- model.matrix(response ~ var_1 + var_2 + var_3 + cat_var, data, # set contrasts.arg to keep all levels
contrasts.arg = list(var_1 = contrasts(data$var_1, contrasts = T), var_3 = contrasts(data$var_3, contrasts = T), cat_var = contrasts(data$cat_var, contrasts = F)
var_2 = contrasts(data$var2, contrasts = T)))
data2 <- cbind(data, dummyMat[,c(4:ncol(dummyMat)]) # just removing intercept column
rf = randomForest(response ~ var_1 + var_2 + var_3 + cat_var, data=data2, ntree=50, mtry=2)
有人可以建议我如何解决这个问题吗？第二种方法（使用“插入符号”）是否正确？
谢谢
我可以告诉你 caret 方法是正确的。 caret 包含用于数据拆分、预处理、特征选择和模型调整以及重采样交叉验证的工具。在这里，我 post 一个使用 caret 包拟合模型的典型工作流程（以您 post 编辑的数据为例）。
首先，我们设置了一种交叉验证方法来调整所选模型的超参数（在您的情况下，ranger 和 randomForest 的调整参数均为 mtry，splitrule 和 min.node.size 对应 ranger）。在示例中，我选择 k-fold corss-validation 且 k=10
library(caret)
control <- trainControl(method="cv",number = 10)
然后我们创建一个网格，其中包含要调整的参数可以采用的可能值
rangergrid <- expand.grid(mtry=2:(ncol(data)-1),splitrule="extratrees",min.node.size=seq(0.1,1,0.1))
rfgrid <- expand.grid(mtry=2:(ncol(data)-1))
最后，我们拟合选择的模型：
random_forest_ranger <- train(response ~.,
data = data,
method = 'ranger',
trControl=control,
tuneGrid=rangergrid)
random_forest_rf <- train(response ~.,
data = data,
method = 'rf',
trControl=control,
tuneGrid=rfgrid)
train 函数的输出如下所示：
> random_forest_rf
Random Forest
162 samples
4 predictor
2 classes: 'a', 'b'
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 146, 146, 146, 145, 146, 146, ...
Resampling results across tuning parameters:
mtry Accuracy Kappa
2 0.6852941 0.00000000
3 0.6852941 0.00000000
4 0.6602941 -0.04499494
Accuracy was used to select the optimal model using the largest value.
The final value used for the model was mtry = 2.
有关 caret 软件包的更多信息，请在线查看 vignette
我正在使用 R 编程语言。我正在尝试将“随机森林”（一种统计模型）拟合到我的数据中，但问题是：我的一个分类变量有超过 53 个类别——显然 R 中的“随机森林”包不允许用户有超过 53 个类别，这使我无法在我的模型中使用此变量。理想情况下，我想使用这个变量。
为了说明这个例子，我创建了一个数据集（称为“数据”），其中一个变量有超过 53 个类别：
#load libraries
library(caret)
library(randomforest)
library(ranger)
#first data set
cat_var <- c("a","b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r",
"s", "t", "u", "v", "w", "x", "y", "z", "aa", "bb", "cc", "dd", "ee", "ff",
"gg", "hh", "ii", "jj", "kk", "ll", "mm", "nn", "oo", "pp", "qq", "rr", "ss", "tt", "uu", "vv", "ww", "xx", "yy", "zz", "aaa", "bbb")
var_1 <- rnorm(54,10,10)
var_2 <- rnorm(54, 5, 5)
var_3 <- rnorm(54, 6,18)
response <- c("a","b")
response <- sample(response, 54, replace=TRUE, prob=c(0.3, 0.7))
data_1 = data.frame(cat_var, var_1, var_2, var_3, response)
data_1$response = as.factor(data_1$response)
data_1$cat_var = as.factor(data_1$cat_var)
#second data set
cat_var <- c("a","b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r",
"s", "t", "u", "v", "w", "x", "y", "z", "aa", "bb", "cc", "dd", "ee", "ff",
"gg", "hh", "ii", "jj", "kk", "ll", "mm", "nn", "oo", "pp", "qq", "rr", "ss", "tt", "uu", "vv", "ww", "xx", "yy", "zz", "aaa", "bbb")
var_1 <- rnorm(54,10,10)
var_2 <- rnorm(54, 5, 5)
var_3 <- rnorm(54, 6,18)
response <- c("a","b")
response <- sample(response, 54, replace=TRUE, prob=c(0.3, 0.7))
data_2 = data.frame(cat_var, var_1, var_2, var_3, response)
data_2$response = as.factor(data_2$response)
data_2$cat_var = as.factor(data_2$cat_var)
# third data set
cat_var <- c("a","b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r",
"s", "t", "u", "v", "w", "x", "y", "z", "aa", "bb", "cc", "dd", "ee", "ff",
"gg", "hh", "ii", "jj", "kk", "ll", "mm", "nn", "oo", "pp", "qq", "rr", "ss", "tt", "uu", "vv", "ww", "xx", "yy", "zz", "aaa", "bbb")
var_1 <- rnorm(54,10,10)
var_2 <- rnorm(54, 5, 5)
var_3 <- rnorm(54, 6,18)
response <- c("a","b")
response <- sample(response, 54, replace=TRUE, prob=c(0.3, 0.7))
data_3 = data.frame(cat_var, var_1, var_2, var_3, response)
data_3$response = as.factor(data_3$response)
data_3$cat_var = as.factor(data_3$cat_var)
#combine data sets
data = rbind(data_1, data_2, data_3)
从这里开始，我对拟合随机森林模型很感兴趣。我查看了不同的 Whosebug 帖子（例如 R randomForest too many categories error even with fewer than 53 categories , R - Random Forest and more than 53 categories），这是我注意到的。
如果您尝试按原样拟合随机森林模型，会发生以下情况：
#random forest using the "Randomforest" library
rf = randomForest(response ~ var_1 + var_2 + var_3 + cat_var, data=data, ntree=50, mtry=2)
Error in randomForest.default(m, y, ...) :
Can not handle categorical predictors with more than 53 categories.
在其中一篇帖子中，一位用户建议使用“插入符号”库来拟合模型——显然插入符号模型没有 53 个类别的限制。这行得通，但我不确定这是否正确：
#random forest using the "caret" and "ranger" libraries: (are these correct?)
random_forest <- train(response ~.,
data = data,
method = 'ranger')
random_forest <- train(response ~.,
data = data,
method = 'rf')
最后，另一个用户建议使用“模型矩阵”方法，但我不确定我是否完全理解这种方法：
#model matrix method
dummyMat <- model.matrix(response ~ var_1 + var_2 + var_3 + cat_var, data, # set contrasts.arg to keep all levels
contrasts.arg = list(var_1 = contrasts(data$var_1, contrasts = T), var_3 = contrasts(data$var_3, contrasts = T), cat_var = contrasts(data$cat_var, contrasts = F)
var_2 = contrasts(data$var2, contrasts = T)))
data2 <- cbind(data, dummyMat[,c(4:ncol(dummyMat)]) # just removing intercept column
rf = randomForest(response ~ var_1 + var_2 + var_3 + cat_var, data=data2, ntree=50, mtry=2)
有人可以建议我如何解决这个问题吗？第二种方法（使用“插入符号”）是否正确？
谢谢
我可以告诉你 caret 方法是正确的。 caret 包含用于数据拆分、预处理、特征选择和模型调整以及重采样交叉验证的工具。在这里，我 post 一个使用 caret 包拟合模型的典型工作流程（以您 post 编辑的数据为例）。
首先，我们设置了一种交叉验证方法来调整所选模型的超参数（在您的情况下，ranger 和 randomForest 的调整参数均为 mtry，splitrule 和 min.node.size 对应 ranger）。在示例中，我选择 k-fold corss-validation 且 k=10
library(caret)
control <- trainControl(method="cv",number = 10)
然后我们创建一个网格，其中包含要调整的参数可以采用的可能值
rangergrid <- expand.grid(mtry=2:(ncol(data)-1),splitrule="extratrees",min.node.size=seq(0.1,1,0.1))
rfgrid <- expand.grid(mtry=2:(ncol(data)-1))
最后，我们拟合选择的模型：
random_forest_ranger <- train(response ~.,
data = data,
method = 'ranger',
trControl=control,
tuneGrid=rangergrid)
random_forest_rf <- train(response ~.,
data = data,
method = 'rf',
trControl=control,
tuneGrid=rfgrid)
train 函数的输出如下所示：
> random_forest_rf
Random Forest
162 samples
4 predictor
2 classes: 'a', 'b'
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 146, 146, 146, 145, 146, 146, ...
Resampling results across tuning parameters:
mtry Accuracy Kappa
2 0.6852941 0.00000000
3 0.6852941 0.00000000
4 0.6602941 -0.04499494
Accuracy was used to select the optimal model using the largest value.
The final value used for the model was mtry = 2.
有关 caret 软件包的更多信息，请在线查看 vignette