如何强制 cva.glmnet() 在弹性网络回归模型中包含特定变量?
How to force cva.glmnet() to include speficied variables in elastic net regression mdoel?
我想在弹性网络回归模型中强制一些自变量。在cva.glmnet()函数中使用penalty.factor时出错。如有任何建议和意见,我们将不胜感激。
# --------------- import data
# outcome: am
# ten predictors: mpg+cyl+disp+hp+drat+wt+qsec+vs+gear+carb
mydata<-mtcars
mydata$am<-factor(mydata$am,levels=c(0,1),labels=c("no","yes"))
mydata$gear<-factor(mydata$gear,levels=c(3,4,5),labels=c("level3","level4","level5"))
str(mydata)
'data.frame': 32 obs. of 11 variables:
$ mpg : num 21 21 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 ...
$ cyl : num 6 6 4 6 8 6 8 4 4 6 ...
$ disp: num 160 160 108 258 360 ...
$ hp : num 110 110 93 110 175 105 245 62 95 123 ...
$ drat: num 3.9 3.9 3.85 3.08 3.15 2.76 3.21 3.69 3.92 3.92 ...
$ wt : num 2.62 2.88 2.32 3.21 3.44 ...
$ qsec: num 16.5 17 18.6 19.4 17 ...
$ vs : num 0 0 1 1 0 1 0 1 1 1 ...
$ am : Factor w/ 2 levels "no","yes": 2 2 2 1 1 1 1 1 1 1 ...
$ gear: Factor w/ 3 levels "level3","level4",..: 2 2 2 1 1 1 1 2 2 2 ...
$ carb: num 4 4 1 1 2 1 4 2 2 4 ...
# --------------- do elastic net cross-validation for alpha and lambda simultaneously
# works
library("glmnetUtils")
set.seed(12345)
cvfit<-cva.glmnet(am~mpg+cyl+disp+hp+drat+wt+qsec+vs+gear+carb,
family="binomial",
alpha=seq(from=0,to=1,by=0.05),
nfolds=10,
data=mydata)
# --------------- force three variables of "cyl", "disp", "hp" in the final model
# does not work
set.seed(12345)
cvfit<-cva.glmnet(am~mpg+cyl+disp+hp+drat+wt+qsec+vs+gear+carb,
family="binomial",
penalty.factor=c(0,0,0,1,1,1,1,1,1,1),
alpha=seq(from=0,to=1,by=0.05),
nfolds=10,
data=mydata)
Error in approx(lambda, seq(lambda), sfrac) :
need at least two non-NA values to interpolate
此外,由于齿轮是一个因素,您需要在 penalty.factor 中再增加 2 个。并且 cva.glmnet()
保留一个因子的所有虚拟变量,而不是省略一个,以便正则化将影响缩小到总体均值。如果省略 1 级,则将剩余系数缩小为 0 会迫使预测接近基线水平(参见作者的以下评论)。
无论如何,您可以查看第一个示例:
set.seed(12345)
cvfit<-cva.glmnet(am~mpg+cyl+disp+hp+drat+wt+qsec+vs+gear+carb,
family="binomial",
alpha=seq(from=0,to=1,by=0.05),
nfolds=10,
data=mydata)
dim(cvfit$modlist[[1]]$glmnet.fit$beta)
[1] 12 100
因此 penalty.factor 需要向量 12。如果我 运行 下面,它有效:
cvfit<-cva.glmnet(am~mpg+cyl+disp+hp+drat+wt+qsec+vs+gear+carb,
family="binomial",
penalty.factor=c(0,0,0,1,1,1,1,1,1,1,1,1),
alpha=seq(from=0,to=1,by=0.05),
nfolds=10,
data=mydata)
您可以看到前 3 个变量永远不会为零:
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13]
mpg 0 0 0 0 0 0 0 0 0 0 0 0 0
cyl 0 0 0 0 0 0 0 0 0 0 0 0 0
disp 0 0 0 0 0 0 0 0 0 0 0 0 0
hp 0 6 7 7 7 8 9 9 10 11 12 14 15
drat 0 7 7 7 8 8 8 9 10 10 11 13 14
wt 0 13 14 15 16 17 18 19 20 21 22 23 25
qsec 0 1 1 1 1 1 1 1 1 1 1 1 1
vs 0 11 12 13 14 15 15 16 17 17 18 19 19
gearlevel3 0 4 4 5 5 5 5 5 5 6 6 6 7
gearlevel4 0 50 49 50 51 52 53 55 58 61 65 70 77
gearlevel5 0 1 1 1 1 1 1 1 1 1 1 1 1
carb 0 12 17 23 29 37 46 62 65 70 75 80 88
[,14] [,15] [,16] [,17] [,18] [,19] [,20] [,21]
mpg 0 0 0 0 0 0 0 0
cyl 0 0 0 0 0 0 0 0
disp 0 0 0 0 0 0 0 0
hp 17 20 22 26 32 40 62 73
drat 16 19 23 27 33 41 58 73
wt 26 28 30 33 38 48 78 73
qsec 1 1 1 1 1 1 1 2
vs 20 22 23 25 29 35 58 73
gearlevel3 7 8 8 9 11 14 37 73
gearlevel4 84 87 85 84 82 80 78 73
gearlevel5 1 1 1 1 1 1 1 1
carb 88 87 85 84 82 80 78 73
我想在弹性网络回归模型中强制一些自变量。在cva.glmnet()函数中使用penalty.factor时出错。如有任何建议和意见,我们将不胜感激。
# --------------- import data
# outcome: am
# ten predictors: mpg+cyl+disp+hp+drat+wt+qsec+vs+gear+carb
mydata<-mtcars
mydata$am<-factor(mydata$am,levels=c(0,1),labels=c("no","yes"))
mydata$gear<-factor(mydata$gear,levels=c(3,4,5),labels=c("level3","level4","level5"))
str(mydata)
'data.frame': 32 obs. of 11 variables:
$ mpg : num 21 21 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 ...
$ cyl : num 6 6 4 6 8 6 8 4 4 6 ...
$ disp: num 160 160 108 258 360 ...
$ hp : num 110 110 93 110 175 105 245 62 95 123 ...
$ drat: num 3.9 3.9 3.85 3.08 3.15 2.76 3.21 3.69 3.92 3.92 ...
$ wt : num 2.62 2.88 2.32 3.21 3.44 ...
$ qsec: num 16.5 17 18.6 19.4 17 ...
$ vs : num 0 0 1 1 0 1 0 1 1 1 ...
$ am : Factor w/ 2 levels "no","yes": 2 2 2 1 1 1 1 1 1 1 ...
$ gear: Factor w/ 3 levels "level3","level4",..: 2 2 2 1 1 1 1 2 2 2 ...
$ carb: num 4 4 1 1 2 1 4 2 2 4 ...
# --------------- do elastic net cross-validation for alpha and lambda simultaneously
# works
library("glmnetUtils")
set.seed(12345)
cvfit<-cva.glmnet(am~mpg+cyl+disp+hp+drat+wt+qsec+vs+gear+carb,
family="binomial",
alpha=seq(from=0,to=1,by=0.05),
nfolds=10,
data=mydata)
# --------------- force three variables of "cyl", "disp", "hp" in the final model
# does not work
set.seed(12345)
cvfit<-cva.glmnet(am~mpg+cyl+disp+hp+drat+wt+qsec+vs+gear+carb,
family="binomial",
penalty.factor=c(0,0,0,1,1,1,1,1,1,1),
alpha=seq(from=0,to=1,by=0.05),
nfolds=10,
data=mydata)
Error in approx(lambda, seq(lambda), sfrac) :
need at least two non-NA values to interpolate
此外,由于齿轮是一个因素,您需要在 penalty.factor 中再增加 2 个。并且 cva.glmnet()
保留一个因子的所有虚拟变量,而不是省略一个,以便正则化将影响缩小到总体均值。如果省略 1 级,则将剩余系数缩小为 0 会迫使预测接近基线水平(参见作者的以下评论)。
无论如何,您可以查看第一个示例:
set.seed(12345)
cvfit<-cva.glmnet(am~mpg+cyl+disp+hp+drat+wt+qsec+vs+gear+carb,
family="binomial",
alpha=seq(from=0,to=1,by=0.05),
nfolds=10,
data=mydata)
dim(cvfit$modlist[[1]]$glmnet.fit$beta)
[1] 12 100
因此 penalty.factor 需要向量 12。如果我 运行 下面,它有效:
cvfit<-cva.glmnet(am~mpg+cyl+disp+hp+drat+wt+qsec+vs+gear+carb,
family="binomial",
penalty.factor=c(0,0,0,1,1,1,1,1,1,1,1,1),
alpha=seq(from=0,to=1,by=0.05),
nfolds=10,
data=mydata)
您可以看到前 3 个变量永远不会为零:
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13]
mpg 0 0 0 0 0 0 0 0 0 0 0 0 0
cyl 0 0 0 0 0 0 0 0 0 0 0 0 0
disp 0 0 0 0 0 0 0 0 0 0 0 0 0
hp 0 6 7 7 7 8 9 9 10 11 12 14 15
drat 0 7 7 7 8 8 8 9 10 10 11 13 14
wt 0 13 14 15 16 17 18 19 20 21 22 23 25
qsec 0 1 1 1 1 1 1 1 1 1 1 1 1
vs 0 11 12 13 14 15 15 16 17 17 18 19 19
gearlevel3 0 4 4 5 5 5 5 5 5 6 6 6 7
gearlevel4 0 50 49 50 51 52 53 55 58 61 65 70 77
gearlevel5 0 1 1 1 1 1 1 1 1 1 1 1 1
carb 0 12 17 23 29 37 46 62 65 70 75 80 88
[,14] [,15] [,16] [,17] [,18] [,19] [,20] [,21]
mpg 0 0 0 0 0 0 0 0
cyl 0 0 0 0 0 0 0 0
disp 0 0 0 0 0 0 0 0
hp 17 20 22 26 32 40 62 73
drat 16 19 23 27 33 41 58 73
wt 26 28 30 33 38 48 78 73
qsec 1 1 1 1 1 1 1 2
vs 20 22 23 25 29 35 58 73
gearlevel3 7 8 8 9 11 14 37 73
gearlevel4 84 87 85 84 82 80 78 73
gearlevel5 1 1 1 1 1 1 1 1
carb 88 87 85 84 82 80 78 73