如何执行 bootstrap 来查找 R 中 k-nn 模型的置信区间？

Question

我有一个包含 2 列的训练 df，例如

   a     b
1  1000  20
2  1008  13
...
n  ...   ...

现在，因为我需要根据特定 'a' 值找到 95% CI 的 'b' 估计值，'k' 值为我的选择并将 CI 结果与 'k's 的其他特定值进行比较。我的问题是如何用 1000 bootstrap 次重复执行 bootstrap 因为我需要使用拟合的 knn 模型来训练 kernel = 'gaussian' 并且 k 只能在范围 1-20 ? 我发现这个模型的最佳 k 是 k = 5，并且尝试 bootstrap 但它不起作用

library(kknn)
library(boot)

boot.kn = function(formula, data, indices)
{
  # Create a bootstrapped version
  d = data[indices,]
  
  # Fit a model for bs
  fit.kn =  fitted(train.kknn(formula,data, kernel= "gaussian", ks = 5))
  
  # Do I even need this complicated block
  target = as.character(fit.kn$terms[[2]])
  rv = my.pred.stats(fit.kn, d[,target])
  return(rv)
}
bs = boot(data=df, statistic=boot.kn, R=1000, formula=b ~ a)
boot.ci(bs,conf=0.95,type="bca")

如果我不够清楚，请告诉我更多信息。谢谢。

Answer 1

这是一种使用 k 最近邻算法在 a 上回归 b 的方法。

首先，一个数据集。这是 iris 数据集的子集，保留前两列。删除一行以备后用。

i <- which(iris$Sepal.Length == 5.3)
df1 <- iris[-i, 1:2]
newdata <- iris[i, 1:2]
names(df1) <- c("a", "b")
names(newdata) <- c("a", "b")

现在加载要使用的包并确定 k 包 kknn 的最佳值。

library(caret)
library(kknn)
library(boot)

fit <- kknn::train.kknn(
  formula = b ~ a,
  data = df1,
  kmax = 15,
  kernel = "gaussian",
  distance = 1
)
k <- fit$best.parameters$k
k
#[1] 9

和 bootstrap 对新点的预测 a <- 5.3。

boot.kn <- function(data, indices, formula, newdata, k){
  d <- data[indices, ]
  fit <- knnreg(formula, data = d)
  predict(fit, newdata = newdata)
}

set.seed(2021)
R <- 1e4
bs <- boot(df1, boot.kn, R = R, formula = b ~ a, newdata = newdata, k = k)
ci <- boot.ci(bs, level = 0.95, type = "bca")

ci
#BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS
#Based on 10000 bootstrap replicates
#
#CALL : 
#boot.ci(boot.out = bs, type = "bca", level = 0.95)
#
#Intervals : 
#Level       BCa          
#95%   ( 3.177,  3.740 )  
#Calculations and Intervals on Original Scale

绘制结果。

old_par <- par(mfrow = c(2, 1),
               oma = c(5, 4, 0, 0) + 0.1,
               mar = c(1, 1, 1, 1) + 0.1)

hist(bs$t, main = "Histogram of bootstrap values")
abline(v = 3.7, col = "red")
abline(v = mean(bs$t), col = "blue")
abline(v = ci$bca[4:5], col = "blue", lty = "dashed")

plot(b ~ a, df1)
points(5.3, 3.7, col = "red", pch = 19)
points(5.3, mean(bs$t), col = "blue", pch = 19)
arrows(x0 = 5.3, y0 = ci$bca[4],
       x1 = 5.3, y1 = ci$bca[5],
       col = "blue", angle = 90, code = 3)

par(old_par)

如何执行 bootstrap 来查找 R 中 k-nn 模型的置信区间？

How can I perform bootstrap to find the confidence interval for a k-nn model in R?

r

knn