pROC：灵敏度下限未正确显示在 ROC 曲线上

Question

我在我的代码中找不到错误：在 0.3173677 的阈值下，应绘制以下灵敏度 + 95 % CI：

--> 低：0.70590 中值：0.8235 上值：0.9412

不幸的是，se 的下界没有绘制正确。正如您在图中看到的那样，它大约是 0.68，应该更接近 0.71。我的其他地块没有这个问题。我已将我的其他代码与您在此 post 中看到的代码进行了比较，但我找不到错误。

希望你能帮帮我。

  y_pred_gesund = c(0.00578237, 0.05329814, 0.06638863, 0.11657998, 0.09474952,
       0.04716528, 0.25119904, 0.47698594, 0.35833103, 0.15637362,
       0.27704507, 0.65919458, 0.24238073, 0.14910174, 0.69475357,
       0.50563552, 0.80002238, 0.7051007 , 0.44420608, 0.21037749,
       0.06397617, 0.11677541, 0.22466454, 0.56244532, 0.67774251,
       0.29863889, 0.89408624, 0.17934509, 0.18724151, 0.79841682,
       0.26589102, 0.19238419, 0.21952037, 0.06087548, 0.126379  ,
       0.03503891, 0.04095548, 0.22670706, 0.14717827, 0.72644492,
       0.40010552, 0.13536545, 0.1537661 , 0.20038037, 0.23552361,
       0.08008775, 0.59824239, 0.39967286, 0.07968716, 0.08699301,
       0.35625533, 0.6098194 , 0.60030958, 0.50903341, 0.79489929,
       0.47671656, 0.7150811 , 0.5059392 , 0.11725486, 0.63610276,
       0.33609659, 0.48355634, 0.87606138, 0.92928469, 0.69375865,
       0.97552441, 0.76878145, 0.64389963, 0.85130681, 0.61455325,
       0.56401338, 0.26522224, 0.95388585, 0.63344538, 0.1296849 ,
       0.47407066, 0.8483158 , 0.90031392, 0.8269378 )

y_true_gesund = c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
       0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1)


set.seed(99)
boot = 2000

rocobj <- roc(y_true_gesund, y_pred_gesund)
print(ci.thresholds(rocobj,method = 'bootstrap',boot.n = boot))
print(ci.auc(rocobj, method = 'bootstrap', boot.n=boot))
ci.auc(rocobj, method = 'bootstrap', boot.n=boot)
###
ci.sp.obj <- ci.sp(rocobj, sensitivities=seq(0, 1, .01), boot.n=boot, boot.stratified = TRUE)
plot(rocobj, main = '',
     grid = TRUE,percent=TRUE, xlab = 'Spezifität',ylab = 'Sensitivität')
plot(ci.sp.obj, type="shape", col="lightgrey",density = 50)
ci1 <- ci.thresholds(rocobj,.95, thresholds = 0.3173677, boot.n=boot , boot.stratified = TRUE) 
plot(ci1)

OUT
      95% CI (2000 stratified bootstrap replicates):
     thresholds  sp.low sp.median sp.high  se.low se.median se.high
      0.0730379 0.06667    0.1778  0.2889 1.00000    1.0000  1.0000
      0.1170151 0.13330    0.2444  0.3778 0.82350    0.9118  1.0000
      0.1280320 0.13330    0.2667  0.4000 0.76470    0.8824  0.9706
      0.2582106 0.46670    0.6222  0.7556 0.73530    0.8529  0.9419
      0.3173677 0.55560    0.6889  0.8222 0.70590    0.8235  0.9412
      0.3790019 0.57780    0.7111  0.8444 0.61760    0.7647  0.9118
      0.4591384 0.62220    0.7556  0.8667 0.58820    0.7353  0.8824
      0.4802711 0.66670    0.7778  0.8889 0.50000    0.6765  0.8235
      0.5057874 0.66670    0.8000  0.9111 0.47060    0.6471  0.7941
      0.5632294 0.71110    0.8222  0.9333 0.41180    0.5882  0.7353
      0.6857506 0.75560    0.8667  0.9556 0.20590    0.3529  0.5000
      0.7100909 0.82220    0.9111  0.9778 0.17650    0.3235  0.4706
      0.7476132 0.84440    0.9333  1.0000 0.14710    0.2941  0.4412
      0.8134801 0.93330    0.9778  1.0000 0.11760    0.2353  0.3824
      0.8972001 1.00000    1.0000  1.0000 0.02941    0.1176  0.2353
    95% CI: 0.6418-0.8627 (2000 stratified bootstrap replicates)
    2.5%
    0.637892156862745
    50%
    0.761437908496732
    97.5%
    0.864722222222222

Answer 1

这就是 bootstrap 的随机性发挥作用的地方。如果你只是运行你的 ci.threshold 函数几次你就会看到这个：

for(i in 1:5) {print(ci.thresholds(rocobj, thresholds = 0.3173678)); cat("\n")}

#> 95% CI (2000 stratified bootstrap replicates):
#>  thresholds sp.low sp.median sp.high se.low se.median se.high
#>   0.3173678 0.5556    0.6889  0.8222 0.7051    0.8235  0.9412
#> 
#> 95% CI (2000 stratified bootstrap replicates):
#>  thresholds sp.low sp.median sp.high se.low se.median se.high
#>   0.3173678 0.5556    0.6889  0.8222 0.7059    0.8235  0.9412
#> 
#> 95% CI (2000 stratified bootstrap replicates):
#>  thresholds sp.low sp.median sp.high se.low se.median se.high
#>   0.3173678 0.5556    0.6889  0.8222 0.6765    0.8235  0.9412
#> 
#> 95% CI (2000 stratified bootstrap replicates):
#>  thresholds sp.low sp.median sp.high se.low se.median se.high
#>   0.3173678 0.5556    0.6889  0.8222 0.6765    0.8235  0.9412
#> 
#> 95% CI (2000 stratified bootstrap replicates):
#>  thresholds sp.low sp.median sp.high se.low se.median se.high
#>   0.3173678 0.5556    0.6889  0.8222 0.6765    0.8235  0.9412

您会注意到 se.low 大多数时候是 0.6765，但偶尔会达到 0.7051 甚至 0.7059。当 bootstrapping 相对较小的离散数字时，您可以看到这一点：结果被“量化”，因此它们通常都是相同的值，但是有时不同的值。

pROC：灵敏度下限未正确显示在 ROC 曲线上

pROC: Lower Bound of Sensitivity is not displayed right on ROC-Curve

plot

r

confidence-interval

proc