如何在物种分布建模中获得每个模型和整体模型的 AUC 分数

Question

我想获得评估指标作为每个模型的 AUC 分数和 SDM 的集成模型。分布建模是使用 R 中的 BIOMOD2 包进行的。使用 get_evaluation 我得到 Testing.data、截止值、灵敏度和特异性的值，但我想在 table 中获得 AUC 分数每个模型，这样我就可以有一个箱线图。我知道命令 models_scores_graph 提供了一个图表，但我希望它以 table 形式出现。 models_score_graph之前的代码如下：

DataSpecies <- read.csv(system.file("external/species/mammals_table.csv",
                            package="biomod2"), row.names = 1)



myRespName <- 'GuloGulo'

myResp <- as.numeric(DataSpecies[,myRespName])

myRespXY <- DataSpecies[,c("X_WGS84","Y_WGS84")]

myExpl = raster::stack( system.file( "external/bioclim/current/bio3.grd", 
                             package="biomod2"),
                system.file( "external/bioclim/current/bio4.grd", 
                             package="biomod2"), 
                system.file( "external/bioclim/current/bio7.grd", 
                             package="biomod2"),  
                system.file( "external/bioclim/current/bio11.grd", 
                             package="biomod2"), 
                system.file( "external/bioclim/current/bio12.grd", 
                             package="biomod2"))



 myBiomodData <- BIOMOD_FormatingData(resp.var = myResp,
                             expl.var = myExpl,
                             resp.xy = myRespXY,
                             resp.name = myRespName)

myBiomodOption <- BIOMOD_ModelingOptions()

myBiomodModelOut <- BIOMOD_Modeling( myBiomodData, 
                             models = c('SRE','CTA','RF'), 
                             models.options = myBiomodOption, 
                             NbRunEval=1, 
                             DataSplit=80, 
                             Yweights=NULL, 
                             VarImport=3, 
                             models.eval.meth = c('TSS'),
                             SaveObj = TRUE,
                             rescal.all.models = FALSE,
                             do.full.models = FALSE,
                             modeling.id='test')

myBiomodModelOut_scores <- get_evaluations(myBiomodModelOut)

models_scores_graph(myBiomodModelOut, by = "models" ,
                   metrics = c("ROC","TSS"))

Answer 1

要获得AUC输出，您需要在BIOMOD_Modeling()中将其指定为评估方法。正如你所拥有的，你只使用了 TSS，所以它显示为 Testing.data.

您可以完全删除 models.eval.meth 参数（这将最终显示 Kappa、TSS 和 AUC）或指定您想要的方法（如下添加 ROC）。

myBiomodModelOut <- BIOMOD_Modeling( myBiomodData, 
                                     models = c('SRE','CTA','RF'), 
                                     models.options = myBiomodOption, 
                                     NbRunEval=1, 
                                     DataSplit=80, 
                                     Yweights=NULL, 
                                     VarImport=3, 
                                     models.eval.meth = c('ROC', 'TSS'),
                                     SaveObj = TRUE,
                                     rescal.all.models = FALSE,
                                     do.full.models = FALSE,
                                     modeling.id='test')

更新：如果将 NbRunEval 改为 3，它将对每个算法进行运行 3 次评估运行。评估指标将反映在 get_evaluations() 函数中（见下文）。

myBiomodModelOut <- BIOMOD_Modeling( myBiomodData, 
                                     models = c('SRE','CTA','RF'), 
                                     models.options = myBiomodOption, 
                                     NbRunEval=3, 
                                     DataSplit=80, 
                                     Yweights=NULL, 
                                     VarImport=3,
                                     models.eval.meth = c('ROC', 'TSS'),
                                     SaveObj = TRUE,
                                     rescal.all.models = FALSE,
                                     do.full.models = FALSE,
                                     modeling.id='test')

(myBiomodModelOut_scores <- get_evaluations(myBiomodModelOut))

> , , SRE, RUN1, AllData
> 
>     Testing.data Cutoff Sensitivity Specificity
> ROC        0.876    500      88.636      86.575
> TSS        0.752    495      88.636      86.575
> 
> , , CTA, RUN1, AllData
> 
>     Testing.data Cutoff Sensitivity Specificity
> ROC        0.976    191      94.697      94.521
> TSS        0.892    187      94.697      94.521
> 
> , , RF, RUN1, AllData
> 
>     Testing.data Cutoff Sensitivity Specificity
> ROC        0.989    513      95.455      98.082
> TSS        0.935    515      95.455      98.082
> 
> , , SRE, RUN2, AllData
> 
>     Testing.data Cutoff Sensitivity Specificity
> ROC        0.853    500      85.606      84.932
> TSS        0.705    495      85.606      84.932
> 
> , , CTA, RUN2, AllData
> 
>     Testing.data Cutoff Sensitivity Specificity
> ROC        0.939     94      94.697      92.055
> TSS        0.868     96      94.697      92.055
> 
> , , RF, RUN2, AllData
>
>     Testing.data Cutoff Sensitivity Specificity
> ROC        0.987    351      95.455       95.89
> TSS        0.913    354      95.455       95.89
> 
> , , SRE, RUN3, AllData
> 
>     Testing.data Cutoff Sensitivity Specificity
> ROC        0.872    500      86.364      87.945
> TSS        0.743    495      86.364      87.945
> 
> , , CTA, RUN3, AllData
> 
>     Testing.data Cutoff Sensitivity Specificity
> ROC        0.942    159      90.909      91.233
> TSS        0.821    157      90.909      91.233
> 
> , , RF, RUN3, AllData
> 
>     Testing.data Cutoff Sensitivity Specificity
> ROC        0.986    251      95.455      93.973
> TSS        0.893    202      96.970      92.329

您可以直接从 myBiomodModelOut_scores 数组访问这些数据，也可以使用您已有的函数进行绘图（见下文）。

models_scores_graph(myBiomodModelOut, by = "models" ,
                    metrics = c("ROC","TSS"))

如果您想以数据框格式使用这些数据，您可以提取评估分数：

# note that the values in the output here are different from the above 
# array since I ran it in another R session and didn't set a seed...

(eval.df <- t(data.frame(RUN_1 = myBiomodModelOut_scores[ , 1, , 1, ],
                         RUN_2 = myBiomodModelOut_scores[ , 1, , 2, ],
                         RUN_3 = myBiomodModelOut_scores[ , 1, , 3, ])))
>             ROC   TSS
> RUN_1.SRE 0.853 0.706
> RUN_1.CTA 0.917 0.830
> RUN_1.RF  0.982 0.870
> RUN_2.SRE 0.873 0.747
> RUN_2.CTA 0.960 0.847
> RUN_2.RF  0.982 0.902
> RUN_3.SRE 0.873 0.745
> RUN_3.CTA 0.965 0.872
> RUN_3.RF  0.988 0.918

如何在物种分布建模中获得每个模型和整体模型的 AUC 分数

How to get the AUC score per model and for ensemble model in Species Distribution Modeling

r

auc