`caret' 和归一化互信息 (NMI) 中的混淆矩阵：线性判别分析、朴素贝叶斯和分类树
Confusion matrix in `caret' and normalised mutual information (NMI): Linear discriminant analysis, Naive Bayes and Classification Trees
提前为问题的长度道歉，但此页面的大部分内容包含分步编码以说明我在尝试解决错误消息时的思考过程。我已将数据分成训练 (70%) 和测试 (30%) 集，用于三种监督机器学习算法，称为线性判别分析 (LDA)、朴素贝叶斯 (NB) 和分类树 (CT)，使用 "caret" R 中的包（数据和代码的可重现示例如下）。因此，每个算法都使用 100 次运行的重复 10 折交叉验证进行训练。这是一个探索性分类练习，响应变量有 2 类，即家庭（即 "G8" 和 "V4"）、12 个预测变量和 80 个观察值。如果有人对如何解决这些问题有任何见解并有助于理解错误消息，那么非常感谢。这将不胜感激，因为我是 R 的新手。
目标
我的目标是使用 "caret" 包中的函数 "confusionMatrix" 生成混淆矩阵。我的objective就是用这个函数得到Kappa系数，分类准确率，灵敏度，特异度，单向假设检验和相关的统计量。从混淆矩阵中，我打算计算归一化互信息 (NMI) 系数来评估模型性能。我的数据称为“mydat”（可以在本页底部找到）。
问题
(1) 对于混淆矩阵，输出错误信息为：
Error in sort.list(y) : 'x' must be atomic for 'sort.list'
Have you called 'sort' on a list?
我尝试了不同的代码组合来解决这个问题，但无法理解问题所在
(2) 我的 NMI 分数偏离很大，应该在 0 到 1 之间。我想知道我的编码中是否有任何注意事项，或者反过来，是否有更简单直接的方法来计算 NMI来自混淆矩阵（等式如下）？
想法是使用 `caret' 包中的函数 "confusionMatrix" 为 LDA、NB 和 CT 输出三个独立混淆矩阵的混淆矩阵统计信息（如下示例）：
#The example below was randomly extracted
from `A short introduction to the "caret"
package' by Max Kuhn (2015) to highlight
what I am attempting to achieve with my data.
The only text changed is the grouping factors,
(i.e G8 and v4).
Confusion Matrix and Statistics
Reference
Prediction G8 V4
G8 20 7
V4 7 17
Kappa : 0.4491
Mcnemar's Test P-Value : 1.000000
Sensitivity : 0.7407
Specificity : 0.7083
Pos Pred Value : 0.7407
eg Pred Value : 0.7083
Prevalence : 0.5294
etection Rate : 0.3922
Detection Prevalence : 0.5294
Balanced Accuracy : 0.7245
'Positive' Class : M
算法和混淆矩阵的代码
library(pROC)
library(MASS)
library(caret)
library(e1071)
# Randomly permute the data before subsetting
mydat$Family <- factor(mydat$Family, levels = c("G8", "v4"))
mydat_idx <- sample(1:nrow(mydat), replace = FALSE)
mydat1 <- mydat[mydat_idx, ]
#Produce 70 % training and 30 % test set
mydat_resampled_idx <- createDataPartition(mydat_idx, times = 1, p = 0.7, list = FALSE)
mydat_resampled_train <- mydat1[mydat_resampled_idx, ] # Training portion of the data
mydat_resampled_test <- mydat1[-mydat_resampled_idx, ] # Test portion
set.seed(1234)
重复 10 倍线性判别分析
lda_mod <-train(x = mydat_resampled_train[, 2:13],
y = as.factor(mydat_resampled_train[, 1]),
method = "lda",
trControl = trainControl(method = "repeatedcv", number=10, repeats=100, classProbs = TRUE))
# Generate model predictions
lda_pred<- predict(lda_mod, newdata = mydat_resampled_train[ , 2:13], type = "prob")
# Store the predictions with the data set
mydat_resampled_train['lda_pred'] <- lda_pred["G8"] # Here we only want the probability associated
# with the class (Y = 1), or in this case, `G8'
重复 10 次朴素贝叶斯
nb_tune <- data.frame(usekernel =TRUE, fL = 0)
nb_mod <- train(x = mydat_resampled_train[, 2:13],
y = as.factor(mydat_resampled_train[, 1]),
method = "nb",
trControl = trainControl(method = "repeatedcv", number=10, repeats=100, classProbs = TRUE),
tuneGrid = nb_tune)
# Model predictions
nb_pred <- predict(nb_mod, newdata = mydat_resampled_train[ , 2:13], type = "prob")
mydat_resampled_train['nb_pred'] <- nb_pred["G8"]
重复 10 次分类树
ct_mod <- train(x = mydat_resampled_train[, 2:13],
y = as.factor(mydat_resampled_train[, 1]),
method = "rpart",
trControl = trainControl(method = "repeatedcv", number=10, repeats=100, classProbs = TRUE))
ct_pred <- predict(ct_mod, newdata = mydat_resampled_train[ , 2:13], type = "prob")
mydat_resampled_train['ct_pred'] <- ct_pred["G8"]
混淆矩阵代码和伴随的错误消息
#LDA
confusionMatrix(lda_pred, mydat_resampled_test$Family)
#NB
confusionMatrix(nb_pred, mydat_resampled_test$Family)
#CT
confusionMatrix(ct_pred, mydat_resampled_test$Family)
每个模型returns相同的错误信息，在网上和教程中搜索后，我仍然无法解决问题。
Error in sort.list(y) : 'x' must be atomic for 'sort.list'
Have you called 'sort' on a list?
归一化互信息 (NMI)
• a is the number of correct predictions that an instance is negative,
• b is the number of incorrect predictions that an instance is positive,
• c is the number of incorrect of predictions that an instance is negative,
• d is the number of correct predictions that an instance is positive,
• N is the count total
参考：菲尔丁，A.H。 & 贝尔，J.F。 (1997) 方法回顾
保护存在的预测误差评估/
缺席模型。环境保护, 24, 38–49.
#Equation to calculate the NMI after Fielding and Bell (1997)
NMI= -a*In(a)-b*In(b)-c*In(c)-d*In(d)+(a+b)*In(a+b)+(c+d)*In(c+d)
-----------------------------------------------------------
n*In(n)-((a+c)*In(a+c)+(b+d)*In(b+d))
#This is how I approached this problem
a-d的值取自上面
Kuhn (2015) 的混淆矩阵示例。
a<-20
b<-7
c<-7
d<-17
N<-80
NMI 编码
#I believe the function `solve' is used to inverse a value. Is this correct?
NMI_top.row<-((-a)*(solve(a))-((b*solve(b))-(c*solve(c))-(d*solve(d)))+((a+b)*(solve(a+b)+(c+d)*(solve(c+d)))))
NMI_bottom.row<-(N*solve(N))-(((a+c))*(solve(a+c)+(b+d))*solve((b+d)))
NMI<-(NMI_top.row/NMI_bottom.row)
Answer: This answer is way off since it should lie between 0 to 1
[,1]
[1,] -1.0752
#
数据
mydat <- structure(list(Family = structure(c(2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = c("G8", "v4"), class="factor"),
Swimming = c(-0.4805568, 0.12600625, 0.06823834, 0.67480139,
0.64591744, 0.21265812, -0.01841352, 0.12600625, -0.2206012,
0.27042603, 0.03935439, -0.45167284, -0.04729748, -0.10506539,
0.0971223, -0.07618143, 0.29930998, 0.01047043, -0.24948516,
-0.04729748, -0.01841352, -0.19171725, -0.4805568, 0.01047043,
-0.42278889, -0.45167284, -0.30725307, 0.24154207, 1.45466817,
-0.01841352, 0.38596185, 0.15489021, -0.04729748, 0.27042603,
-0.07618143, -0.10506539, -0.01841352, 0.01047043, 0.06823834,
-0.16283329, -0.01841352, -0.39390493, -0.04729748, 0.01047043,
0.01047043, 0.06823834, -0.04729748, -0.2206012, -0.16283329,
-0.07618143, -0.2206012, -0.19171725, -0.16283329, -0.2206012,
-0.13394934, -0.27836911, -0.04729748, 0.01047043, 0.12600625,
0.06823834, 0.06823834, 0.32819394, 0.32819394, -0.27836911,
0.18377416, 0.55926557, -0.19171725, -0.19171725, 0.01047043,
-0.19171725, -0.01841352, -0.07618143, -0.13394934, -0.39390493,
-0.04729748, -0.27836911, 0.70368535, 0.29930998, -0.13394934,
0.21265812), Not.Swimming = c(-0.0862927, -0.074481895, -0.056765686,
-0.050860283, -0.050860283, -0.068576492, -0.068576492, 0.05543697,
0.114491, -0.021333268, -0.04495488, 0.008193747, -0.056765686,
0.008193747, 0.037720761, 0.01409915, 0.108585597, -0.074481895,
0.002288344, 0.049531567, 0.043626164, 0.049531567, 0.020004552,
0.008193747, 0.025909955, 0.031815358, 0.049531567, -0.039049477,
-0.003617059, 0.002288344, 0.084963985, -0.080387298, 0.067247776,
0.031815358, 0.037720761, 0.025909955, 0.126301805, 0.031815358,
0.037720761, -0.050860283, -0.039049477, -0.003617059, 0.008193747,
-0.039049477, -0.003617059, 0.008193747, 0.01409915, -0.015427865,
0.020004552, 0.031815358, 0.020004552, -0.033144074, -0.039049477,
-0.009522462, -0.003617059, -0.04495488, -0.050860283, -0.04495488,
-0.068576492, -0.033144074, -0.027238671, -0.068576492, 0.01409915,
0.002288344, 0.025909955, -0.009522462, -0.009522462, 0.025909955,
0.15582882, 0.002288344, -0.04495488, -0.015427865, 0.008193747,
0.037720761, 0.008193747, -0.015427865, -0.056765686, 0.079058582,
-0.056765686, 0.025909955), Running = c(-0.157157188, 0.057316151,
0.064711783, 0.153459372, 0.072107416, 0.057316151, -0.053618335,
0.012942357, -0.03882707, 0.049920519, 0.012942357, -0.075805232,
0.035129254, -0.046222702, 0.109085578, -0.03882707, 0.057316151,
0.020337989, 0.035129254, 0.057316151, 0.005546724, -0.016640173,
-0.142365923, 0.220020063, -0.149761556, -0.134970291, 0.042524886,
0.072107416, 0.064711783, 0.020337989, 0.049920519, 0.020337989,
0.138668107, 0.049920519, 0.020337989, -0.083200864, -0.024035805,
-0.016640173, -0.03882707, -0.03882707, 0.005546724, -0.090596497,
-0.00924454, -0.016640173, -0.075805232, -0.090596497, 0.012942357,
-0.075805232, -0.061013967, -0.03882707, -0.112783394, -0.068409599,
-0.090596497, -0.053618335, -0.075805232, -0.090596497, 0.064711783,
0.012942357, 0.042524886, -0.061013967, -0.061013967, 0.064711783,
0.175646269, -0.068409599, 0.027733621, 0.042524886, -0.03882707,
-0.00924454, 0.027733621, -0.031431438, -0.046222702, -0.031431438,
-0.068409599, -0.120179026, 0.035129254, -0.061013967, 0.39751524,
0.138668107, 0.020337989, 0.035129254), Not.Running = c(-0.438809944,
-0.539013927, -0.539013927, -0.539013927, -0.472211271, -0.071395338,
-0.071395338, 0.296019267, 0.563229889, -0.03799401, 0.195815284,
-0.171599321, -0.305204632, 0.062209973, -0.104796666, 0.095611301,
0.028808645, -0.071395338, 0.329420595, 0.296019267, -0.171599321,
-0.071395338, 0.596631217, 0.062209973, 0.028808645, -0.138197994,
0.095611301, -0.104796666, 0.296019267, 0.028808645, -0.03799401,
-0.33860596, 0.129012629, 0.195815284, -0.03799401, 0.396223251,
0.362821923, -0.138197994, 0.26261794, -0.405408616, -0.205000649,
0.129012629, 0.195815284, -0.205000649, -0.004592683, -0.205000649,
-0.071395338, -0.171599321, -0.104796666, -0.138197994, -0.104796666,
-0.071395338, -0.104796666, -0.03799401, -0.004592683, -0.238401977,
0.028808645, -0.305204632, -0.305204632, -0.271803305, -0.03799401,
-0.372007288, 0.095611301, 0.195815284, 0.162413956, 0.229216612,
0.229216612, 0.396223251, 0.630032545, 0.463025906, 0.496427234,
0.062209973, -0.071395338, 0.229216612, -0.071395338, -0.071395338,
-0.205000649, 0.229216612, -0.305204632, 0.396223251), Fighting = c(-0.67708172,
-0.58224128, -0.11436177, -0.34830152, -0.84568695, -0.32933343,
0.35984044, -0.3251183, 1.51478626, 0.11114773, 0.27975296,
-0.89626852, 0.12379312, 0.66965255, 1.56536783, 0.56427428,
-0.71291033, -0.75927677, -0.75295407, -1.00164679, -1.03958296,
0.82139726, -1.07541157, -1.0311527, -0.98900139, -1.06908888,
-1.20186549, 0.58324237, -0.9700333, 0.22917139, 0.41042201,
-1.11545531, -0.19023412, 0.25446217, -0.05324237, 0.09007207,
1.21129685, 0.62539368, 1.32932051, 0.40199175, 0.44625062,
0.60221046, 0.33665722, -0.63493041, -0.282967, -0.32722587,
-0.11646933, -0.10171637, 0.13643851, -0.57802615, 0.05002833,
-0.1607282, -0.29139726, 0.13222338, -0.41152848, 0.68229794,
-0.24292325, -0.11646933, -0.21341734, -0.24292325, -0.24292325,
0.09007207, -0.34197883, -0.30825778, -0.08696342, -0.8119659,
0.49683219, -0.13754498, -0.4831857, 0.39988418, 0.90148474,
0.28396809, 1.05322945, 1.24923303, 0.47154141, 1.27873894,
0.05002833, 1.54218461, 0.74763247, 0.11747042), Not.Fighting = c(-0.097624192,
-0.160103675, -0.092996082, -0.234153433, -0.136963126, -0.15778962,
-0.15778962, -0.023574435, 0.00188017, -0.224897213, -0.109194467,
-0.069855533, -0.123078796, -0.111508522, -0.143905291, -0.099938247,
-0.118450687, 1.519900201, 0.177748344, 0.108326696, 0.652129604,
0.638245274, -0.072169588, 0.087500202, -0.18093017, -0.146219346,
-0.049029039, -0.125392851, -0.134649071, -0.060599313, -0.086053918,
-0.197128554, -0.083739863, -0.092996082, 0.844196163, 0.055103433,
1.971140911, -0.111508522, -0.224897213, -0.187872334, -0.160103675,
-0.194814499, -0.053657149, -0.206384774, 0.108326696, -0.164731785,
0.187004564, 0.025020719, 0.057417488, 0.434608441, 0.057417488,
0.073615872, -0.035144709, -0.051343094, -0.134649071, -0.185558279,
0.013450444, -0.134649071, -0.215640993, -0.185558279, -0.005061995,
-0.238781543, -0.099938247, -0.16704584, -0.208698829, 0.048161268,
0.048161268, -0.037458764, 0.16154996, 0.031962884, -0.102252302,
-0.123078796, -0.139277181, -0.208698829, -0.118450687, -0.072169588,
-0.044400929, -0.030516599, -0.132335016, -0.037458764),
Resting = c(0.01081204879, -0.03398160805, 0.057108797, -0.04063432116,
-0.13084281035, -0.02997847693, 0.12732080268, -0.1028170581,
0.08155320398, -0.17932134171, -0.14338902206, -0.02058415581,
-0.11528274705, -0.11764091337, 0.04389156236, 0.01399844913,
-0.05755560242, 0.04711630687, 0.0158428036, 0.093485909,
0.09677967302, 0.02053612974, -0.03608286844, 0.07805238146,
-9.686695e-05, -0.02285413055, -0.00424187149, 0.01446241356,
0.03187450017, 0.11323315542, -0.01171898422, -0.06499053655,
-0.07758659568, -0.07399758157, -0.11503350996, 0.02167111711,
0.01904454162, 0.05768779393, 0.05555202379, -0.01031175326,
-0.00458313459, 0.17430774591, 0.00481502094, -0.00928412956,
0.09047589183, 0.08917985896, -0.05671203072, -0.05333390954,
0.08541446168, 0.10140397965, -0.02509342995, -0.0369877908,
0.04609635201, 0.06524159499, 0.0845977309, -0.03239032508,
-0.03208740616, 0.06264952925, 0.05241547086, -0.03437271856,
-0.03437271856, -0.06747523863, -0.01270059491, 0.10014629095,
-0.02872845706, -0.00950652573, 0.04867308008, 0.02486518629,
-0.05951115497, -0.02353665674, -0.01967923345, -0.10148651548,
-0.00480936518, -0.00098261723, -0.13970798195, -0.00286148145,
-0.05492902692, 0.10732815358, 0.11660744219, -0.02016620439
), Not.Resting = c(-0.77046287, 0.773856776, -2.593072768,
-2.837675606, -1.680828329, -0.947623773, -0.947623773, -2.607366431,
-0.637055341, -1.818396455, 2.170944974, -0.658126752, -0.808243774,
2.377766908, 2.111220276, -0.322326312, 2.218858946, 3.920878638,
-0.304945754, 1.038591535, 1.752268128, 0.907465624, 1.137774798,
-3.663486997, 2.350924346, 0.067293462, -1.898454393, -2.497647463,
-4.471716512, -1.465081244, -0.232806371, -3.043893581, -2.323908986,
1.437404886, 1.079056696, 1.110865131, 1.404724068, -1.706664294,
0.736746935, -0.005516985, 1.727170333, 1.685228831, 1.836016918,
0.46617392, 1.697173771, 1.057314221, 0.933704227, 0.482480775,
0.680713089, 0.090780703, 0.680713089, -0.982921741, -2.281900378,
0.97208909, 0.027767791, -0.1628815, -0.530221948, -0.385741863,
-0.972251823, 0.002267358, -1.134447998, 0.626424009, -0.722750217,
-0.382722075, -0.356550578, -1.851614124, -1.851614124, 1.731465143,
0.254319006, 2.043778341, -0.28991392, 1.386940871, 0.054207713,
0.594212936, 1.551821303, 3.100704184, 0.327263666, -1.055195336,
-1.134447998, 1.730726972), Hunting = c(-0.67708172, -0.58224128,
-0.11436177, -0.34830152, -0.84568695, -0.32933343, 0.35984044,
-0.3251183, 1.51478626, 0.11114773, 0.27975296, -0.89626852,
0.12379312, 0.66965255, 1.56536783, 0.56427428, -0.71291033,
-0.75927677, -0.75295407, -1.00164679, -1.03958296, 0.82139726,
-1.07541157, -1.0311527, -0.98900139, -1.06908888, -1.20186549,
0.58324237, -0.9700333, 0.22917139, 0.41042201, -1.11545531,
-0.19023412, 0.25446217, -0.05324237, 0.09007207, 1.21129685,
0.62539368, 1.32932051, 0.40199175, 0.44625062, 0.60221046,
0.33665722, -0.63493041, -0.282967, -0.32722587, -0.11646933,
-0.10171637, 0.13643851, -0.57802615, 0.05002833, -0.1607282,
-0.29139726, 0.13222338, -0.41152848, 0.68229794, -0.24292325,
-0.11646933, -0.21341734, -0.24292325, -0.24292325, 0.09007207,
-0.34197883, -0.30825778, -0.08696342, -0.8119659, 0.49683219,
-0.13754498, -0.4831857, 0.39988418, 0.90148474, 0.28396809,
1.05322945, 1.24923303, 0.47154141, 1.27873894, 0.05002833,
1.54218461, 0.74763247, 0.11747042), Not.Hunting = c(-0.097624192,
-0.160103675, -0.092996082, -0.234153433, -0.136963126, -0.15778962,
-0.15778962, -0.023574435, 0.00188017, -0.224897213, -0.109194467,
-0.069855533, -0.123078796, -0.111508522, -0.143905291, -0.099938247,
-0.118450687, 1.519900201, 0.177748344, 0.108326696, 0.652129604,
0.638245274, -0.072169588, 0.087500202, -0.18093017, -0.146219346,
-0.049029039, -0.125392851, -0.134649071, -0.060599313, -0.086053918,
-0.197128554, -0.083739863, -0.092996082, 0.844196163, 0.055103433,
1.971140911, -0.111508522, -0.224897213, -0.187872334, -0.160103675,
-0.194814499, -0.053657149, -0.206384774, 0.108326696, -0.164731785,
0.187004564, 0.025020719, 0.057417488, 0.434608441, 0.057417488,
0.073615872, -0.035144709, -0.051343094, -0.134649071, -0.185558279,
0.013450444, -0.134649071, -0.215640993, -0.185558279, -0.005061995,
-0.238781543, -0.099938247, -0.16704584, -0.208698829, 0.048161268,
0.048161268, -0.037458764, 0.16154996, 0.031962884, -0.102252302,
-0.123078796, -0.139277181, -0.208698829, -0.118450687, -0.072169588,
-0.044400929, -0.030516599, -0.132335016, -0.037458764),
Grooming = c(0.01081204879, -0.03398160805, 0.057108797,
-0.04063432116, -0.13084281035, -0.02997847693, 0.12732080268,
-0.1028170581, 0.08155320398, -0.17932134171, -0.14338902206,
-0.02058415581, -0.11528274705, -0.11764091337, 0.04389156236,
0.01399844913, -0.05755560242, 0.04711630687, 0.0158428036,
0.093485909, 0.09677967302, 0.02053612974, -0.03608286844,
0.07805238146, -9.686695e-05, -0.02285413055, -0.00424187149,
0.01446241356, 0.03187450017, 0.11323315542, -0.01171898422,
-0.06499053655, -0.07758659568, -0.07399758157, -0.11503350996,
0.02167111711, 0.01904454162, 0.05768779393, 0.05555202379,
-0.01031175326, -0.00458313459, 0.17430774591, 0.00481502094,
-0.00928412956, 0.09047589183, 0.08917985896, -0.05671203072,
-0.05333390954, 0.08541446168, 0.10140397965, -0.02509342995,
-0.0369877908, 0.04609635201, 0.06524159499, 0.0845977309,
-0.03239032508, -0.03208740616, 0.06264952925, 0.05241547086,
-0.03437271856, -0.03437271856, -0.06747523863, -0.01270059491,
0.10014629095, -0.02872845706, -0.00950652573, 0.04867308008,
0.02486518629, -0.05951115497, -0.02353665674, -0.01967923345,
-0.10148651548, -0.00480936518, -0.00098261723, -0.13970798195,
-0.00286148145, -0.05492902692, 0.10732815358, 0.11660744219,
-0.02016620439), Not.Grooming = c(-0.77046287, 0.773856776,
-2.593072768, -2.837675606, -1.680828329, -0.947623773, -0.947623773,
-2.607366431, -0.637055341, -1.818396455, 2.170944974, -0.658126752,
-0.808243774, 2.377766908, 2.111220276, -0.322326312, 2.218858946,
3.920878638, -0.304945754, 1.038591535, 1.752268128, 0.907465624,
1.137774798, -3.663486997, 2.350924346, 0.067293462, -1.898454393,
-2.497647463, -4.471716512, -1.465081244, -0.232806371, -3.043893581,
-2.323908986, 1.437404886, 1.079056696, 1.110865131, 1.404724068,
-1.706664294, 0.736746935, -0.005516985, 1.727170333, 1.685228831,
1.836016918, 0.46617392, 1.697173771, 1.057314221, 0.933704227,
0.482480775, 0.680713089, 0.090780703, 0.680713089, -0.982921741,
-2.281900378, 0.97208909, 0.027767791, -0.1628815, -0.530221948,
-0.385741863, -0.972251823, 0.002267358, -1.134447998, 0.626424009,
-0.722750217, -0.382722075, -0.356550578, -1.851614124, -1.851614124,
1.731465143, 0.254319006, 2.043778341, -0.28991392, 1.386940871,
0.054207713, 0.594212936, 1.551821303, 3.100704184, 0.327263666,
-1.055195336, -1.134447998, 1.730726972), Other = c(0.019502286,
-0.290451956, 0.359948884, 0.557840914, 0.117453376, 0.126645924,
0.126645924, 0.196486873, 0.152780228, 0.354469789, -0.261430968,
0.176448238, -0.007374708, -0.557848621, -0.213674557, -0.005819262,
-0.470070992, -0.786078864, 0.006063789, -0.27184265, -0.349418792,
-0.338096262, -0.165119403, 0.346566439, -0.344191931, 0.074321265,
0.179825379, 0.278407054, 0.593125727, 0.199177375, -0.058900625,
0.633875622, 0.428150308, -0.206023441, -0.436958199, -0.291839246,
-0.907641911, 0.448567295, -0.127186127, 0.024715134, -0.41634503,
-0.330697382, -0.469720666, -0.047494017, -0.301732446, -0.138901021,
0.098101379, -0.002063769, -0.02832419, 0.071630763, -0.02832419,
0.295110588, 0.347112947, -0.083577573, -0.036886152, 0.189045953,
0.467596992, 0.303378276, 0.218879697, 0.092005711, 0.27011134,
-0.012909856, 0.262292068, 0.107125772, 0.123422927, 0.299426602,
0.299426602, -0.326871824, -0.022088391, -0.428508341, -0.014675497,
-0.114462294, 0.087227267, -0.031519161, -0.159318008, -0.397875854,
0.101520559, 0.244481505, 0.529968994, -0.32661959)), .Names = c("Family",
"Swimming", "Not.Swimming", "Running", "Not.Running", "Fighting",
"Not.Fighting", "Resting", "Not.Resting", "Hunting", "Not.Hunting",
"Grooming", "Not.Grooming", "Other"), class = "data.frame", row.names=c(NA, -80L))
对于混淆矩阵问题，您为其提供 class 概率（例如 lda_pred）而不是因子预测。有关对象应该是什么的详细信息，请参阅 ?confusionMatrix。
对于 NMI，confusionMatrix 会给你一个 table 来使用。例如：
> library(caret)
>
> set.seed(1)
> dat <- twoClassSim(200)
> dat2 <- twoClassSim(200)
>
> set.seed(2)
> mod <- train(Class ~ ., data = dat, method = "lda")
>
> preds <- predict(mod, dat2)
>
> cm <- confusionMatrix(preds, dat2$Class)
> cm$table
Reference
Prediction Class1 Class2
Class1 78 30
Class2 17 75
提前为问题的长度道歉，但此页面的大部分内容包含分步编码以说明我在尝试解决错误消息时的思考过程。我已将数据分成训练 (70%) 和测试 (30%) 集，用于三种监督机器学习算法，称为线性判别分析 (LDA)、朴素贝叶斯 (NB) 和分类树 (CT)，使用 "caret" R 中的包（数据和代码的可重现示例如下）。因此，每个算法都使用 100 次运行的重复 10 折交叉验证进行训练。这是一个探索性分类练习，响应变量有 2 类，即家庭（即 "G8" 和 "V4"）、12 个预测变量和 80 个观察值。如果有人对如何解决这些问题有任何见解并有助于理解错误消息，那么非常感谢。这将不胜感激，因为我是 R 的新手。
目标
我的目标是使用 "caret" 包中的函数 "confusionMatrix" 生成混淆矩阵。我的objective就是用这个函数得到Kappa系数，分类准确率，灵敏度，特异度，单向假设检验和相关的统计量。从混淆矩阵中，我打算计算归一化互信息 (NMI) 系数来评估模型性能。我的数据称为“mydat”（可以在本页底部找到）。
问题
(1) 对于混淆矩阵，输出错误信息为：
Error in sort.list(y) : 'x' must be atomic for 'sort.list'
Have you called 'sort' on a list?
我尝试了不同的代码组合来解决这个问题，但无法理解问题所在
(2) 我的 NMI 分数偏离很大，应该在 0 到 1 之间。我想知道我的编码中是否有任何注意事项，或者反过来，是否有更简单直接的方法来计算 NMI来自混淆矩阵（等式如下）？
想法是使用 `caret' 包中的函数 "confusionMatrix" 为 LDA、NB 和 CT 输出三个独立混淆矩阵的混淆矩阵统计信息（如下示例）：
#The example below was randomly extracted
from `A short introduction to the "caret"
package' by Max Kuhn (2015) to highlight
what I am attempting to achieve with my data.
The only text changed is the grouping factors,
(i.e G8 and v4).
Confusion Matrix and Statistics
Reference
Prediction G8 V4
G8 20 7
V4 7 17
Kappa : 0.4491
Mcnemar's Test P-Value : 1.000000
Sensitivity : 0.7407
Specificity : 0.7083
Pos Pred Value : 0.7407
eg Pred Value : 0.7083
Prevalence : 0.5294
etection Rate : 0.3922
Detection Prevalence : 0.5294
Balanced Accuracy : 0.7245
'Positive' Class : M
算法和混淆矩阵的代码
library(pROC)
library(MASS)
library(caret)
library(e1071)
# Randomly permute the data before subsetting
mydat$Family <- factor(mydat$Family, levels = c("G8", "v4"))
mydat_idx <- sample(1:nrow(mydat), replace = FALSE)
mydat1 <- mydat[mydat_idx, ]
#Produce 70 % training and 30 % test set
mydat_resampled_idx <- createDataPartition(mydat_idx, times = 1, p = 0.7, list = FALSE)
mydat_resampled_train <- mydat1[mydat_resampled_idx, ] # Training portion of the data
mydat_resampled_test <- mydat1[-mydat_resampled_idx, ] # Test portion
set.seed(1234)
重复 10 倍线性判别分析
lda_mod <-train(x = mydat_resampled_train[, 2:13],
y = as.factor(mydat_resampled_train[, 1]),
method = "lda",
trControl = trainControl(method = "repeatedcv", number=10, repeats=100, classProbs = TRUE))
# Generate model predictions
lda_pred<- predict(lda_mod, newdata = mydat_resampled_train[ , 2:13], type = "prob")
# Store the predictions with the data set
mydat_resampled_train['lda_pred'] <- lda_pred["G8"] # Here we only want the probability associated
# with the class (Y = 1), or in this case, `G8'
重复 10 次朴素贝叶斯
nb_tune <- data.frame(usekernel =TRUE, fL = 0)
nb_mod <- train(x = mydat_resampled_train[, 2:13],
y = as.factor(mydat_resampled_train[, 1]),
method = "nb",
trControl = trainControl(method = "repeatedcv", number=10, repeats=100, classProbs = TRUE),
tuneGrid = nb_tune)
# Model predictions
nb_pred <- predict(nb_mod, newdata = mydat_resampled_train[ , 2:13], type = "prob")
mydat_resampled_train['nb_pred'] <- nb_pred["G8"]
重复 10 次分类树
ct_mod <- train(x = mydat_resampled_train[, 2:13],
y = as.factor(mydat_resampled_train[, 1]),
method = "rpart",
trControl = trainControl(method = "repeatedcv", number=10, repeats=100, classProbs = TRUE))
ct_pred <- predict(ct_mod, newdata = mydat_resampled_train[ , 2:13], type = "prob")
mydat_resampled_train['ct_pred'] <- ct_pred["G8"]
混淆矩阵代码和伴随的错误消息
#LDA
confusionMatrix(lda_pred, mydat_resampled_test$Family)
#NB
confusionMatrix(nb_pred, mydat_resampled_test$Family)
#CT
confusionMatrix(ct_pred, mydat_resampled_test$Family)
每个模型returns相同的错误信息，在网上和教程中搜索后，我仍然无法解决问题。
Error in sort.list(y) : 'x' must be atomic for 'sort.list'
Have you called 'sort' on a list?
归一化互信息 (NMI)
• a is the number of correct predictions that an instance is negative,
• b is the number of incorrect predictions that an instance is positive,
• c is the number of incorrect of predictions that an instance is negative,
• d is the number of correct predictions that an instance is positive,
• N is the count total
参考：菲尔丁，A.H。 & 贝尔，J.F。 (1997) 方法回顾 保护存在的预测误差评估/ 缺席模型。环境保护, 24, 38–49.
#Equation to calculate the NMI after Fielding and Bell (1997)
NMI= -a*In(a)-b*In(b)-c*In(c)-d*In(d)+(a+b)*In(a+b)+(c+d)*In(c+d)
-----------------------------------------------------------
n*In(n)-((a+c)*In(a+c)+(b+d)*In(b+d))
#This is how I approached this problem
a-d的值取自上面 Kuhn (2015) 的混淆矩阵示例。
a<-20
b<-7
c<-7
d<-17
N<-80
NMI 编码
#I believe the function `solve' is used to inverse a value. Is this correct?
NMI_top.row<-((-a)*(solve(a))-((b*solve(b))-(c*solve(c))-(d*solve(d)))+((a+b)*(solve(a+b)+(c+d)*(solve(c+d)))))
NMI_bottom.row<-(N*solve(N))-(((a+c))*(solve(a+c)+(b+d))*solve((b+d)))
NMI<-(NMI_top.row/NMI_bottom.row)
Answer: This answer is way off since it should lie between 0 to 1
[,1]
[1,] -1.0752
#
数据
mydat <- structure(list(Family = structure(c(2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = c("G8", "v4"), class="factor"),
Swimming = c(-0.4805568, 0.12600625, 0.06823834, 0.67480139,
0.64591744, 0.21265812, -0.01841352, 0.12600625, -0.2206012,
0.27042603, 0.03935439, -0.45167284, -0.04729748, -0.10506539,
0.0971223, -0.07618143, 0.29930998, 0.01047043, -0.24948516,
-0.04729748, -0.01841352, -0.19171725, -0.4805568, 0.01047043,
-0.42278889, -0.45167284, -0.30725307, 0.24154207, 1.45466817,
-0.01841352, 0.38596185, 0.15489021, -0.04729748, 0.27042603,
-0.07618143, -0.10506539, -0.01841352, 0.01047043, 0.06823834,
-0.16283329, -0.01841352, -0.39390493, -0.04729748, 0.01047043,
0.01047043, 0.06823834, -0.04729748, -0.2206012, -0.16283329,
-0.07618143, -0.2206012, -0.19171725, -0.16283329, -0.2206012,
-0.13394934, -0.27836911, -0.04729748, 0.01047043, 0.12600625,
0.06823834, 0.06823834, 0.32819394, 0.32819394, -0.27836911,
0.18377416, 0.55926557, -0.19171725, -0.19171725, 0.01047043,
-0.19171725, -0.01841352, -0.07618143, -0.13394934, -0.39390493,
-0.04729748, -0.27836911, 0.70368535, 0.29930998, -0.13394934,
0.21265812), Not.Swimming = c(-0.0862927, -0.074481895, -0.056765686,
-0.050860283, -0.050860283, -0.068576492, -0.068576492, 0.05543697,
0.114491, -0.021333268, -0.04495488, 0.008193747, -0.056765686,
0.008193747, 0.037720761, 0.01409915, 0.108585597, -0.074481895,
0.002288344, 0.049531567, 0.043626164, 0.049531567, 0.020004552,
0.008193747, 0.025909955, 0.031815358, 0.049531567, -0.039049477,
-0.003617059, 0.002288344, 0.084963985, -0.080387298, 0.067247776,
0.031815358, 0.037720761, 0.025909955, 0.126301805, 0.031815358,
0.037720761, -0.050860283, -0.039049477, -0.003617059, 0.008193747,
-0.039049477, -0.003617059, 0.008193747, 0.01409915, -0.015427865,
0.020004552, 0.031815358, 0.020004552, -0.033144074, -0.039049477,
-0.009522462, -0.003617059, -0.04495488, -0.050860283, -0.04495488,
-0.068576492, -0.033144074, -0.027238671, -0.068576492, 0.01409915,
0.002288344, 0.025909955, -0.009522462, -0.009522462, 0.025909955,
0.15582882, 0.002288344, -0.04495488, -0.015427865, 0.008193747,
0.037720761, 0.008193747, -0.015427865, -0.056765686, 0.079058582,
-0.056765686, 0.025909955), Running = c(-0.157157188, 0.057316151,
0.064711783, 0.153459372, 0.072107416, 0.057316151, -0.053618335,
0.012942357, -0.03882707, 0.049920519, 0.012942357, -0.075805232,
0.035129254, -0.046222702, 0.109085578, -0.03882707, 0.057316151,
0.020337989, 0.035129254, 0.057316151, 0.005546724, -0.016640173,
-0.142365923, 0.220020063, -0.149761556, -0.134970291, 0.042524886,
0.072107416, 0.064711783, 0.020337989, 0.049920519, 0.020337989,
0.138668107, 0.049920519, 0.020337989, -0.083200864, -0.024035805,
-0.016640173, -0.03882707, -0.03882707, 0.005546724, -0.090596497,
-0.00924454, -0.016640173, -0.075805232, -0.090596497, 0.012942357,
-0.075805232, -0.061013967, -0.03882707, -0.112783394, -0.068409599,
-0.090596497, -0.053618335, -0.075805232, -0.090596497, 0.064711783,
0.012942357, 0.042524886, -0.061013967, -0.061013967, 0.064711783,
0.175646269, -0.068409599, 0.027733621, 0.042524886, -0.03882707,
-0.00924454, 0.027733621, -0.031431438, -0.046222702, -0.031431438,
-0.068409599, -0.120179026, 0.035129254, -0.061013967, 0.39751524,
0.138668107, 0.020337989, 0.035129254), Not.Running = c(-0.438809944,
-0.539013927, -0.539013927, -0.539013927, -0.472211271, -0.071395338,
-0.071395338, 0.296019267, 0.563229889, -0.03799401, 0.195815284,
-0.171599321, -0.305204632, 0.062209973, -0.104796666, 0.095611301,
0.028808645, -0.071395338, 0.329420595, 0.296019267, -0.171599321,
-0.071395338, 0.596631217, 0.062209973, 0.028808645, -0.138197994,
0.095611301, -0.104796666, 0.296019267, 0.028808645, -0.03799401,
-0.33860596, 0.129012629, 0.195815284, -0.03799401, 0.396223251,
0.362821923, -0.138197994, 0.26261794, -0.405408616, -0.205000649,
0.129012629, 0.195815284, -0.205000649, -0.004592683, -0.205000649,
-0.071395338, -0.171599321, -0.104796666, -0.138197994, -0.104796666,
-0.071395338, -0.104796666, -0.03799401, -0.004592683, -0.238401977,
0.028808645, -0.305204632, -0.305204632, -0.271803305, -0.03799401,
-0.372007288, 0.095611301, 0.195815284, 0.162413956, 0.229216612,
0.229216612, 0.396223251, 0.630032545, 0.463025906, 0.496427234,
0.062209973, -0.071395338, 0.229216612, -0.071395338, -0.071395338,
-0.205000649, 0.229216612, -0.305204632, 0.396223251), Fighting = c(-0.67708172,
-0.58224128, -0.11436177, -0.34830152, -0.84568695, -0.32933343,
0.35984044, -0.3251183, 1.51478626, 0.11114773, 0.27975296,
-0.89626852, 0.12379312, 0.66965255, 1.56536783, 0.56427428,
-0.71291033, -0.75927677, -0.75295407, -1.00164679, -1.03958296,
0.82139726, -1.07541157, -1.0311527, -0.98900139, -1.06908888,
-1.20186549, 0.58324237, -0.9700333, 0.22917139, 0.41042201,
-1.11545531, -0.19023412, 0.25446217, -0.05324237, 0.09007207,
1.21129685, 0.62539368, 1.32932051, 0.40199175, 0.44625062,
0.60221046, 0.33665722, -0.63493041, -0.282967, -0.32722587,
-0.11646933, -0.10171637, 0.13643851, -0.57802615, 0.05002833,
-0.1607282, -0.29139726, 0.13222338, -0.41152848, 0.68229794,
-0.24292325, -0.11646933, -0.21341734, -0.24292325, -0.24292325,
0.09007207, -0.34197883, -0.30825778, -0.08696342, -0.8119659,
0.49683219, -0.13754498, -0.4831857, 0.39988418, 0.90148474,
0.28396809, 1.05322945, 1.24923303, 0.47154141, 1.27873894,
0.05002833, 1.54218461, 0.74763247, 0.11747042), Not.Fighting = c(-0.097624192,
-0.160103675, -0.092996082, -0.234153433, -0.136963126, -0.15778962,
-0.15778962, -0.023574435, 0.00188017, -0.224897213, -0.109194467,
-0.069855533, -0.123078796, -0.111508522, -0.143905291, -0.099938247,
-0.118450687, 1.519900201, 0.177748344, 0.108326696, 0.652129604,
0.638245274, -0.072169588, 0.087500202, -0.18093017, -0.146219346,
-0.049029039, -0.125392851, -0.134649071, -0.060599313, -0.086053918,
-0.197128554, -0.083739863, -0.092996082, 0.844196163, 0.055103433,
1.971140911, -0.111508522, -0.224897213, -0.187872334, -0.160103675,
-0.194814499, -0.053657149, -0.206384774, 0.108326696, -0.164731785,
0.187004564, 0.025020719, 0.057417488, 0.434608441, 0.057417488,
0.073615872, -0.035144709, -0.051343094, -0.134649071, -0.185558279,
0.013450444, -0.134649071, -0.215640993, -0.185558279, -0.005061995,
-0.238781543, -0.099938247, -0.16704584, -0.208698829, 0.048161268,
0.048161268, -0.037458764, 0.16154996, 0.031962884, -0.102252302,
-0.123078796, -0.139277181, -0.208698829, -0.118450687, -0.072169588,
-0.044400929, -0.030516599, -0.132335016, -0.037458764),
Resting = c(0.01081204879, -0.03398160805, 0.057108797, -0.04063432116,
-0.13084281035, -0.02997847693, 0.12732080268, -0.1028170581,
0.08155320398, -0.17932134171, -0.14338902206, -0.02058415581,
-0.11528274705, -0.11764091337, 0.04389156236, 0.01399844913,
-0.05755560242, 0.04711630687, 0.0158428036, 0.093485909,
0.09677967302, 0.02053612974, -0.03608286844, 0.07805238146,
-9.686695e-05, -0.02285413055, -0.00424187149, 0.01446241356,
0.03187450017, 0.11323315542, -0.01171898422, -0.06499053655,
-0.07758659568, -0.07399758157, -0.11503350996, 0.02167111711,
0.01904454162, 0.05768779393, 0.05555202379, -0.01031175326,
-0.00458313459, 0.17430774591, 0.00481502094, -0.00928412956,
0.09047589183, 0.08917985896, -0.05671203072, -0.05333390954,
0.08541446168, 0.10140397965, -0.02509342995, -0.0369877908,
0.04609635201, 0.06524159499, 0.0845977309, -0.03239032508,
-0.03208740616, 0.06264952925, 0.05241547086, -0.03437271856,
-0.03437271856, -0.06747523863, -0.01270059491, 0.10014629095,
-0.02872845706, -0.00950652573, 0.04867308008, 0.02486518629,
-0.05951115497, -0.02353665674, -0.01967923345, -0.10148651548,
-0.00480936518, -0.00098261723, -0.13970798195, -0.00286148145,
-0.05492902692, 0.10732815358, 0.11660744219, -0.02016620439
), Not.Resting = c(-0.77046287, 0.773856776, -2.593072768,
-2.837675606, -1.680828329, -0.947623773, -0.947623773, -2.607366431,
-0.637055341, -1.818396455, 2.170944974, -0.658126752, -0.808243774,
2.377766908, 2.111220276, -0.322326312, 2.218858946, 3.920878638,
-0.304945754, 1.038591535, 1.752268128, 0.907465624, 1.137774798,
-3.663486997, 2.350924346, 0.067293462, -1.898454393, -2.497647463,
-4.471716512, -1.465081244, -0.232806371, -3.043893581, -2.323908986,
1.437404886, 1.079056696, 1.110865131, 1.404724068, -1.706664294,
0.736746935, -0.005516985, 1.727170333, 1.685228831, 1.836016918,
0.46617392, 1.697173771, 1.057314221, 0.933704227, 0.482480775,
0.680713089, 0.090780703, 0.680713089, -0.982921741, -2.281900378,
0.97208909, 0.027767791, -0.1628815, -0.530221948, -0.385741863,
-0.972251823, 0.002267358, -1.134447998, 0.626424009, -0.722750217,
-0.382722075, -0.356550578, -1.851614124, -1.851614124, 1.731465143,
0.254319006, 2.043778341, -0.28991392, 1.386940871, 0.054207713,
0.594212936, 1.551821303, 3.100704184, 0.327263666, -1.055195336,
-1.134447998, 1.730726972), Hunting = c(-0.67708172, -0.58224128,
-0.11436177, -0.34830152, -0.84568695, -0.32933343, 0.35984044,
-0.3251183, 1.51478626, 0.11114773, 0.27975296, -0.89626852,
0.12379312, 0.66965255, 1.56536783, 0.56427428, -0.71291033,
-0.75927677, -0.75295407, -1.00164679, -1.03958296, 0.82139726,
-1.07541157, -1.0311527, -0.98900139, -1.06908888, -1.20186549,
0.58324237, -0.9700333, 0.22917139, 0.41042201, -1.11545531,
-0.19023412, 0.25446217, -0.05324237, 0.09007207, 1.21129685,
0.62539368, 1.32932051, 0.40199175, 0.44625062, 0.60221046,
0.33665722, -0.63493041, -0.282967, -0.32722587, -0.11646933,
-0.10171637, 0.13643851, -0.57802615, 0.05002833, -0.1607282,
-0.29139726, 0.13222338, -0.41152848, 0.68229794, -0.24292325,
-0.11646933, -0.21341734, -0.24292325, -0.24292325, 0.09007207,
-0.34197883, -0.30825778, -0.08696342, -0.8119659, 0.49683219,
-0.13754498, -0.4831857, 0.39988418, 0.90148474, 0.28396809,
1.05322945, 1.24923303, 0.47154141, 1.27873894, 0.05002833,
1.54218461, 0.74763247, 0.11747042), Not.Hunting = c(-0.097624192,
-0.160103675, -0.092996082, -0.234153433, -0.136963126, -0.15778962,
-0.15778962, -0.023574435, 0.00188017, -0.224897213, -0.109194467,
-0.069855533, -0.123078796, -0.111508522, -0.143905291, -0.099938247,
-0.118450687, 1.519900201, 0.177748344, 0.108326696, 0.652129604,
0.638245274, -0.072169588, 0.087500202, -0.18093017, -0.146219346,
-0.049029039, -0.125392851, -0.134649071, -0.060599313, -0.086053918,
-0.197128554, -0.083739863, -0.092996082, 0.844196163, 0.055103433,
1.971140911, -0.111508522, -0.224897213, -0.187872334, -0.160103675,
-0.194814499, -0.053657149, -0.206384774, 0.108326696, -0.164731785,
0.187004564, 0.025020719, 0.057417488, 0.434608441, 0.057417488,
0.073615872, -0.035144709, -0.051343094, -0.134649071, -0.185558279,
0.013450444, -0.134649071, -0.215640993, -0.185558279, -0.005061995,
-0.238781543, -0.099938247, -0.16704584, -0.208698829, 0.048161268,
0.048161268, -0.037458764, 0.16154996, 0.031962884, -0.102252302,
-0.123078796, -0.139277181, -0.208698829, -0.118450687, -0.072169588,
-0.044400929, -0.030516599, -0.132335016, -0.037458764),
Grooming = c(0.01081204879, -0.03398160805, 0.057108797,
-0.04063432116, -0.13084281035, -0.02997847693, 0.12732080268,
-0.1028170581, 0.08155320398, -0.17932134171, -0.14338902206,
-0.02058415581, -0.11528274705, -0.11764091337, 0.04389156236,
0.01399844913, -0.05755560242, 0.04711630687, 0.0158428036,
0.093485909, 0.09677967302, 0.02053612974, -0.03608286844,
0.07805238146, -9.686695e-05, -0.02285413055, -0.00424187149,
0.01446241356, 0.03187450017, 0.11323315542, -0.01171898422,
-0.06499053655, -0.07758659568, -0.07399758157, -0.11503350996,
0.02167111711, 0.01904454162, 0.05768779393, 0.05555202379,
-0.01031175326, -0.00458313459, 0.17430774591, 0.00481502094,
-0.00928412956, 0.09047589183, 0.08917985896, -0.05671203072,
-0.05333390954, 0.08541446168, 0.10140397965, -0.02509342995,
-0.0369877908, 0.04609635201, 0.06524159499, 0.0845977309,
-0.03239032508, -0.03208740616, 0.06264952925, 0.05241547086,
-0.03437271856, -0.03437271856, -0.06747523863, -0.01270059491,
0.10014629095, -0.02872845706, -0.00950652573, 0.04867308008,
0.02486518629, -0.05951115497, -0.02353665674, -0.01967923345,
-0.10148651548, -0.00480936518, -0.00098261723, -0.13970798195,
-0.00286148145, -0.05492902692, 0.10732815358, 0.11660744219,
-0.02016620439), Not.Grooming = c(-0.77046287, 0.773856776,
-2.593072768, -2.837675606, -1.680828329, -0.947623773, -0.947623773,
-2.607366431, -0.637055341, -1.818396455, 2.170944974, -0.658126752,
-0.808243774, 2.377766908, 2.111220276, -0.322326312, 2.218858946,
3.920878638, -0.304945754, 1.038591535, 1.752268128, 0.907465624,
1.137774798, -3.663486997, 2.350924346, 0.067293462, -1.898454393,
-2.497647463, -4.471716512, -1.465081244, -0.232806371, -3.043893581,
-2.323908986, 1.437404886, 1.079056696, 1.110865131, 1.404724068,
-1.706664294, 0.736746935, -0.005516985, 1.727170333, 1.685228831,
1.836016918, 0.46617392, 1.697173771, 1.057314221, 0.933704227,
0.482480775, 0.680713089, 0.090780703, 0.680713089, -0.982921741,
-2.281900378, 0.97208909, 0.027767791, -0.1628815, -0.530221948,
-0.385741863, -0.972251823, 0.002267358, -1.134447998, 0.626424009,
-0.722750217, -0.382722075, -0.356550578, -1.851614124, -1.851614124,
1.731465143, 0.254319006, 2.043778341, -0.28991392, 1.386940871,
0.054207713, 0.594212936, 1.551821303, 3.100704184, 0.327263666,
-1.055195336, -1.134447998, 1.730726972), Other = c(0.019502286,
-0.290451956, 0.359948884, 0.557840914, 0.117453376, 0.126645924,
0.126645924, 0.196486873, 0.152780228, 0.354469789, -0.261430968,
0.176448238, -0.007374708, -0.557848621, -0.213674557, -0.005819262,
-0.470070992, -0.786078864, 0.006063789, -0.27184265, -0.349418792,
-0.338096262, -0.165119403, 0.346566439, -0.344191931, 0.074321265,
0.179825379, 0.278407054, 0.593125727, 0.199177375, -0.058900625,
0.633875622, 0.428150308, -0.206023441, -0.436958199, -0.291839246,
-0.907641911, 0.448567295, -0.127186127, 0.024715134, -0.41634503,
-0.330697382, -0.469720666, -0.047494017, -0.301732446, -0.138901021,
0.098101379, -0.002063769, -0.02832419, 0.071630763, -0.02832419,
0.295110588, 0.347112947, -0.083577573, -0.036886152, 0.189045953,
0.467596992, 0.303378276, 0.218879697, 0.092005711, 0.27011134,
-0.012909856, 0.262292068, 0.107125772, 0.123422927, 0.299426602,
0.299426602, -0.326871824, -0.022088391, -0.428508341, -0.014675497,
-0.114462294, 0.087227267, -0.031519161, -0.159318008, -0.397875854,
0.101520559, 0.244481505, 0.529968994, -0.32661959)), .Names = c("Family",
"Swimming", "Not.Swimming", "Running", "Not.Running", "Fighting",
"Not.Fighting", "Resting", "Not.Resting", "Hunting", "Not.Hunting",
"Grooming", "Not.Grooming", "Other"), class = "data.frame", row.names=c(NA, -80L))
对于混淆矩阵问题，您为其提供 class 概率（例如 lda_pred）而不是因子预测。有关对象应该是什么的详细信息，请参阅 ?confusionMatrix。
对于 NMI，confusionMatrix 会给你一个 table 来使用。例如：
> library(caret)
>
> set.seed(1)
> dat <- twoClassSim(200)
> dat2 <- twoClassSim(200)
>
> set.seed(2)
> mod <- train(Class ~ ., data = dat, method = "lda")
>
> preds <- predict(mod, dat2)
>
> cm <- confusionMatrix(preds, dat2$Class)
> cm$table
Reference
Prediction Class1 Class2
Class1 78 30
Class2 17 75