相同的测试和预测值给出 NER 的精度、召回率和 f1 分数为 0

Question

我正在使用 sklearns crfsuite 来计算 f1、精度和召回分数，但出现异常情况。出于测试目的，我给出了相同的测试值和预测值。

from sklearn_crfsuite import scorers
from sklearn_crfsuite import metrics

cls = [i for i, _ in enumerate(CLASSES)]
cls.append(7)
cls.append(8)

print(metrics.flat_classification_report(
    test["y"], test["y"], labels=cls, digits=3
))

              precision    recall  f1-score   support

           0      1.000     1.000     1.000       551
           1      0.000     0.000     0.000         0
           2      0.000     0.000     0.000         0
           3      1.000     1.000     1.000      1196
           4      1.000     1.000     1.000      2593
           5      1.000     1.000     1.000     95200
           6      1.000     1.000     1.000      1165
           7      1.000     1.000     1.000      9636
           8      1.000     1.000     1.000    506363

   micro avg      1.000     1.000     1.000    616704
   macro avg      0.778     0.778     0.778    616704
weighted avg      1.000     1.000     1.000    616704

为什么标签 1 和标签 2 的分数都是 0。它应该给出 1 作为其余数据。谁能给我解释一下原因吗？

需要帮助。提前致谢！

Answer 1

您的数据中似乎实际上没有类 1 和 2，因为这两个类的支持度为零，但由于您包含了类 1和传递给 flat_classification_report() 的标签列表中的 2 个在计算各种指标时仍会考虑它们。

from sklearn_crfsuite import metrics
import numpy as np
np.random.seed(0)

cmin = 0
cmax = 8

labels = np.arange(1 + cmax)
print(np.unique(labels))
# [0 1 2 3 4 5 6 7 8]

y = np.random.randint(cmin, 1 + cmax, 1000).reshape(-1, 1)
print(np.unique(y))
# [0 1 2 3 4 5 6 7 8]

# classification report when "y" takes on all the specified labels
print(metrics.flat_classification_report(y_true=y, y_pred=y, labels=labels, digits=3))
#               precision    recall  f1-score   support
#            0      1.000     1.000     1.000       117
#            1      1.000     1.000     1.000       106
#            2      1.000     1.000     1.000       106
#            3      1.000     1.000     1.000       132
#            4      1.000     1.000     1.000       110
#            5      1.000     1.000     1.000       115
#            6      1.000     1.000     1.000       104
#            7      1.000     1.000     1.000       109
#            8      1.000     1.000     1.000       101
#     accuracy                          1.000      1000
#    macro avg      1.000     1.000     1.000      1000
# weighted avg      1.000     1.000     1.000      1000

# classification report when "y" takes on all the specified labels apart from 1 and 2,
# but 1 and 2 are still included among the possible labels
y = y[np.logical_and(y != 1, y != 2)].reshape(-1, 1)
print(np.unique(y))
# [0 3 4 5 6 7 8]

print(metrics.flat_classification_report(y_true=y, y_pred=y, labels=labels, digits=3))
#               precision    recall  f1-score   support
#            0      1.000     1.000     1.000       117
#            1      0.000     0.000     0.000         0
#            2      0.000     0.000     0.000         0
#            3      1.000     1.000     1.000       132
#            4      1.000     1.000     1.000       110
#            5      1.000     1.000     1.000       115
#            6      1.000     1.000     1.000       104
#            7      1.000     1.000     1.000       109
#            8      1.000     1.000     1.000       101
#    micro avg      1.000     1.000     1.000       788
#    macro avg      0.778     0.778     0.778       788
# weighted avg      1.000     1.000     1.000       788

# classification report when "y" takes on all the specified labels apart from 1 and 2,
# and 1 and 2 are not included among the possible labels
labels = labels[np.logical_and(labels != 1, labels != 2)]
print(np.unique(labels))
# [0 3 4 5 6 7 8]

print(metrics.flat_classification_report(y_true=y, y_pred=y, labels=labels, digits=3))
#               precision    recall  f1-score   support
#            0      1.000     1.000     1.000       117
#            3      1.000     1.000     1.000       132
#            4      1.000     1.000     1.000       110
#            5      1.000     1.000     1.000       115
#            6      1.000     1.000     1.000       104
#            7      1.000     1.000     1.000       109
#            8      1.000     1.000     1.000       101
#     accuracy                          1.000       788
#    macro avg      1.000     1.000     1.000       788
# weighted avg      1.000     1.000     1.000       788

相同的测试和预测值给出 NER 的精度、召回率和 f1 分数为 0

Same test and prediction values gives 0 precision, recall, f1 score for NER

python

named-entity-recognition

scikit-learn

precision-recall