如何使用 Spacy v3 在命名实体识别中生成 Precision、Recall 和 F-score？正在寻求 ents_p、ents_r、ents_f 的小型自定义 NER 模型

Question

示例代码如下，您可以在该示例中添加一个或多个实体用于训练目的（您也可以使用带有小示例的空白模型进行演示）。我正在寻找用于自定义 NER 模型评估（精度、召回率、f 分数）的完整工作解决方案，提前感谢所有 NLP 专家。

import spacy
nlp = spacy.load('en_core_web_sm')
example_text = "Agra is famous for Tajmahal, The CEO of Facebook will visit India shortly to meet Murari Mahaseth and to visit Tajmahal"
ner = nlp.get_pipe("ner")
doc = nlp(example_text)
for e in doc.ents:
    print(e.text + ' - ' + str(e.start_char) + ' - ' + str(e.end_char) + ' - ' + e.label_ + ' - ' + str(
        spacy.explain(e.label_)))

Answer 1

我举个简单的例子:

import spacy
from spacy.scorer import Scorer
from spacy.tokens import Doc
from spacy.training.example import Example

examples = [
    ('Who is Talha Tayyab?',
     {(7, 19, 'PERSON')}),
    ('I like London and Berlin.',
     {(7, 13, 'LOC'), (18, 24, 'LOC')}),
     ('Agra is famous for Tajmahal, The CEO of Facebook will visit India shortly to meet Murari Mahaseth and to visit Tajmahal.',
     {(0, 4, 'LOC'), (40, 48, 'ORG'), (60, 65, 'GPE'), (82, 97, 'PERSON'), (111, 119, 'GPE')})
]

def my_evaluate(ner_model, examples):
    scorer = Scorer()
    example = []
    for input_, annotations in examples:
        pred = ner_model(input_)
        print(pred,annotations)
        temp = Example.from_dict(pred, dict.fromkeys(annotations))
        example.append(temp)
    scores = scorer.score(example)
    return scores

ner_model = spacy.load('en_core_web_sm') # for spaCy's pretrained use 'en_core_web_sm'
results = my_evaluate(ner_model, examples)
print(results)

我说了这只是一个例子，大家可以根据自己的需要进行修改。

如何使用 Spacy v3 在命名实体识别中生成 Precision、Recall 和 F-score？正在寻求 ents_p、ents_r、ents_f 的小型自定义 NER 模型

How to generate Precision, Recall and F-score in Named Entity Recognition using Spacy v3? Seeking ents_p, ents_r, ents_f for a small custom NER model

python

named-entity-recognition

precision-recall

spacy-3