警告：[W108] 基于规则的词形还原器未找到令牌的 POS 注释 'This'

Question

这条消息是关于什么的？如何删除此警告消息？

import scispacy
import spacy
import en_core_sci_lg
from spacy_langdetect import LanguageDetector
from spacy.language import Language
from spacy.tokens import Doc


def create_lang_detector(nlp, name):
    return LanguageDetector()


Language.factory("language_detector", func=create_lang_detector)
nlp = en_core_sci_lg.load(disable=["tagger", "ner"])
nlp.max_length = 2000000
nlp.add_pipe('language_detector', last=True)

doc = nlp('This is some English text. Das ist ein Haus. This is a house.')

Warning:

[W108] The rule-based lemmatizer did not find POS annotation for the token 'This'. Check that your pipeline includes components that assign token.pos, typically 'tagger'+'attribute_ruler' or 'morphologizer'.

[W108] The rule-based lemmatizer did not find POS annotation for the token 'is'. Check that your pipeline includes components that assign token.pos, typically 'tagger'+'attribute_ruler' or 'morphologizer'.

[W108] The rule-based lemmatizer did not find POS annotation for the token 'some'. Check that your pipeline includes components that assign token.pos, typically 'tagger'+'attribute_ruler' or 'morphologizer'.
. . . .

Answer 1

lemmatizer 是与 spacy v3 中的标记器分开的组件。禁用词形还原器和标记器以避免这些警告：

nlp = en_core_sci_lg.load(disable=["tagger", "ner", "lemmatizer"])

警告：[W108] 基于规则的词形还原器未找到令牌的 POS 注释 'This'

Warning: [W108] The rule-based lemmatizer did not find POS annotation for the token 'This'

python

spacy

spacy-3