用前面提到的人替换人称代词（嘈杂的 coref）

Question

我想做一个嘈杂的解决方案，这样给定一个人称代词，这个代词就会被前一个（最近的）人代替。

例如：

Alex is looking at buying a U.K. startup for billion. He is very confident that this is going to happen. Sussan is also in the same situation. However, she has lost hope.

输出为：

Alex is looking at buying a U.K. startup for billion. Alex is very confident that this is going to happen. Sussan is also in the same situation. However, Susan has lost hope.

另一个例子，

Peter is a friend of Gates. But Gates does not like him.

在这种情况下，输出将是：

Peter is a friend of Gates. But Gates does not like Gates.

是的！太吵了

使用spacy：我已经使用 NER 提取了 Person，但是我怎样才能适当地替换代词？

代码：

import spacy
nlp = spacy.load("en_core_web_sm")
for ent in doc.ents:
  if ent.label_ == 'PERSON':
    print(ent.text, ent.label_)

Answer 1

我已经编写了一个适用于您的两个示例的函数：

考虑使用更大的模型，例如 en_core_web_lg 以获得更准确的标记。

import spacy
from string import punctuation

nlp = spacy.load("en_core_web_lg")

def pronoun_coref(text):
    doc = nlp(text)
    pronouns = [(tok, tok.i) for tok in doc if (tok.tag_ == "PRP")]
    names = [(ent.text, ent[0].i) for ent in doc.ents if ent.label_ == 'PERSON']
    doc = [tok.text_with_ws for tok in doc]
    for p in pronouns:
        replace = max(filter(lambda x: x[1] < p[1], names),
                      key=lambda x: x[1], default=False)
        if replace:
            replace = replace[0]
            if doc[p[1] - 1] in punctuation:
                replace = ' ' + replace
            if doc[p[1] + 1] not in punctuation:
                replace = replace + ' '
            doc[p[1]] = replace
    doc = ''.join(doc)
    return doc

Answer 2

有专门的 neuralcoref 库来解析指代。请参阅下面的最小可重现示例：

import spacy
import neuralcoref

nlp = spacy.load('en_core_web_sm')
neuralcoref.add_to_pipe(nlp)
doc = nlp(
'''Alex is looking at buying a U.K. startup for  billion. 
He is very confident that this is going to happen. 
Sussan is also in the same situation. 
However, she has lost hope.
Peter is a friend of Gates. But Gates does not like him.
          ''')

print(doc._.coref_resolved)

Alex is looking at buying a U.K. startup for  billion. 
Alex is very confident that this is going to happen. 
Sussan is also in the same situation. 
However, Sussan has lost hope.
Peter is a friend of Gates. But Gates does not like Peter.

请注意，如果您 pip 安装它，您可能会遇到 neuralcoref 的一些问题，因此最好从源代码构建它，正如我概述的那样 here

用前面提到的人替换人称代词（嘈杂的 coref）

Replace personal pronoun with previous person mentioned (noisy coref)

python

nlp

python-3.x

spacy

coreference-resolution