使用 Stanza 和 CoreNLPClient 提取名词短语

Question

我正在尝试使用 Stanza（使用 Stanza CoreNLP）从句子中提取名词短语。这只能通过 Stanza 中的 CoreNLPClient 模块来完成。

# Import client module
from stanza.server import CoreNLPClient
# Construct a CoreNLPClient with some basic annotators, a memory allocation of 4GB, and port number 9001
client = CoreNLPClient(annotators=['tokenize','ssplit','pos','lemma','ner', 'parse'], memory='4G', endpoint='http://localhost:9001')

这是一个句子的例子，我在客户端使用tregrex函数来获取所有的名词短语。 Tregex 函数 returns python 中的 dict of dicts。因此，我需要先处理 tregrex 的输出，然后再将其传递给 NLTK 中的 Tree.fromstring 函数，以正确地将名词短语提取为字符串。

pattern = 'NP'
text = "Albert Einstein was a German-born theoretical physicist. He developed the theory of relativity."
matches = client.tregrex(text, pattern) ``

因此，我想出了方法 stanza_phrases，它必须循环遍历 dict of dicts，这是 tregrex 的输出，并在 NLTK 中正确格式化 Tree.fromstring .

def stanza_phrases(matches):
  Nps = []
  for match in matches:
    for items in matches['sentences']:
      for keys,values in items.items():
        s = '(ROOT\n'+ values['match']+')'
        Nps.extend(extract_phrase(s, pattern))
  return set(Nps)

生成一棵供 NLTK 使用的树

from nltk.tree import Tree
def extract_phrase(tree_str, label):
    phrases = []
    trees = Tree.fromstring(tree_str)
    for tree in trees:
        for subtree in tree.subtrees():
            if subtree.label() == label:
                t = subtree
                t = ' '.join(t.leaves())
                phrases.append(t)

    return phrases

这是我的输出：

{'Albert Einstein', 'He', 'a German-born theoretical physicist', 'relativity',  'the theory', 'the theory of relativity'}

有没有一种方法可以用更少的行数提高代码效率（尤其是 stanza_phrases 和 extract_phrase 方法）

Answer 1

from stanza.server import CoreNLPClient

# get noun phrases with tregex
def noun_phrases(_client, _text, _annotators=None):
    pattern = 'NP'
    matches = _client.tregex(_text,pattern,annotators=_annotators)
    print("\n".join(["\t"+sentence[match_id]['spanString'] for sentence in matches['sentences'] for match_id in sentence]))

# English example
with CoreNLPClient(timeout=30000, memory='16G') as client:
    englishText = "Albert Einstein was a German-born theoretical physicist. He developed the theory of relativity."
    print('---')
    print(englishText)
    noun_phrases(client,englishText,_annotators="tokenize,ssplit,pos,lemma,parse")

# French example
with CoreNLPClient(properties='french', timeout=30000, memory='16G') as client:
    frenchText = "Je suis John."
    print('---')
    print(frenchText)
    noun_phrases(client,frenchText,_annotators="tokenize,ssplit,mwt,pos,lemma,parse")

使用 Stanza 和 CoreNLPClient 提取名词短语

Extract Noun Phrases with Stanza and CoreNLPClient

python

nlp

stanford-nlp

stanford-stanza