gensim lemmatize 错误生成器引发了 StopIteration

Question

我正在尝试执行简单的代码来对字符串进行词形还原，但出现关于迭代的错误。我找到了一些关于重新安装的解决方案 web.py，但这对我不起作用。

python代码

from gensim.utils import lemmatize
lemmatize("gone")

错误是

---------------------------------------------------------------------------
StopIteration                             Traceback (most recent call last)
I:\Anaconda\lib\site-packages\pattern\text\__init__.py in _read(path, encoding, comment)
    608             yield line
--> 609     raise StopIteration
    610 

StopIteration: 

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
<ipython-input-4-9daceee1900f> in <module>
      1 from gensim.utils import lemmatize
----> 2 lemmatize("gone")

-------------------------------------------------------------------------------------

I:\Anaconda\lib\site-packages\pattern\text\__init__.py in <genexpr>(.0)
    623     def load(self):
    624         # Arnold NNP x
--> 625         dict.update(self, (x.split(" ")[:2] for x in _read(self._path) if len(x.split(" ")) > 1))
    626 
    627 #--- FREQUENCY -------------------------------------------------------------------------------------

RuntimeError: generator raised StopIteration

Answer 1

该错误消息具有误导性——当没有任何内容可以正确词形化时就会出现。

默认情况下，lemmatize() 只接受单词标签 NN|VB|JJ|RB。传入一个匹配任何字符串的正则表达式来改变这个：

>>> import re
>>> lemmatize("gone", allowed_tags=re.compile('.*'))
[b'go/VB']

gensim lemmatize 错误生成器引发了 StopIteration

gensim lemmatize error generator raised StopIteration

python

nlp

gensim

lemmatization