NLTK WordNetLemmatizer 中的多线程?
Multi Threading in NLTK WordNetLemmatizer?
我正在尝试使用多线程来加快进程。我正在使用 wordnetlemmatizer 对单词进行词形还原,sentiwordnet 可以进一步使用这些单词来计算文本的情感。我使用 WordNetLemmatizer 的情感分析函数如下:
import nltk
from nltk.corpus import sentiwordnet as swn
def SentimentA(doc, file_path):
sentences = nltk.sent_tokenize(doc)
# print(sentences)
stokens = [nltk.word_tokenize(sent) for sent in sentences]
taggedlist = []
for stoken in stokens:
taggedlist.append(nltk.pos_tag(stoken))
wnl = nltk.WordNetLemmatizer()
score_list = []
for idx, taggedsent in enumerate(taggedlist):
score_list.append([])
for idx2, t in enumerate(taggedsent):
newtag = ''
lemmatized = wnl.lemmatize(t[0])
if t[1].startswith('NN'):
newtag = 'n'
elif t[1].startswith('JJ'):
newtag = 'a'
elif t[1].startswith('V'):
newtag = 'v'
elif t[1].startswith('R'):
newtag = 'r'
else:
newtag = ''
if (newtag != ''):
synsets = list(swn.senti_synsets(lemmatized, newtag))
score = 0
if (len(synsets) > 0):
for syn in synsets:
score += syn.pos_score() - syn.neg_score()
score_list[idx].append(score / len(synsets))
return SentiCal(score_list)
在 运行 4 个线程之后,前 3 个线程出现以下错误,最后一个线程运行正常。
AttributeError: 'WordNetCorpusReader' object has no attribute '_LazyCorpusLoader__args'
我已经尝试按照此 NLTK issue 中给出的方式在本地导入 NLTK 包
并尝试了 page.
上给出的解决方案
快速破解:
import nltk
from nltk.corpus import sentiwordnet as swn
# Do this first, that'll do something eval()
# to "materialize" the LazyCorpusLoader
next(swn.all_senti_synsets())
# Your other code here.
更多详细信息稍后...仍在输入中
我正在尝试使用多线程来加快进程。我正在使用 wordnetlemmatizer 对单词进行词形还原,sentiwordnet 可以进一步使用这些单词来计算文本的情感。我使用 WordNetLemmatizer 的情感分析函数如下:
import nltk
from nltk.corpus import sentiwordnet as swn
def SentimentA(doc, file_path):
sentences = nltk.sent_tokenize(doc)
# print(sentences)
stokens = [nltk.word_tokenize(sent) for sent in sentences]
taggedlist = []
for stoken in stokens:
taggedlist.append(nltk.pos_tag(stoken))
wnl = nltk.WordNetLemmatizer()
score_list = []
for idx, taggedsent in enumerate(taggedlist):
score_list.append([])
for idx2, t in enumerate(taggedsent):
newtag = ''
lemmatized = wnl.lemmatize(t[0])
if t[1].startswith('NN'):
newtag = 'n'
elif t[1].startswith('JJ'):
newtag = 'a'
elif t[1].startswith('V'):
newtag = 'v'
elif t[1].startswith('R'):
newtag = 'r'
else:
newtag = ''
if (newtag != ''):
synsets = list(swn.senti_synsets(lemmatized, newtag))
score = 0
if (len(synsets) > 0):
for syn in synsets:
score += syn.pos_score() - syn.neg_score()
score_list[idx].append(score / len(synsets))
return SentiCal(score_list)
在 运行 4 个线程之后,前 3 个线程出现以下错误,最后一个线程运行正常。
AttributeError: 'WordNetCorpusReader' object has no attribute '_LazyCorpusLoader__args'
我已经尝试按照此 NLTK issue 中给出的方式在本地导入 NLTK 包 并尝试了 page.
上给出的解决方案快速破解:
import nltk
from nltk.corpus import sentiwordnet as swn
# Do this first, that'll do something eval()
# to "materialize" the LazyCorpusLoader
next(swn.all_senti_synsets())
# Your other code here.
更多详细信息稍后...仍在输入中