NLTK 计算子短语的频率

Question

对于这句话："I see a tall tree outside. A man is under the tall tree"

如何计算tall tree的频率？我可以在搭配中使用二元组，例如

bgs= nltk.bigrams(tokens)
fdist1= nltk.FreqDist(bgs)
pairs = fdist1.most_common(500)

但我只需要计算一个特定的子短语。

Answer 1

count() 方法应该这样做：

string = "I see a tall tree outside. A man is under the tall tree"
string.count("tall tree")

Answer 2

@uday1889 的回答有一些瑕疵：

>>> string = "I see a tall tree outside. A man is under the tall tree"
>>> string.count("tall tree")
2
>>> string = "The see a stall tree outside. A man is under the tall trees"
>>> string.count("tall tree")
2
>>> string = "I would like to install treehouses at my yard"
>>> string.count("tall tree")
1

一个廉价的技巧是在 str.count() 中填充 space:

>>> string = "I would like to install treehouses at my yard"
>>> string.count("tall tree")
1
>>> string.count(" tall tree ")
0
>>> string = "The see a stall tree outside. A man is under the tall trees"
>>> string.count(" tall tree ")
0
>>> string = "I see a tall tree outside. A man is under the tall tree"
>>> string.count(" tall tree ")
1

但是如您所见，当子字符串位于句子的开头或结尾或标点符号旁边时会出现一些问题。

>>> from nltk.util import ngrams
>>> from nltk import word_tokenize
>>> string = "I see a tall tree outside. A man is under the tall tree"
>>> len([i for i in ngrams(word_tokenize(string),n=2) if i==('tall', 'tree')])
2
>>> string = "I would like to install treehouses at my yard"
>>> len([i for i in ngrams(word_tokenize(string),n=2) if i==('tall', 'tree')])
0

NLTK 计算子短语的频率

NLTK count frequency of sub phrase

python

nltk