在 scikit 学习中从 LDA 获取主题词分布

Question

我想知道在scikit的LDA实现中是否有一种方法可以学习returns主题词分布。就像 genism show_topics() 方法一样。我检查了文档，但没有找到任何东西。

Answer 1

看看sklearn.decomposition.LatentDirichletAllocation.components_:

components_ : array, [n_topics, n_features]

Topic word distribution. components_[i, j] represents word j in topic i.

这是一个最小的例子：

import numpy as np
from sklearn.decomposition import LatentDirichletAllocation
from sklearn.feature_extraction.text import CountVectorizer

data = ['blah blah foo bar', 'foo foo foo foo bar', 'bar bar bar bar foo',
        'foo bar bar bar baz foo', 'foo foo foo bar baz', 'blah banana', 
        'cookies candy', 'more text please', 'hey there are more words here',
        'bananas', 'i am a real boy', 'boy', 'girl']

vectorizer = CountVectorizer()
X = vectorizer.fit_transform(data)

vocab = vectorizer.get_feature_names()

n_top_words = 5
k = 2

model = LatentDirichletAllocation(n_topics=k, random_state=100)

id_topic = model.fit_transform(X)

topic_words = {}

for topic, comp in enumerate(model.components_):
    # for the n-dimensional array "arr":
    # argsort() returns a ranked n-dimensional array of arr, call it "ranked_array"
    # which contains the indices that would sort arr in a descending fashion
    # for the ith element in ranked_array, ranked_array[i] represents the index of the
    # element in arr that should be at the ith index in ranked_array
    # ex. arr = [3,7,1,0,3,6]
    # np.argsort(arr) -> [3, 2, 0, 4, 5, 1]
    # word_idx contains the indices in "topic" of the top num_top_words most relevant
    # to a given topic ... it is sorted ascending to begin with and then reversed (desc. now)    
    word_idx = np.argsort(comp)[::-1][:n_top_words]

    # store the words most relevant to the topic
    topic_words[topic] = [vocab[i] for i in word_idx]

查看结果：

for topic, words in topic_words.items():
    print('Topic: %d' % topic)
    print('  %s' % ', '.join(words))

Topic: 0
  more, blah, here, hey, words
Topic: 1
  foo, bar, blah, baz, boy

您显然应该使用更大的文本尝试此代码，但这是为给定主题数量获取最多信息的单词的一种方法。

在 scikit 学习中从 LDA 获取主题词分布

Getting topic-word distribution from LDA in scikit learn

python

scikit-learn

lda