如何从 Python 中的单词词典中删除 unigrams

Question

我正在尝试在 python 中创建一个只有双字母组的词云。现在我有一个字典如下：

Word_dict

{'delivered later requested_delivered later requested': 0.07590105638848002,
 'delayed delivery_delayed delivery': 0.043280231684707335,
 'guidelines followed_guidelines followed': 0.04056653336980544,
 'delayed pickup_delayed pickup': 0.02733236942769188,
 'delivered later requested_delayed delivery': 0.023815416411579027,
 'delayed delivery_delivered later requested': 0.02332477975624476,
 'guidelines followed_delivered later requested': 0.02131881396186928,
 'delivered later requested_guidelines followed': 0.020793441968104277,
 'delayed pickup_delayed delivery': 0.020619765275950556,
 'delayed delivery_guidelines followed': 0.01998150343228563,
 'delayed delivery_delayed pickup': 0.019464815273128308,
 'guidelines followed_delayed delivery': 0.018900366023628715,
 'delivered later requested_delayed pickup': 0.01870932166225962,
 'delayed pickup_delivered later requested': 0.0185660383912328,
 'guidelines followed_delayed pickup': 0.015148949473108336,
 'delayed pickup_guidelines followed': 0.01475383499845862,
 'super user activity fom_super user activity fom': 0.010490072206084763}

我需要从 dictionary.How 中删除不带下划线的单字母组或单词

可以。

预期输出

{' requested_delivered ': 0.07590105638848002,
 'delivery_delayed ': 0.043280231684707335,
 'followed_guidelines': 0.04056653336980544,
 'pickup_delayed ': 0.02733236942769188,
 ' requested_delayed ': 0.023815416411579027}

怎么做？

我的代码

def preprocess(x):
    x = re.sub('[^a-z\s]', '', x.lower())                  # get rid of noise
    x = [w for w in x.split() if w not in set(newstopwords)]  # remove stopwords
    return ' '.join(x)

data['Clean_addr'] = data['Reason Code Level 1'].apply(preprocess)

# setup and score the bigrams using the raw frequency.
finder = BigramCollocationFinder.from_words(text_content)
bigram_measures = BigramAssocMeasures()
scored = finder.score_ngrams(bigram_measures.raw_freq)

# By default finder.score_ngrams is sorted, however don't rely on this default behavior.
# Sort highest to lowest based on the score.
scoredList = sorted(scored, key=itemgetter(1), reverse=True)

# word_dict is the dictionary we'll use for the word cloud.
# Load dictionary with the FOR loop below.
# The dictionary will look like this with the bigram and the score from above.
# word_dict = {'bigram A': 0.000697411,
#             'bigram B': 0.000524882}

word_dict = {}

listLen = len(scoredList)

# Get the bigram and make a contiguous string for the dictionary key. 
# Set the key to the scored value. 
for i in range(listLen):
    word_dict['_'.join(scoredList[i][0])] = scoredList[i][1]



# -----

Answer 1

如果你能从你的数据集中保证整个集合中只有零个或一个包含下划线的短语：

# starting from scoredList in the example above (had to add scored.items()) so that you're iterating over key/value pairs
scoredList = sorted(scored.items(), key=itemgetter(1), reverse=True)

new_data = {}
for key, value in scoredList:
    words = [word for word in key.split(' ') if '_' in word]
    if len(words) == 1:
        new_data[words[0]] = value
    elif len(words) > 1:
        raise ValueError('oh no...')

print(new_data)

如何从 Python 中的单词词典中删除 unigrams

How to remove unigrams from word dictionary in Python

regex

python-3.x

word-cloud

pandas