使用字典替换数据框中的互联网首字母缩略词

Question

我正在从事一个文本挖掘项目，我正在尝试使用手动准备的字典替换文本（在数据框列中）中出现的缩写词、俚语和互联网首字母缩略词。

我面临的问题是代码在数据框列中文本的第一个词处停止，并且没有用字典中的查找词替换它

这是我使用的示例字典和代码：

abbr_dict = {"abt":"about", "b/c":"because"}

def _lookup_words(input_text):
    words = input_text.split()
    new_words = [] 
    for word in words:
        if word.lower() in abbr_dict:
            word = abbr_dict[word.lower()]
        new_words.append(word)
        new_text = " ".join(new_words) 
        return new_text
df['new_text'] = df['text'].apply(_lookup_words)

示例输入：

df['text'] =
However, industry experts are divided ab whether a Bitcoin ETF is necessary or not.

期望的输出：

df['New_text'] =
However, industry experts are divided about whether a Bitcoin ETF is necessary or not.

当前输出：

df['New_text'] =
However

Answer 1

您可以尝试使用 lambda 和 join 以及 split：

import pandas as pd

abbr_dict = {"abt":"about", "b/c":"because"}

df = pd.DataFrame({'text': ['However, industry experts are divided abt whether a Bitcoin ETF is necessary or not.']})

df['new_text'] = df['text'].apply(lambda row: " ".join(abbr_dict[w] 
                             if w.lower() in abbr_dict else w for w in row.split()))

或者要修复上面的代码，我认为您需要将 join for new_text 和 return 语句移到 for 循环之外：

def _lookup_words(input_text):
    words = input_text.split()
    new_words = [] 
    for word in words:
        if word.lower() in abbr_dict:
            word = abbr_dict[word.lower()]
        new_words.append(word)
    new_text = " ".join(new_words) # ..... change here
    return new_text # ..... change here also
df['new_text'] = df['text'].apply(_lookup_words)

使用字典替换数据框中的互联网首字母缩略词

To replace internet acronyms in a dataframe using dictionary

python

text-mining

pandas