pandas get_close_matches - 返回空值
pandas get_close_matches - returning empty value
我正在处理一个需求,有 2 个 CSV 如下 -
CSV1.csv
Short Description Category
Device is DOWN! Server Down
CPU Warning Monitoron XSSXSXSXSXSX.com CPU Utilization
CPU Warning Monitoron XSSXSXSXSXSX.com CPU Utilization
CPU Warning Monitoron XSSXSXSXSXSX.com CPU Utilization
CPU Warning Monitoron XSSXSXSXSXSX.com CPU Utilization
Device Performance Alerts was triggered on Physical memory Memory Utilization
Device Performance Alerts was triggered on Physical memory Memory Utilization
Device Performance Alerts was triggered on Physical memory Memory Utilization
Disk Space Is Lowon ;E: Disk Space Utilization
Disk Space Is Lowon;C: Disk Space Utilization
Network Interface Down Interface Down
Active Directory
和reference.csv
Category Complexity
Server Down Simple
Network Interface down Complex
Drive Cleanup Windows Medium
CPU Utilization Medium
Memory Utilization Medium
Disk Space Utilization Unix Simple
Windows Service Restart Medium
UNIX Service Restart Medium
Web Tomcat Instance Restart Simple
Expected Output
Short Description Category Complexity
Device is DOWN! Server Down Simple
CPU Warning Monitoron XSSXSXSXSXSX.com CPU Utilization Medium
CPU Warning Monitoron XSSXSXSXSXSX.com CPU Utilization Medium
CPU Warning Monitoron XSSXSXSXSXSX.com CPU Utilization Medium
CPU Warning Monitoron XSSXSXSXSXSX.com CPU Utilization Medium
Device Performance Alerts was triggered on Physical memory Memory Utilization Medium
Device Performance Alerts was triggered on Physical memory Memory Utilization Medium
Device Performance Alerts was triggered on Physical memory Memory Utilization Medium
Disk Space Is Lowon ;E: Disk Space Utilization Medium
Disk Space Is Lowon;C: Disk Space Utilization Medium
Network Interface Down Interface Down Complex
我试过下面的代码 - 但在输出数据框中我可以看到空白 []
不确定我错过了什么。在输出复杂性列中,我只能看到每一行的 []。我试图获得完全匹配,但我需要获得所有可能的组合,所以我使用 get_close_matches。如何在下面的代码中传递数据框中的可能性参数,我不知道传递可能性的方法。
我尝试了一些其他的可能性,比如 exact 但没有给出预期的结果,因为我正在寻找所有可能的组合,同时将列与字符串进行比较
import pandas as pd
import difflib
df1 = pd.read_csv('csv1.csv')
df1 = df1.fillna('')
df2 = pd.read_csv('reference.csv')
my_dict = dict(zip(df2['Category'].values, df2['Complexity'].values))
def match_key(key, default_value):
if not key:
return default_value
for d_key in my_dict.keys():
if key in d_key or d_key in key:
return my_dict[d_key]
return default_value
df1['Complexity'] = df1['Category'].apply(lambda x: difflib.get_close_matches(x, list(my_dict.keys(), n=1)))
df1 = df1.explode('Complexity')
df1['Complexity'] = df1['Complexity'].map(my_dict)
print(df1)
difflib.get_close_matches
期望第一个参数是 'word',在你的例子中是 x
,第二个参数是 'possibilities'。您提供的是空字符串。这就是为什么你的函数不起作用,它试图匹配一个基本上什么都没有的词。
my_dict
包含有效选项作为键,因此我们可以将它们用作 'possibilities'
的列表
# Use n=1, so only tries to get 1 match
df1['Complexity'] = df1['Category'].apply(lambda x: difflib.get_close_matches(x, list(my_dict.keys()), n=1))
# The output of get_close_matches is a list, we use explode to convert it to a string
df1 = df1.explode('Complexity')
# We can now apply our map, to the *Complexity* column,
# which is technically the best match *Category*, via get_close_matches
df1['Complexity'] = df1['Complexity'].map(my_dict)
原始错误答案
但是,与其继续使用 difflib
,我认为您可以改变您的方法。您想要将 my_dict
应用于 df1
的 Category
列。这在传统上被称为应用 map
。 pandas
通过 pandas.Series.map
准备好此实施。
df1['Complexity'] = df1['Category'].map(my_dict)
我正在处理一个需求,有 2 个 CSV 如下 -
CSV1.csv
Short Description Category
Device is DOWN! Server Down
CPU Warning Monitoron XSSXSXSXSXSX.com CPU Utilization
CPU Warning Monitoron XSSXSXSXSXSX.com CPU Utilization
CPU Warning Monitoron XSSXSXSXSXSX.com CPU Utilization
CPU Warning Monitoron XSSXSXSXSXSX.com CPU Utilization
Device Performance Alerts was triggered on Physical memory Memory Utilization
Device Performance Alerts was triggered on Physical memory Memory Utilization
Device Performance Alerts was triggered on Physical memory Memory Utilization
Disk Space Is Lowon ;E: Disk Space Utilization
Disk Space Is Lowon;C: Disk Space Utilization
Network Interface Down Interface Down
Active Directory
和reference.csv
Category Complexity
Server Down Simple
Network Interface down Complex
Drive Cleanup Windows Medium
CPU Utilization Medium
Memory Utilization Medium
Disk Space Utilization Unix Simple
Windows Service Restart Medium
UNIX Service Restart Medium
Web Tomcat Instance Restart Simple
Expected Output
Short Description Category Complexity
Device is DOWN! Server Down Simple
CPU Warning Monitoron XSSXSXSXSXSX.com CPU Utilization Medium
CPU Warning Monitoron XSSXSXSXSXSX.com CPU Utilization Medium
CPU Warning Monitoron XSSXSXSXSXSX.com CPU Utilization Medium
CPU Warning Monitoron XSSXSXSXSXSX.com CPU Utilization Medium
Device Performance Alerts was triggered on Physical memory Memory Utilization Medium
Device Performance Alerts was triggered on Physical memory Memory Utilization Medium
Device Performance Alerts was triggered on Physical memory Memory Utilization Medium
Disk Space Is Lowon ;E: Disk Space Utilization Medium
Disk Space Is Lowon;C: Disk Space Utilization Medium
Network Interface Down Interface Down Complex
我试过下面的代码 - 但在输出数据框中我可以看到空白 []
不确定我错过了什么。在输出复杂性列中,我只能看到每一行的 []。我试图获得完全匹配,但我需要获得所有可能的组合,所以我使用 get_close_matches。如何在下面的代码中传递数据框中的可能性参数,我不知道传递可能性的方法。
我尝试了一些其他的可能性,比如 exact 但没有给出预期的结果,因为我正在寻找所有可能的组合,同时将列与字符串进行比较
import pandas as pd
import difflib
df1 = pd.read_csv('csv1.csv')
df1 = df1.fillna('')
df2 = pd.read_csv('reference.csv')
my_dict = dict(zip(df2['Category'].values, df2['Complexity'].values))
def match_key(key, default_value):
if not key:
return default_value
for d_key in my_dict.keys():
if key in d_key or d_key in key:
return my_dict[d_key]
return default_value
df1['Complexity'] = df1['Category'].apply(lambda x: difflib.get_close_matches(x, list(my_dict.keys(), n=1)))
df1 = df1.explode('Complexity')
df1['Complexity'] = df1['Complexity'].map(my_dict)
print(df1)
difflib.get_close_matches
期望第一个参数是 'word',在你的例子中是 x
,第二个参数是 'possibilities'。您提供的是空字符串。这就是为什么你的函数不起作用,它试图匹配一个基本上什么都没有的词。
my_dict
包含有效选项作为键,因此我们可以将它们用作 'possibilities'
# Use n=1, so only tries to get 1 match
df1['Complexity'] = df1['Category'].apply(lambda x: difflib.get_close_matches(x, list(my_dict.keys()), n=1))
# The output of get_close_matches is a list, we use explode to convert it to a string
df1 = df1.explode('Complexity')
# We can now apply our map, to the *Complexity* column,
# which is technically the best match *Category*, via get_close_matches
df1['Complexity'] = df1['Complexity'].map(my_dict)
原始错误答案
但是,与其继续使用 difflib
,我认为您可以改变您的方法。您想要将 my_dict
应用于 df1
的 Category
列。这在传统上被称为应用 map
。 pandas
通过 pandas.Series.map
准备好此实施。
df1['Complexity'] = df1['Category'].map(my_dict)