如何使用 scikit-learn 获取带有集群组的新 pandas 数据框列
How to get a new pandas data frame column with cluster groups using scikit-learn
我是 scikit-learn 的新手。我有一个 pandas DataFrame，其中 1 列包含我想要聚类的文本。
作为最终结果，我想让我的数据框为每一行显示一个额外的列，其中包含它所属的集群组。
我的 DF 看起来像这样：
event_date,event_desc,event_url
2020-05-28 07:03:00,Roche’s OCREVUS (ocrelizumab) shorter 2-hour infusion time approved in Europe,https://www.roche.com/investors/updates/inv-update-2020-05-28b.htm
2020-05-28 07:00:00,Roche initiates phase III clinical trial of Actemra/RoActemra plus remdesivir in hospitalised patients with severe COVID-19 pneumonia,https://www.roche.com/investors/updates/inv-update-2020-05-28.htm
2020-05-27 07:00:00,Roche’s Port Delivery System with ranibizumab shows positive phase III results in neovascular age-related macular degeneration,https://www.roche.com/investors/updates/inv-update-2020-05-27.htm
2020-05-25 15:00:00,Reminder: Invitation to Roche’s virtual event on key oncology data presented at ASCO 2020,https://www.roche.com/investors/updates/inv-update-2020-05-25.htm
2020-05-22 07:03:00,Roche acquires Stratos Genomics to further develop DNA based sequencing for diagnostic use,https://www.roche.com/investors/updates/inv-update-2020-05-22b.htm
2020-05-22 07:00:00,New longer-term data reinforce safety of Roche’s satralizumab in adults and adolescents with neuromyelitis optica spectrum disorder,https://www.roche.com/investors/updates/inv-update-2020-05-22.htm
2020-05-19 07:00:00,FDA approves Roche’s Tecentriq as a first-line monotherapy for certain people with metastatic non-small cell lung cancer,https://www.roche.com/investors/updates/inv-update-2020-05-19.htm
2020-05-15 07:00:00,Roche launches new blood gas digital solution designed to improve patient care,https://www.roche.com/investors/updates/inv-update-2020-05-15.htm
2020-05-14 07:00:00,Roche to present first clinical data on novel anti-TIGIT cancer immunotherapy tiragolumab at ASCO,https://www.roche.com/investors/updates/inv-update-2020-05-14.htm
2020-05-11 14:15:00,A formative figure in the company's history: Roche Honorary Chairman Fritz Gerber dies at the age of 91,https://www.roche.com/investors/updates/inv-update-2020-05-11c.htm
2020-05-11 07:00:00,Changes to the Roche Enlarged Corporate Executive Committee,https://www.roche.com/investors/updates/inv-update-2020-05-11.htm
2020-05-07 07:00:00,New data at the ASCO20 Virtual Scientific Program reflects Roche’s commitment to accelerating progress in cancer care,https://www.roche.com/investors/updates/inv-update-2020-05-07.htm
2020-05-06 15:00:00,Invitation to Roche’s virtual event on key oncology data presented at ASCO 2020,https://www.roche.com/investors/updates/inv-update-2020-05-06.htm
2020-05-03 04:15:00,Roche’s COVID-19 antibody test receives FDA Emergency Use Authorization and is available in markets accepting the CE mark,https://www.roche.com/investors/updates/inv-update-2020-05-03.htm
2020-04-30 14:00:00,Reminder: Invitation to Roche’s Virtual Event “Digital technology and advanced analytics in Roche”,https://www.roche.com/investors/updates/inv-update-2020-04-30.htm
2020-04-28 07:03:00,New 6-year data for Roche’s OCREVUS (ocrelizumab) show earlier treatment initiation nearly halves risk of needing walking aid in relapsing multiple sclerosis,https://www.roche.com/investors/updates/inv-update-2020-04-28b.htm
2020-04-28 07:00:00,Roche’s risdiplam shows significant improvement in survival and motor milestones in infants with Type 1 spinal muscular atrophy (SMA),https://www.roche.com/investors/updates/inv-update-2020-04-28.htm
2020-04-24 17:35:00,Positive Results from the Phase III SAkuraStar Study for Satralizumab in NMOSD Published in The Lancet Neurology,https://www.roche.com/investors/updates/inv-update-2020-04-24.htm
2020-04-22 07:00:00,"First quarter with 2% growth in Swiss francs, 7% at constant exchange rates",https://www.roche.com/investors/updates/inv-update-2020-04-22.htm
2020-04-21 14:00:00,Reminder: Invitation to Roche’s live audio webcast on new AAN 2020 data ,https://www.roche.com/investors/updates/inv-update-2020-04-21b.htm
2020-04-21 07:00:00,Roche receives FDA approval for cobas HPV test for use on the cobas 6800/8800 Systems to identify women at risk for cervical cancer,https://www.roche.com/investors/updates/inv-update-2020-04-21.htm
2020-04-20 07:00:00,US FDA and EMA accept applications for Roche’s OCREVUS (ocrelizumab) shorter 2-hour infusion time,https://www.roche.com/investors/updates/inv-update-2020-04-20.htm
2020-04-17 07:00:00,Roche develops new serology test to detect COVID-19 antibodies,https://www.roche.com/investors/updates/inv-update-2020-04-17.htm
2020-04-15 10:00:00,Reminder: Invitation to Roche’s First Quarter Sales 2020 Audio Webcast and Conference Call,https://www.roche.com/investors/updates/inv-update-2020-04-15b.htm
I 运行 与列 event_desc
的聚类
到目前为止，我使用 useful link 中的以下代码：
import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.cluster import KMeans
my_csv_file_name = 'temp.csv'
result_df = pd.read_csv(csv_file_name, sep=',', parse_dates=['event_date'],
dtype={'event_desc': pd.StringDtype(), 'event_url': pd.StringDtype()})
vectorizer = TfidfVectorizer(stop_words='english')
X = vectorizer.fit_transform(result_df['event_desc'])
true_k = 10
model = KMeans(n_clusters=true_k, init='k-means++', max_iter=100, n_init=1)
model.fit(X)
print("Top terms per cluster:")
order_centroids = model.cluster_centers_.argsort()[:, ::-1]
terms = vectorizer.get_feature_names()
for i in range(true_k):
print("Cluster %d:" % i),
for ind in order_centroids[i, :10]:
print(' %s' % terms[ind]),
print("Prediction")
Y = vectorizer.transform(["chrome browser to open."])
prediction = model.predict(Y)
print(prediction)
它按集群打印“分组”关键字，但我想将每个集群与其自己的行相关联：这意味着初始数据帧将有一个额外的列cluster_group，其中数字介于 0 到 9 之间，对应于所属的cluster group（我选了10组——实际的csv有3000多行）。知道怎么做吗？
非常感谢。最佳
集群组存储在 model 对象的 labels_ 属性中，即 model.labels_ returns 一个数组，其中每个元素包含每一行的集群组在你的数据框中。
import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.cluster import KMeans
my_csv_file_name = 'temp.csv'
result_df = pd.read_csv(my_csv_file_name, sep=',', parse_dates=['event_date'],
dtype={'event_desc': pd.StringDtype(), 'event_url': pd.StringDtype()})
vectorizer = TfidfVectorizer(stop_words='english')
X = vectorizer.fit_transform(result_df['event_desc'])
true_k = 10
model = KMeans(n_clusters=true_k, init='k-means++', max_iter=100, n_init=1)
model.fit(X)
# extract the cluster groups
result_df['cluster_group'] = model.labels_
print(result_df['cluster_group'].sort_values().unique())
[0 1 2 3 4 5 6 7 8 9]
我是 scikit-learn 的新手。我有一个 pandas DataFrame，其中 1 列包含我想要聚类的文本。 作为最终结果，我想让我的数据框为每一行显示一个额外的列，其中包含它所属的集群组。 我的 DF 看起来像这样：
event_date,event_desc,event_url
2020-05-28 07:03:00,Roche’s OCREVUS (ocrelizumab) shorter 2-hour infusion time approved in Europe,https://www.roche.com/investors/updates/inv-update-2020-05-28b.htm
2020-05-28 07:00:00,Roche initiates phase III clinical trial of Actemra/RoActemra plus remdesivir in hospitalised patients with severe COVID-19 pneumonia,https://www.roche.com/investors/updates/inv-update-2020-05-28.htm
2020-05-27 07:00:00,Roche’s Port Delivery System with ranibizumab shows positive phase III results in neovascular age-related macular degeneration,https://www.roche.com/investors/updates/inv-update-2020-05-27.htm
2020-05-25 15:00:00,Reminder: Invitation to Roche’s virtual event on key oncology data presented at ASCO 2020,https://www.roche.com/investors/updates/inv-update-2020-05-25.htm
2020-05-22 07:03:00,Roche acquires Stratos Genomics to further develop DNA based sequencing for diagnostic use,https://www.roche.com/investors/updates/inv-update-2020-05-22b.htm
2020-05-22 07:00:00,New longer-term data reinforce safety of Roche’s satralizumab in adults and adolescents with neuromyelitis optica spectrum disorder,https://www.roche.com/investors/updates/inv-update-2020-05-22.htm
2020-05-19 07:00:00,FDA approves Roche’s Tecentriq as a first-line monotherapy for certain people with metastatic non-small cell lung cancer,https://www.roche.com/investors/updates/inv-update-2020-05-19.htm
2020-05-15 07:00:00,Roche launches new blood gas digital solution designed to improve patient care,https://www.roche.com/investors/updates/inv-update-2020-05-15.htm
2020-05-14 07:00:00,Roche to present first clinical data on novel anti-TIGIT cancer immunotherapy tiragolumab at ASCO,https://www.roche.com/investors/updates/inv-update-2020-05-14.htm
2020-05-11 14:15:00,A formative figure in the company's history: Roche Honorary Chairman Fritz Gerber dies at the age of 91,https://www.roche.com/investors/updates/inv-update-2020-05-11c.htm
2020-05-11 07:00:00,Changes to the Roche Enlarged Corporate Executive Committee,https://www.roche.com/investors/updates/inv-update-2020-05-11.htm
2020-05-07 07:00:00,New data at the ASCO20 Virtual Scientific Program reflects Roche’s commitment to accelerating progress in cancer care,https://www.roche.com/investors/updates/inv-update-2020-05-07.htm
2020-05-06 15:00:00,Invitation to Roche’s virtual event on key oncology data presented at ASCO 2020,https://www.roche.com/investors/updates/inv-update-2020-05-06.htm
2020-05-03 04:15:00,Roche’s COVID-19 antibody test receives FDA Emergency Use Authorization and is available in markets accepting the CE mark,https://www.roche.com/investors/updates/inv-update-2020-05-03.htm
2020-04-30 14:00:00,Reminder: Invitation to Roche’s Virtual Event “Digital technology and advanced analytics in Roche”,https://www.roche.com/investors/updates/inv-update-2020-04-30.htm
2020-04-28 07:03:00,New 6-year data for Roche’s OCREVUS (ocrelizumab) show earlier treatment initiation nearly halves risk of needing walking aid in relapsing multiple sclerosis,https://www.roche.com/investors/updates/inv-update-2020-04-28b.htm
2020-04-28 07:00:00,Roche’s risdiplam shows significant improvement in survival and motor milestones in infants with Type 1 spinal muscular atrophy (SMA),https://www.roche.com/investors/updates/inv-update-2020-04-28.htm
2020-04-24 17:35:00,Positive Results from the Phase III SAkuraStar Study for Satralizumab in NMOSD Published in The Lancet Neurology,https://www.roche.com/investors/updates/inv-update-2020-04-24.htm
2020-04-22 07:00:00,"First quarter with 2% growth in Swiss francs, 7% at constant exchange rates",https://www.roche.com/investors/updates/inv-update-2020-04-22.htm
2020-04-21 14:00:00,Reminder: Invitation to Roche’s live audio webcast on new AAN 2020 data ,https://www.roche.com/investors/updates/inv-update-2020-04-21b.htm
2020-04-21 07:00:00,Roche receives FDA approval for cobas HPV test for use on the cobas 6800/8800 Systems to identify women at risk for cervical cancer,https://www.roche.com/investors/updates/inv-update-2020-04-21.htm
2020-04-20 07:00:00,US FDA and EMA accept applications for Roche’s OCREVUS (ocrelizumab) shorter 2-hour infusion time,https://www.roche.com/investors/updates/inv-update-2020-04-20.htm
2020-04-17 07:00:00,Roche develops new serology test to detect COVID-19 antibodies,https://www.roche.com/investors/updates/inv-update-2020-04-17.htm
2020-04-15 10:00:00,Reminder: Invitation to Roche’s First Quarter Sales 2020 Audio Webcast and Conference Call,https://www.roche.com/investors/updates/inv-update-2020-04-15b.htm
I 运行 与列 event_desc
到目前为止，我使用 useful link 中的以下代码：
import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.cluster import KMeans
my_csv_file_name = 'temp.csv'
result_df = pd.read_csv(csv_file_name, sep=',', parse_dates=['event_date'],
dtype={'event_desc': pd.StringDtype(), 'event_url': pd.StringDtype()})
vectorizer = TfidfVectorizer(stop_words='english')
X = vectorizer.fit_transform(result_df['event_desc'])
true_k = 10
model = KMeans(n_clusters=true_k, init='k-means++', max_iter=100, n_init=1)
model.fit(X)
print("Top terms per cluster:")
order_centroids = model.cluster_centers_.argsort()[:, ::-1]
terms = vectorizer.get_feature_names()
for i in range(true_k):
print("Cluster %d:" % i),
for ind in order_centroids[i, :10]:
print(' %s' % terms[ind]),
print("Prediction")
Y = vectorizer.transform(["chrome browser to open."])
prediction = model.predict(Y)
print(prediction)
它按集群打印“分组”关键字，但我想将每个集群与其自己的行相关联：这意味着初始数据帧将有一个额外的列cluster_group，其中数字介于 0 到 9 之间，对应于所属的cluster group（我选了10组——实际的csv有3000多行）。知道怎么做吗？
非常感谢。最佳
集群组存储在 model 对象的 labels_ 属性中，即 model.labels_ returns 一个数组，其中每个元素包含每一行的集群组在你的数据框中。
import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.cluster import KMeans
my_csv_file_name = 'temp.csv'
result_df = pd.read_csv(my_csv_file_name, sep=',', parse_dates=['event_date'],
dtype={'event_desc': pd.StringDtype(), 'event_url': pd.StringDtype()})
vectorizer = TfidfVectorizer(stop_words='english')
X = vectorizer.fit_transform(result_df['event_desc'])
true_k = 10
model = KMeans(n_clusters=true_k, init='k-means++', max_iter=100, n_init=1)
model.fit(X)
# extract the cluster groups
result_df['cluster_group'] = model.labels_
print(result_df['cluster_group'].sort_values().unique())
[0 1 2 3 4 5 6 7 8 9]