转换预测目标

Question

我有一个数据集，其中每个观察值可能属于不同的标签（多标签 class化）。

我已经对其及其工作进行了 SVM class化。（这里我想知道每个 class 的准确性，所以我应用 OneVsRestClassifier 每个 class，正如您将在代码中看到的那样。）

我想查看测试数据中每个项目的预测值。换句话说，我想看看模型在测试样本中的每个观察结果预测了哪个标签。

例如：这是传递给模型进行预测的数据

,sentences,ADR,WD,EF,INF,SSI,DI,others
0,"extreme weight gain, short-term memory loss, hair loss.",1,0,0,0,0,0,0
1,I am detoxing from Lexapro now.,0,0,0,0,0,0,1
2,I slowly cut my dosage over several months and took vitamin supplements to help.,0,0,0,0,0,0,1
3,I am now 10 days completely off and OMG is it rough.,0,0,0,0,0,0,1
4,"I have flu-like symptoms, dizziness, major mood swings, lots of anxiety, tiredness.",0,1,0,0,0,0,1
5,I have no idea when this will end.,1,0,0,0,0,0,1

然后我的模型预测了这些行的标签，我想查看每一行的预测映射。

我知道我们可以使用 scikit-learn 库中的 Label Binarization 来做到这一点。

问题是 fit_transform 解释的 here 的输入参数与我准备并传递给 SVM classification 的目标数据不同。所以我不知道怎么弄。

这是我的代码：

df = pd.read_csv("finalupdatedothers.csv")
categories = ['ADR','WD','EF','INF','SSI','DI','others']

train,test = train_test_split(df,random_state=42,test_size=0.3,shuffle=True)
X_train = train.sentences
X_test = test.sentences

SVC_pipeline = Pipeline([
                ('tfidf', TfidfVectorizer(stop_words=stop_words)),
                ('clf', OneVsRestClassifier(LinearSVC(), n_jobs=1)),
            ])

for category in categories:
    print('... Processing {} '.format(category))
    SVC_pipeline.fit(X_train,train[category]
    prediction = SVC_pipeline.predict(X_test)
    print('SVM Linear Test accuracy is {} '.format(accuracy_score(test[category], prediction)))
    print 'SVM Linear f1 measurement is {} '.format(f1_score(test[category], prediction, average='weighted'))
    print "\n"

非常感谢你的宝贵时间。

Answer 1

这就是您想要的，我刚刚所做的是，我映射了 prediction，它是一个代表 categories 列表中的 class 标签索引的 numpy 数组。所以这是完整的代码。

import pandas as pd
import numpy as np
from sklearn import svm
from sklearn.datasets import samples_generator
from sklearn.feature_selection import SelectKBest
from sklearn.feature_selection import f_regression
from sklearn.pipeline import Pipeline
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.multiclass import OneVsRestClassifier

from sklearn.svm import LinearSVC
from sklearn.metrics import accuracy_score
from sklearn.metrics import f1_score
df = pd.read_csv("finalupdatedothers.csv")
categories = ['ADR','WD','EF','INF','SSI','DI','others']

train,test = train_test_split(df,random_state=42,test_size=0.3,shuffle=True)
X_train = train.sentences
X_test = test.sentences

SVC_pipeline = Pipeline([
                ('tfidf', TfidfVectorizer(stop_words=[])),
                ('clf', OneVsRestClassifier(LinearSVC(), n_jobs=1)),
            ])


for category in categories:
    print('... Processing {} '.format(category))
    SVC_pipeline.fit(X_train,train[category])
    prediction = SVC_pipeline.predict(X_test)
    print([{X_test.iloc[i]:categories[prediction[i]]} for i in range(len(list(prediction)))  ])

    print('SVM Linear Test accuracy is {} '.format(accuracy_score(test[category], prediction)))
    print ('SVM Linear f1 measurement is {} '.format(f1_score(test[category], prediction, average='weighted')))
    print ("\n")

这是示例输出：

... Processing ADR 
[{'extreme weight gain, short-term memory loss, hair loss.': 'ADR'}, {'I am detoxing from Lexapro now.': 'ADR'}]
SVM Linear Test accuracy is 0.5 
SVM Linear f1 measurement is 0.3333333333333333 


... Processing WD 
[{'extreme weight gain, short-term memory loss, hair loss.': 'ADR'}, {'I am detoxing from Lexapro now.': 'ADR'}]
SVM Linear Test accuracy is 1.0 
SVM Linear f1 measurement is 1.0

希望对您有所帮助。

转换预测目标

Transforming the prediction target

python

machine-learning

scikit-learn

multilabel-classification