如何在情感分析中添加混淆矩阵和k-fold 10 fold

Question

我想使用交叉验证和混淆矩阵 k-fold (k = 10) 方法添加评估模型，但我很困惑数据集：https://github.com/fadholifh/dats/blob/master/cpas.txt

使用 Pyhon 3.7

import sklearn.metrics
import sen
import csv
import os
import re
import nltk
import scipy
import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn import svm
from sklearn.externals import joblib
from sklearn.pipeline import Pipeline
from sklearn import model_selection
from sklearn.model_selection import train_test_split
from sklearn.model_selection import KFold
from sklearn.metrics import confusion_matrix
from Sastrawi.Stemmer.StemmerFactory import StemmerFactory
from Sastrawi.StopWordRemover.StopWordRemoverFactory import StopWordRemoverFactory
factorys = StemmerFactory()
stemmer = factorys.create_stemmer()





if __name__ == "__main__":
    main()

结果是混淆矩阵，对于 k 折，每个折都有 F1 分数、精度和召回率的百分比

Answer 1

df = pd.read_csv("cpas.txt", header=None, delimiter="\t")
X = df[1].values
y = df[0].values

stop_words = stopwords.words('english')
stemmer = PorterStemmer()

def clean_text(text, stop_words, stemmer):
    return " ".join([stemmer.stem(word) for word in word_tokenize(text) 
                    if word not in stop_words and not word.isnumeric()])

X = np.array([clean_text(text, stop_words, stemmer) for text in X])

kfold = KFold(3, shuffle=True, random_state=33)
i = 1
for train_idx, test_idx in kfold.split(X):
    X_train = X[train_idx]
    y_train = y[train_idx]

    X_test = X[test_idx]
    y_test = y[test_idx]

    vectorizer = TfidfVectorizer()
    X_train = vectorizer.fit_transform(X_train)
    X_test = vectorizer.transform(X_test)

    model = LinearSVC()
    model.fit(X_train, y_train)
    print ("Fold : {0}".format(i))
    i += 1
    print (classification_report(y_test, model.predict(X_test)))

你使用交叉验证的原因是在数据较少时进行参数调优。可以使用带有 CV 的网格搜索来执行此操作。

df = pd.read_csv("cpas.txt", header=None, delimiter="\t")
X = df[1].values
labels = df[0].values

text = np.array([clean_text(text, stop_words, stemmer) for text in X])
idx = np.arange(len(text))
np.random.shuffle(idx)

text = text[idx]
labels = labels[idx]

pipeline = Pipeline([
        ('vectorizer', TfidfVectorizer()),
        ('svm', LinearSVC())])

params = {
    'vectorizer__ngram_range' : [(1,1),(1,2),(2,2)],
    'vectorizer__lowercase' : [True, False],
    'vectorizer__norm' : ['l1','l2']}

model = GridSearchCV(pipeline, params, cv=3, verbose=1)
model.fit(text, y)

如何在情感分析中添加混淆矩阵和k-fold 10 fold

how to add confusion matrix and k-fold 10 fold in sentiment analysis

python

sentiment-analysis

confusion-matrix

scikit-learn

cross-validation