从分类器的准确度结果计算标准误差？

Question

下面的代码打印了 10 倍以上的准确度分数，如下所示

from sklearn.datasets import load_digits, load_iris, load_breast_cancer, load_wine
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
from sklearn.utils import shuffle
from sklearn import preprocessing
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import StratifiedKFold
from sklearn.metrics import accuracy_score, zero_one_loss, confusion_matrix
import pandas as pd
import numpy as np


z = pd.read_csv('/home/user/datasets/iris_dataset.csv', header=0)

X = z.iloc[:, :-1]
y = z.iloc[:, -1:]

X = np.array(X)
y = np.array(y)

# Performing standard scaling
scaler = preprocessing.MinMaxScaler()
X_scaled = scaler.fit_transform(X)

# Defining the SVM with 'rbf' kernel
svc = SVC(kernel='rbf',random_state=50)

#X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.10, shuffle=True)

skf = StratifiedKFold(n_splits=10, shuffle=True)
acc_score = []
#skf.get_n_splits(X, y)

for train_index, test_index in skf.split(X, y):
    X_train, X_test = X_scaled[train_index], X_scaled[test_index]
    y_train, y_test = y[train_index], y[test_index]

    # Training the model
    svc.fit(X_train, np.ravel(y_train))

    # Prediction on test dataste
    y_pred = svc.predict(X_test)

    # Obtaining the accuracy scores of the model
    score = accuracy_score(y_test, y_pred)
    acc_score.append(score)
print(acc_score)

代码 (acc_score) 的输出如下所示：

[1.0, 0.9333333333333333, 1.0, 0.9333333333333333, 0.9333333333333333, 1.0, 0.9333333333333333, 0.9333333333333333, 1.0, 1.0]

据此，我如何使用 Python 中的 NumPy 和 sklearn 计算这些准确度结果的 standard error 和 average accuracy？我希望打印这些准确度分数的标准误差以及平均准确度

Answer 1

计算平均值：

print ('ACC mean:', '{0:0.2f}'.format(np.mean(acc_score)))

计算标准误差：

print ('ACC std:', '{0:0.2f}'.format(np.std(acc_score)))

Answer 2

要计算平均值的标准误差（或测量的标准误差），可以使用 scipy：

import scipy
from scipy import stats
acc_score = [1.0, 0.9333333333333333, 1.0, 0.9333333333333333, 0.9333333333333333, 1.0, 0.9333333333333333, 0.9333333333333333, 1.0, 1.0]
print('ACC std:', '{0:0.2f}'.format(scipy.stats.sem(acc_score)))

从分类器的准确度结果计算标准误差？

Calculating standard errors from accuracy results of a classifier?

python-3.x

numpy

scikit-learn

standard-error