KNN算法的准确率很低

Question

我正在关注 senddex 的 youtube 频道 ML 教程。

因此，当我编写有关如何构建您自己的 KNN 算法的代码时，我注意到我的准确性非常低，几乎每次都在 60 年代左右。我做了一些更改，但后来我逐行使用他的代码和相同的数据集，但不知何故他的准确度在 95-98% 的范围内，而我的准确率为 60-70%。实在想不通为什么会有这么大的差异。

我还有第二个问题，与预测的可信度有关。置信度的值应该在 0-1 之间，对吗？但对我来说，它们都是一样的，而且都是在 70 年代。让我用截图来解释

我的代码：

# Importing libraries
import numpy as np
import pandas as pd
from collections import Counter
import warnings
import random

# Algorithm

def k_nearest(data,predict,k=5):
    if len(data)>=k:
        warnings.warn("stupid, your data has more dimensions than prescribed")
    distances = []
    for group in data: # The groups of 2s and 4s
        for features in data[group]: # values in 2 and 4 respectively
            #euclidean_distance = np.sqrt(np.sum((np.array(features) - np.sum(predict)) **2))
            euclidean_distance = np.linalg.norm(np.array(features) - np.array(predict))
            distances.append([euclidean_distance,group])
    votes = [i[1] for i in sorted(distances)] # adding the sorted(ascending) group names
    votes_result = Counter(votes).most_common(1)[0][0] # the most common element
    confidence = float((Counter(votes).most_common(1)[0][1]))/float(k)#ocuurences of the most common element
    
    return votes_result,confidence
#reading the data
df = pd.read_csv("breast_cancer.txt")
df.replace("?",-99999,inplace=True)
#df.replace("?", np.nan,inplace=True)
#df.dropna(inplace=True)
df.drop("id",axis = 1,inplace=True)
full_data = df.astype(float).values.tolist() # Converting to list because our function is written like that
random.shuffle(full_data)
#print(full_data[:10])

test_size = 0.2
train_set = {2:[],4:[]}
test_set = {2:[],4:[]}
train_data = full_data[:-int(test_size*len(full_data))] # Upto the last 20% of the og dateset
test_data = full_data[-int(test_size*len(full_data)):] # The last 20% of the dataset

# Populating the dictionary
for i in train_data:
    train_set[i[-1]].append(i[:-1]) # appending with features and leaving out the label
for i in test_data:
    test_set[i[-1]].append(i[:-1]) # appending with features and leaving out the label

# Testing 
correct,total = 0,0
for group in test_set:
    for data in test_set[group]:
        vote,confidence = k_nearest(train_set, data,k=5)
        if vote == group:
            correct +=1
        else:
            print(confidence)
        total += 1 
print("Accuracy is",correct/total)

Link 到数据集 breast-cancer-wisconsin.data

Answer 1

你的 k_nearest 函数有一个错误，你需要 return 只需要前 k 个距离，而不是整个列表。所以应该是：

votes = [i[1] for i in sorted(distances)[:k]]

而不是在您的代码中：

votes = [i[1] for i in sorted(distances)]

我们可以重写你的函数：

def k_nearest(data,predict,k=5):
    distances = []
    for group in data: 
        for features in data[group]: 
            euclidean_distance = np.linalg.norm(np.array(features) - np.array(predict))
            distances.append([euclidean_distance,group])
    votes = [i[1] for i in sorted(distances)[:k]] 
    votes_result = Counter(votes).most_common(1)[0][0] 
    confidence = float((Counter(votes).most_common(1)[0][1]))/float(k)

return votes_result,confidence

还有运行你的代码，我不太确定要不要替换“？”与 -999 所以我读为 na :

import pandas as pd
from collections import Counter
import random
import numpy as np

url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/breast-cancer-wisconsin.data'
df = pd.read_csv(url,header=None,na_values="?")
df = df.dropna()
full_data = df.iloc[:,1:].astype(float).values.tolist() 
random.seed(999)
random.shuffle(full_data)

test_size = 0.2
train_set = {2:[],4:[]}
test_set = {2:[],4:[]}
train_data = full_data[:-int(test_size*len(full_data))] 
test_data = full_data[-int(test_size*len(full_data)):] 

for i in train_data:
    train_set[i[-1]].append(i[:-1]) 
for i in test_data:
    test_set[i[-1]].append(i[:-1]) 

correct,total = 0,0
for group in test_set:
    for data in test_set[group]:
        vote,confidence = k_nearest(train_set, data,k=5)
        if vote == group:
            correct +=1
        else:
            print(confidence)
        total += 1 
print("Accuracy is",correct/total)

给出：

1.0
0.8
1.0
0.6
0.6
0.6
0.6
Accuracy is 0.9485294117647058

KNN算法的准确率很低

Accuracy of KNN algorithm very low

python

machine-learning

knn