我如何在 .csv 文件的数据集上训练我的 CNN 模型？

Question

大家好，我是 Python 初学者，我正在使用 google Colab 训练我的第一个 CNN 模型，我在训练部分受阻，我知道我必须使用model.fit() 来训练模型，但我不知道 model.fit() 里面有什么，而且我不确定从 .csv 文件中拆分我的数据集的部分。这是我的代码：

from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras.utils import to_categorical
from keras.preprocessing import image
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from keras.utils import to_categorical
from sklearn.metrics import accuracy_score
from tqdm import tqdm
from numpy.random import RandomState
import csv

df = pd.read_csv('classes.csv')
print(df.head(3500))
#splitting the dataset in training and testing sets.
rng = RandomState()

train = df.sample(frac=0.7, random_state=rng)
test = df.loc[~df.index.isin(train.index)]



#model's structure

model = Sequential()
#convolutional layer
model.add(Conv2D(32, kernel_size=(3, 3),activation='relu',input_shape=(28,28,1)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
#flatten output of conv
model.add(Flatten())
#hidden layer
model.add(Dense(128, activation='relu')) 
#output layer
model.add(Dense(10, activation='softmax')) 
#compiling sequential model
model.compile(loss='categorical_crossentropy',optimizer='Adam',metrics=['accuracy'])
model.summary()
#training the model
model.fit()

Answer 1

好的，如果我们能提供帮助，让我们开始吧。第一件事是拆分数据框。最好将其拆分为训练集、验证集和测试集。最好的方法是使用 sklearn 的训练、测试、拆分。相关文档在 [此处。][1] 使用下面的代码将数据 trame 分成 3 组

from sklearn.model_selection import train_test_split
train_split=.8  # percentage of df to use for training
test_split = .05 # percentage of df to use for test
# note valid_split=1- train_split = test_ split in this case .15
train_df, dummy_df= train_test_split(df, train_split=train_split, shuffle=True, random_state=123)
dummy_split=test_split/(1.0 - train_split)
test_df, valid_df=train_test_split(dummy_df, train_split=dummy_split) # note dummy_df is 20% of df

现在您需要创建训练、测试和验证生成器。我假设你有图像输入。使用 ImageDataGenerator.flow_from_dataframe() 创建您的生成器。 [此处][2] 的文档是在 0 到 1 或 -1 到 +1 范围内缩放像素图像的典型做法。我更喜欢后者。下面是生成器的代码

def scalar(img):
       return img/127.5 -1
x_col='Filename' # set this to point to the df column that holds the full path to the image file
y_col- 'Label' # set this to the df column that holds the class labels
batch_size = 32 # set this to the batch size
class_mode='categorical'
image_shape=(64,64) # set this to desired image size
tgen=tf.keras.preprocessing.image.ImageDataGenerator(preprocessing_function=scalar, horizontal_flip=True)
gen= tf.keras.preprocessing.image.ImageDataGenerator(preprocessing_function=scalar)
train_gen=tgen.flow_from_dataframe(train_df, x_col=x_col, 
                                   y_col=y_col,target_size=image_shape,
                                   class_mode=class_mode, batch_size=batch_size, 
                                    shuffle=True, random_state=123)                                   
test_gen=tgen.flow_from_dataframe(test_df, x_col= x_col, y_col=y_col 
                                   class_mode=class_mode, 
                                   batch_size=batch_size, shuffle=False)                                        
valid_gen=tgen.flow_from_dataframe(valid_df, x_col=x_col, shuffle=False,
                                   y_col=y_col,batch_size=batch_size 
                                   class_mode=class_mode , target_size=image_shape)

现在是时候使用 model.fit 训练您的模型了。文档在 [此处。][3]

epochs = 15 # set this to the number of epochs to run
history=model.fit(train_gen, epochs=epochs, validation_data= valid_gen, verbose=1)

模型完成训练后，您想查看它在测试集上的表现如何。你这样做 model.evaluate 如下所示

accuracy = model.evaluate(test_gen, verbose=1)[1]
print (accuracy)

您可以使用您的模型通过 model.predict

进行预测

preds=model.predict(test_gen, verbose=1)

preds 是一个矩阵，其行数等于测试集中的样本数，列数等于您拥有的 classes 数。每个条目都是图像在 class 中的概率值。您想要 select 具有最高概率值的列。使用 np.argmax 查找具有最高概率值的列 (class)。您可以使用 labels-test_gen.labels 获取 test_set 标签。代码如下

correct_preds=0
for i in range (len(preds):
    index=np.argmax(preds[i]) # column with highest probability
    label=test_gen.labels[i] # true class of image
    if index == label:
        correct_preds +=1
pred_accuracy= correct_preds/ len(preds)
print ( pred_accuracy)
    
                                
 


  [1]: https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html
  [2]: https://keras.io/api/preprocessing/image/
  [3]: https://www.tensorflow.org/api_docs/python/tf/keras/Model

我如何在 .csv 文件的数据集上训练我的 CNN 模型？

how can i train my CNN model on dataset from a .csv file?

python

csv

training-data

conv-neural-network

tensorflow