cnn-keras fit_generator 回调中的值错误

value error in cnn- keras fit_generator call back

model.fit_generator() 中出错。作为 python 的初级程序员,我不确定此错误表示什么。

我正在尝试 python 中的迁移学习以使用 imagenet 训练 VGG19,我在回调中遇到值错误。谁能建议我应该对此代码进行哪些更改??

我尝试在 colab 中执行以下代码,但出现错误

!apt-get install -y -qq software-properties-common python-software-properties module-init-tools
!add-apt-repository -y ppa:alessandro-strada/ppa 2>&1 > /dev/null
!apt-get update -qq 2>&1 > /dev/null
!apt-get -y install -qq google-drive-ocamlfuse fuse
from google.colab import auth
auth.authenticate_user()
from oauth2client.client import GoogleCredentials
creds = GoogleCredentials.get_application_default()
import getpass
!google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret} < /dev/null 2>&1 | grep URL
vcode = getpass.getpass()
!echo {vcode} | google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret}
!mkdir -p drive
!google-drive-ocamlfuse drive
!pip install opencv-python
!pip install opencv-contrib-python
!apt update && apt install -y libsm6 libxext6
!pip install -q keras
from glob import glob

import cv2
import numpy as np

from sklearn.utils import shuffle
from sklearn.model_selection import train_test_split

from keras.utils import to_categorical
from keras.models import Model
from keras.layers import Dropout, Dense, Flatten
from keras.optimizers import SGD
from keras.losses import categorical_crossentropy
from keras.regularizers import l2
from keras.applications.vgg19 import VGG19
model = VGG19(include_top=False, weights='imagenet', pooling='avg')
for layer in model.layers:
    layer.trainable = False
x = model.output
predictions = Dense(7, activation='softmax')(x)
model_final = Model(input=model.input, output=predictions)
from keras.callbacks import ReduceLROnPlateau
lr_reducer = ReduceLROnPlateau(monitor='val_loss', factor=0.9, patience=4, verbose=1)
model_final.compile(loss=categorical_crossentropy,
                  optimizer=SGD(lr=0.001, momentum=0.9, nesterov=True),
                  metrics=['accuracy'])
model_final.fit(np.array(X_train), np.array(y_train),
              batch_size=32,
              epochs=10,
              verbose=1,
              validation_split=0.1,
              shuffle=True)
for layer in model_final.layers[7:]:
  layer.trainable = True
model_final.compile(loss=categorical_crossentropy,
                  optimizer=SGD(lr=0.001, momentum=0.9, nesterov=True),
                  metrics=['accuracy'])
from keras.preprocessing.image import ImageDataGenerator
train_generator = ImageDataGenerator(
    featurewise_center = True,
    featurewise_std_normalization = True,
    rotation_range=30,
    shear_range=0.2,
    zoom_range=0.2,
    width_shift_range=0.2,
    height_shift_range=0.2,
    horizontal_flip=True)

train_generator.fit(np.array(X_train))

test_generator = ImageDataGenerator(
    featurewise_center = True,
    featurewise_std_normalization = True)

test_generator.fit(np.array(X_train))

model_final.fit_generator(train_generator.flow(np.array(X_train), np.array(y_train), batch_size=32),
                          validation_data=test_generator.flow(np.array(X_test), np.array(y_test)),
                          steps_per_epoch=len(X_train)/32, 
                          epochs=50)
ValueError                                Traceback (most recent call last)
<ipython-input-39-f9af6d0d8994> in <module>()
      2                           validation_data=test_generator.flow(np.array(X_test), np.array(y_test)),
      3                           steps_per_epoch=len(X_train)/32,
----> 4                           epochs=50)

2 frames
/usr/local/lib/python3.6/dist-packages/keras/engine/training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
     66     if (val_gen and not isinstance(validation_data, Sequence) and
     67             not validation_steps):
---> 68         raise ValueError('`validation_steps=None` is only valid for a'
     69                          ' generator based on the `keras.utils.Sequence`'
     70                          ' class. Please specify `validation_steps` or use'

ValueError: `validation_steps=None` is only valid for a generator based on the `keras.utils.Sequence` class. Please specify `validation_steps` or use the `keras.utils.Sequence` class.

您必须在 model_final.fit_generator 中指定 validation_steps。这是因为生成器不知道将使用的数据总数,它只知道batch_size(默认为batch_size=32)。因此,您必须通过提供每个时期的步数来手动告诉生成器何时停止加载数据。 step其实就是批次的数量。

如果你想在每个epoch中使用所有测试数据进行验证:

model_final.fit_generator(train_generator.flow(np.array(X_train), np.array(y_train), batch_size=32),
                          validation_data=test_generator.flow(np.array(X_test), np.array(y_test), batch_size=32),
                          steps_per_epoch=len(X_train)/32, 
                          validation_steps=len(X_test)/32,
                          epochs=50)

报错信息还提到validation_step=None仅在使用继承自Sequence的生成器时有效。在这种情况下,validation_step 将自动设置为 len(validation_data)See here. It can be done only because __len__(self) method is defined in Sequence object See here,但不在您的 ImageDataGenerator