SciKeras - 用于最佳超参数的 RandomizedSearchCV

Question

我正在尝试遵循本书第 10 章的示例 Hands-On Machine Learning with SciKit-Learn、Keras 和 TensorFlow，其中涉及超参数的优化一个 DNN 模型。

数据集是 MNIST 时装模特，项目的目标是对 10 类中的图像进行分类。

(X_train_full, y_train_full), (X_test, y_test) = fashion_mnist.load_data()

没有验证，所以我将使用前 5k 个元素创建该集合：

X_valid, X_train = X_train_full[:5000], X_train_full[5000:]
y_valid, y_train = y_train_full[:5000], y_train_full[5000:]

一个简单的 DNN 模型的可能实现如下，使用顺序 Keras API:

model = keras.models.Sequential()
model.add(keras.layers.Flatten(input_shape=[28, 28]))
model.add(keras.layers.Dense(300, activation="relu"))
model.add(keras.layers.Dense(200, activation="relu"))
model.add(keras.layers.Dense(100, activation="relu"))
model.add(keras.layers.Dense(10, activation="softmax"))

这本书然后建议研究超参数 space 以找到最好的参数，使用 RandomizedSearchCV。该示例使用 keras.wrappers.scikit_learn.KerasRegressor，现在 SciKeras 已弃用 KerasRegressor。

我创建了一个包含 ML 模型的函数：

input_shape=X_train[0].shape
def build_model(n_hidden = 1, n_neurons = 30, learning_rate=3e-3, input_shape=input_shape):
    grid_model = keras.models.Sequential()
    grid_model.add(keras.layers.Flatten(input_shape=input_shape))
    for layer in range(n_hidden):
        grid_model.add(keras.layers.Dense(n_neurons, activation="relu"))
    grid_model.add(keras.layers.Dense(10, activation="softmax"))
    opt = tf.keras.optimizers.SGD(learning_rate=learning_rate)
    grid_model.compile(loss="sparse_categorical_crossentropy", optimizer=opt, metrics=["accuracy"])
    return grid_model

然后我定义了要探索的模型超参数：

param_distribs = {
    "n_hidden": [0, 1, 2, 3, 4, 5],
    "n_neurons": np.arange(1, 300),
    "learning_rate": reciprocal(3e-4, 3e-2)
}

然后我使用 SciKeras 在 Keras 模型周围创建了一个包装器，提供了参数 space:

keras_reg = KerasRegressor(build_model, n_hidden=param_distribs["n_hidden"], n_neurons=param_distribs["n_neurons"], learning_rate=param_distribs["learning_rate"], verbose=1)

最后一步是定义一个RandomizedSearchCV对象并使用fit方法开始研究：

rnd_search_cv = RandomizedSearchCV(keras_reg, param_distribs, n_iter=10, cv=3)

rnd_search_cv.fit(X_train, y_train, epochs=100, validation_data=(X_valid, y_valid), callbacks=[keras.callbacks.EarlyStopping(patience=10)])

对于每个纪元，最后一行给我以下错误：

/home/docker_user/.local/lib/python3.8/site-packages/sklearn/model_selection/_validation.py:776: UserWarning: Scoring failed. The score on this train-test partition for these parameters will be set to nan. Details: 
Traceback (most recent call last):
  File "/home/docker_user/.local/lib/python3.8/site-packages/sklearn/model_selection/_validation.py", line 767, in _score
    scores = scorer(estimator, X_test, y_test)
  File "/home/docker_user/.local/lib/python3.8/site-packages/sklearn/metrics/_scorer.py", line 429, in _passthrough_scorer
    return estimator.score(*args, **kwargs)
  File "/home/docker_user/.local/lib/python3.8/site-packages/scikeras/wrappers.py", line 1100, in score
    return self.scorer(y, y_pred, sample_weight=sample_weight, **score_args)
  File "/home/docker_user/.local/lib/python3.8/site-packages/scikeras/wrappers.py", line 1697, in scorer
    return sklearn_r2_score(y_true, y_pred, **kwargs)
  File "/home/docker_user/.local/lib/python3.8/site-packages/sklearn/metrics/_regression.py", line 911, in r2_score
    y_type, y_true, y_pred, multioutput = _check_reg_targets(
  File "/home/docker_user/.local/lib/python3.8/site-packages/sklearn/metrics/_regression.py", line 100, in _check_reg_targets
    check_consistent_length(y_true, y_pred)
  File "/home/docker_user/.local/lib/python3.8/site-packages/sklearn/utils/validation.py", line 387, in check_consistent_length
    raise ValueError(
ValueError: Found input variables with inconsistent numbers of samples: [18334, 183340]

二次元的一个因子10让我思考...我也检查了数据的形状，它们很好...

print(X_train.shape, y_train.shape)
(55000, 28, 28) (55000,)

你能帮我处理这个错误吗？

Answer 1

顾名思义，scikeras.wrappers.KerasRregressor should be employed for regression and uses the r2_score by default. Hence, it will not convert the softmax output to a class index like the ones you have in your validation set. The last dense layer's output is of size 10 (hence the factor of 10). Instead, you should use scikeras.wrappers.KerasClassifier默认使用准确性作为评分。只需将 KerasRegressor 换成 KerasClassifier 就可以了。

SciKeras - 用于最佳超参数的 RandomizedSearchCV

SciKeras - RandomizedSearchCV for best hyper-parameters

python

tensorflow

keras