使用 Keras-tuner 调整超参数时出现 "accuracy" 错误

Question

我原来的MLP模型如下：

def create_model(n_hidden_1, n_hidden_2, num_classes, num_features):
    # create the model
    model = Sequential()
    model.add(tf.keras.layers.InputLayer(input_shape=(num_features,)))
    model.add(tf.keras.layers.Dense(n_hidden_1, activation='sigmoid'))
    model.add(tf.keras.layers.Dense(n_hidden_2, activation='sigmoid'))
    model.add(tf.keras.layers.Dense(num_classes, activation='softmax'))
    # instantiate the optimizer
    opt = keras.optimizers.SGD(learning_rate=LEARNING_RATE)
    # compile the model
    model.compile(
        optimizer=opt,
        loss="categorical_crossentropy",
        metrics="categorical_accuracy"
    )
    # return model
    return model

为了调整上述模型，我创建了一个 Keras-tuner 模型，如下所示：

def _model(hp):
    model = keras.Sequential()
    model.add(tf.keras.layers.InputLayer(input_shape=(6)))
    model.add(tf.keras.layers.Dense(
            hp.Int("dense_1_units", min_value=32, max_value=2048, step=32, default=128),
            activation="sigmoid"
        ))
    model.add(tf.keras.layers.Dense(
        hp.Int("dense_2_units", min_value=32, max_value=2048, step=32, default=128),
        activation="sigmoid"
    ))
    model.add(tf.keras.layers.Dense(3, activation="softmax"))
    model.compile(
        optimizer=tf.keras.optimizers.SGD(
            hp.Choice("learning_rate", values=[1e-1, 1e-2, 1e-3])
        ),
        loss="categorical_crossentropy",
        metrics="categorical_accuracy"
    )
    return model

   tuner = RandomSearch(
        _model,
        objective="val_accuracy",
        max_trials=10,
        overwrite=True,
        directory="tuner_random_directory",
        project_name="tuner_random_project_name",
    )

我收到以下输出：

user@server:~/ $ python3 tuner.py

training data size :  1120988
validation data size :  280246
Search space summary
Default search space size: 3
dense_1_units (Int)
{'default': 128, 'conditions': [], 'min_value': 32, 'max_value': 2048, 'step': 32, 'sampling': None}
dense_2_units (Int)
{'default': 128, 'conditions': [], 'min_value': 32, 'max_value': 2048, 'step': 32, 'sampling': None}
learning_rate (Choice)
{'default': 0.1, 'conditions': [], 'values': [0.1, 0.01, 0.001], 'ordered': True}

Search: Running Trial #1

Hyperparameter    |Value             |Best Value So Far
dense_1_units     |1504              |?
dense_2_units     |1440              |?
learning_rate     |0.1               |?

Epoch 1/2
35031/35031 [==============================] - 811s 23ms/step - loss: 0.5475 - categorical_accuracy: 0.7495 - val_loss: 0.5155 - val_categorical_accuracy: 0.7599
WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
Epoch 2/2
35031/35031 [==============================] - 807s 23ms/step - loss: 0.5091 - categorical_accuracy: 0.7650 - val_loss: 0.4943 - val_categorical_accuracy: 0.7751
WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
Traceback (most recent call last):
  File "my_tuner_app_tuner_random.py", line 229, in <module>
    tuner.search(train_x, train_y, epochs=2, validation_data=(validate_x, validate_y))
  File "/home/user/.local/lib/python3.7/site-packages/keras_tuner/engine/base_tuner.py", line 144, in search
    self.run_trial(trial, *fit_args, **fit_kwargs)
  File "/home/user/.local/lib/python3.7/site-packages/keras_tuner/engine/multi_execution_tuner.py", line 103, in run_trial
    trial.trial_id, metrics=averaged_metrics, step=self._reported_step
  File "/home/user/.local/lib/python3.7/site-packages/keras_tuner/engine/oracle.py", line 224, in update_trial
    self._check_objective_found(metrics)
  File "/home/user/.local/lib/python3.7/site-packages/keras_tuner/engine/oracle.py", line 407, in _check_objective_found
    objective_names, metrics.keys()

ValueError: Objective value missing in metrics reported to the Oracle, expected: ['val_accuracy'], found: dict_keys(['loss', 'categorical_accuracy', 'val_loss', 'val_categorical_accuracy'])

user@server:~/ $

为什么会收到一个警告和一个值错误？

我该如何解决这些问题？

Answer 1

我可以想象您会收到此警告和错误，因为您必须在 RandomSearch 目标中使用模型和 Keras Tuner 模型中完全相同的指标，即 categorical_accuracy。所以，也许，试试：

tuner = RandomSearch(
        _model,
        objective="val_categorical_accuracy",
        max_trials=10,
        overwrite=True,
        directory="tuner_random_directory",
        project_name="tuner_random_project_name",
)

因为您想最大限度地提高验证的分类准确性。

使用 Keras-tuner 调整超参数时出现 "accuracy" 错误

Error regarding "accuracy" in hyper-parameter tuning using Keras-tuner

python

tensorflow

machine-learning

keras

keras-tuner