Keras 的 NaN 随机搜索的调谐器分数

Question

我正在试用 Keras (2.8.0) 自动调谐器来解决回归问题。这是我的代码：

import pandas as pd
from tensorflow import keras
from keras import layers, losses
from keras_tuner.tuners import RandomSearch

df = pd.read_csv('./datasets/data.csv')

X = df.iloc[:, :-1]
y = df.iloc[:, -1]


def build_model(hp):
    model = keras.Sequential()
    for i in range(hp.Int('num_layers', 2, 20)):
        model.add(layers.Dense(units=hp.Int('units_' + str(i),
                                            min_value=32,
                                            max_value=512,
                                            step=32),
                               activation='relu'))
    model.add(layers.Dense(1, activation='linear'))
    model.compile(
        optimizer=keras.optimizers.Adam(
            hp.Choice('learning_rate', [1e-2, 1e-3, 1e-4])),
        loss=losses.MeanAbsoluteError(reduction="auto", name="mean_absolute_error"),
        metrics=['mean_absolute_error'])
    return model


tuner = RandomSearch(
    build_model,
    objective='val_mean_absolute_error',
    max_trials=5,
    executions_per_trial=3,
    directory='project',
    project_name='TunerTest')

model_summary = tuner.search_space_summary()

print(model_summary)

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)

tuner.search(X_train, y_train,
             epochs=5,
             validation_data=(X_test, y_test))

tuner.results_summary()

代码运行没有任何错误（假设）但在结果中，对于所有迭代，Score 是 nan。以下是其中一次运行的输出：

Showing 10 best trials
<keras_tuner.engine.objective.Objective object at 0x7fef945235d0>
Trial summary
Hyperparameters:
num_layers: 12
units_0: 128
units_1: 224
learning_rate: 0.01
units_2: 32
units_3: 32
units_4: 32
units_5: 32
units_6: 32
units_7: 32
units_8: 32
units_9: 32
units_10: 32
units_11: 32
Score: nan

我在这里做错了什么？

Answer 1

这可能是由于您的数据具有 nan 个值或非常接近于零的值。另一个问题可能是数据的差异。也许尝试对其进行归一化以避免爆炸梯度或什至应用梯度剪裁。使用随机数据时，您的代码有效并且结果不是 nan:

import pandas as pd
from tensorflow import keras
from keras import layers, losses
from keras_tuner.tuners import RandomSearch
import numpy as np

X = np.random.random((500, 5))
y = np.random.random((500, 1))

def build_model(hp):
    model = keras.Sequential()
    for i in range(hp.Int('num_layers', 2, 20)):
        model.add(layers.Dense(units=hp.Int('units_' + str(i),
                                            min_value=32,
                                            max_value=512,
                                            step=32),
                               activation='relu'))
    model.add(layers.Dense(1, activation='linear'))
    model.compile(
        optimizer=keras.optimizers.Adam(
            hp.Choice('learning_rate', [1e-2, 1e-3, 1e-4])),
        loss=losses.MeanAbsoluteError(reduction="auto", name="mean_absolute_error"),
        metrics=['mean_absolute_error'])
    return model


tuner = RandomSearch(
    build_model,
    objective='val_mean_absolute_error',
    max_trials=5,
    executions_per_trial=3,
    directory='project',
    project_name='TunerTest')

model_summary = tuner.search_space_summary()

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)

tuner.search(X_train, y_train,
             epochs=5,
             validation_data=(X_test, y_test))

tuner.results_summary()

...
Trial summary
Hyperparameters:
num_layers: 20
units_0: 512
units_1: 384
learning_rate: 0.01
units_2: 32
units_3: 32
units_4: 32
units_5: 32
units_6: 32
units_7: 32
units_8: 32
units_9: 32
units_10: 32
units_11: 32
units_12: 32
units_13: 32
units_14: 32
units_15: 32
units_16: 32
units_17: 32
units_18: 32
units_19: 32
Score: 0.24690431356430054

Answer 2

如果我没记错的话，这是 Krish Naik 的 Dl 视频中的一个例子；我尝试了完全相同的代码并遇到了以下问题。我只是替换了数据集中的空值，它工作正常。

Keras 的 NaN 随机搜索的调谐器分数

NaN for Keras Tuner score for RandomSearch

python

deep-learning

keras

tensorflow

keras-tuner