tf.keras 中的 MirroredStrategy 指标 SparseTopKCategoricalAccuracy 失败

Question

我的计算平台有两个 GPU，我使用 MirroredStrategy() 在两个 GPU 之间分担负载。当我使用 SparseTopKCategoricalAccuracy 作为指标时，我运行遇到了问题。我的代码如下：

with mirrored_strategy.scope():

    model=Sequential()

    model.add(Dense(512, kernel_initializer= 'he_uniform', input_shape=(X_train.shape[1],),
                activity_regularizer=l1(1.e-3)))
    model.add(Activation('relu'))

    model.add(BatchNormalization(momentum=0.0)) 

    model.add(Dropout(0.2))
    model.add(Dense(4096, kernel_initializer= 'he_uniform', activation='relu',
                activity_regularizer=l1(1.e-4)))

    model.add(BatchNormalization(momentum=0.0))

    model.add(Dropout(0.3))

    model.add(Dense(512, kernel_initializer= 'he_uniform', activation='relu',
                activity_regularizer=l1(1.e-4)))

    model.add(BatchNormalization(momentum=0.0))

    model.add(Dropout(0.5))
    model.add(Dense(num_classes, activation='softmax'))


opt=Adam(learning_rate=0.001)
m=SparseTopKCategoricalAccuracy(k=10)
model.compile(optimizer=opt, loss='sparse_categorical_crossentropy', metrics=m)  

es = EarlyStopping(monitor='val_sparse_top_k_categorical_accuracy', patience=10, mode='max', verbose=0, restore_best_weights=True)
lr = LearningRateScheduler(lr_scheduler)enter code here

我收到的报错信息如下：

ValueError: Metric (<tensorflow.python.keras.metrics.SparseTopKCategoricalAccuracy object at 0x7fee452d4f90>) passed to model.compile was created inside of a different distribution strategy scope than the model. All metrics must be created in the same distribution strategy scope as the model (in this case <tensorflow.python.distribute.mirrored_strategy.MirroredStrategy object at 0x7ff5cd1ff490>). If you pass in a string identifier for a metric to compile the metric will automatically be created in the correct distribution strategy scope.

如果我去掉“with mirrored_strateg.scope():”，它可以工作，但只有一个 GPU。我需要修复什么才能在两个 GPU 上实现运行？

Answer 1

原因在错误信息中：

All metrics must be created in the same distribution strategy scope as the model

在镜像策略范围内实例化指标。

with mirrored_strategy.scope():
    # Define model ...
    m = SparseTopKCategoricalAccuracy(k=10)

model.compile(optimizer=opt, loss='sparse_categorical_crossentropy', metrics=m)

es = EarlyStopping(monitor='val_sparse_top_k_categorical_accuracy', patience=10, mode='max', verbose=0, restore_best_weights=True)
lr = LearningRateScheduler(lr_scheduler)

tf.keras 中的 MirroredStrategy 指标 SparseTopKCategoricalAccuracy 失败

Metrics SparseTopKCategoricalAccuracy failed with MirroredStrategy in tf.keras

metrics

keras

tensorflow

tensorflow2.0