张量流中的自定义 f1_score 指标

Question

我想为 tf.keras 实施 f1_score 指标。

from tensorflow.keras.models import Model, Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.losses import binary_crossentropy
from tensorflow.keras.metrics import Accuracy, BinaryAccuracy
from sklearn.metrics import accuracy_score
import numpy as np
import tensorflow as tf

class F1_Score(tf.keras.metrics.Metric):

    def __init__(self, name='f1_score', **kwargs):
        super().__init__(name=name, **kwargs)
        self.f1 = self.add_weight(name='f1', initializer='zeros')

    def update_state(self, y_true, y_pred, sample_weight=None):
        p = Precision(thresholds=0.5)(y_true, y_pred)
        r = Recall(thresholds=0.5)(y_true, y_pred)
        self.f1 = 2 * ((p * r) / (p + r + 1e-6))

    def result(self):
        return self.f1

    def reset_states(self):
        self.f1.assign(0)
        
model = Sequential([
  Dense(64, activation='relu', input_shape=(784,)),
  Dense(64, activation='relu'),
  Dense(4, activation='sigmoid'),
])
x = np.random.normal(size=(10, 784))
y = np.random.choice(2, size=(10, 4))
model.compile(optimizer=Adam(0.001), loss='binary_crossentropy',
                  metrics=['accuracy', , F1_Score()])
model.fit(x[:1], y[:1], batch_size=1, epochs=1, verbose=1)

我收到一个错误：

ValueError: tf.function-decorated function tried to create variables on non-first call.

Answer 1

您收到此错误是因为您想在 update_state 函数期间实例化一些 tf.Variable。从 class Precision 和 Recall 实例化对象时，您正在创建一些 tf.Variables.

在构造函数中实例化对象，在update_state函数中调用：

class F1_Score(tf.keras.metrics.Metric):

    def __init__(self, name='f1_score', **kwargs):
        super().__init__(name=name, **kwargs)
        self.f1 = self.add_weight(name='f1', initializer='zeros')
        self.precision_fn = Precision(thresholds=0.5)
        self.recall_fn = Recall(thresholds=0.5)

    def update_state(self, y_true, y_pred, sample_weight=None):
        p = self.precision_fn(y_true, y_pred)
        r = self.recall_fn(y_true, y_pred)
        # since f1 is a variable, we use assign
        self.f1.assign(2 * ((p * r) / (p + r + 1e-6)))

    def result(self):
        return self.f1

    def reset_states(self):
        # we also need to reset the state of the precision and recall objects
        self.precision_fn.reset_states()
        self.recall_fn.reset_states()
        self.f1.assign(0)

行为解释：

Tensorflow 只允许在第一次调用 tf.function 时创建变量，请参阅 documentation :

tf.function only allows creating new tf.Variable objects when it is called for the first time

Keras 指标包装在 tf.function 中以允许与 tensorflow v1 兼容。您可以在 code

中找到此评论

If update_state is not in eager/tf.function and it is not from a built-in metric, wrap it in tf.function. This is so that users writing custom metrics in v1 need not worry about control dependencies and return ops.

您的 class 中还有另一个错误，即您覆盖了您在计算 f1 分数时创建的 f1 tf.Variable。要更新变量的值，您需要使用 assign。我们一定不要忘记重置正在使用的 Precision 和 Recall Metrics 对象的状态！

Answer 2

您可以使用 tensorflow-addons，它有一个 built-in 方法用于 F1-Score。（别忘了pip install tensorflow-addons）

往下看：

  model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.00001),
                  loss=tf.keras.losses.CategoricalCrossentropy(),
                  metrics=[tf.keras.metrics.CategoricalAccuracy(),
                           tfa.metrics.F1Score(num_classes=n_classes, average='macro'),
                           tfa.metrics.FBetaScore(beta=2.0, num_classes=n_classes, average='macro')])

如果您确实有multi-label分类问题，您可以将其更改为：

  model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.00001),
                      loss=tf.keras.losses.BinaryCrossentropy(),
                      metrics=[tf.keras.metrics.BinaryAccuracy(),
                               tfa.metrics.F1Score(num_classes=1, average='macro',threshold=0.5),
                               tfa.metrics.FBetaScore(beta=2.0, num_classes=1, average='macro',threshold=0.5)])

Answer 3

这是我为 Tensorflow 2.0 评分 f1 的代码：

class F1Score(tf.keras.metrics.Metric):
  def __init__(self, name='F1Score', **kwargs):
    super(F1Score, self).__init__(name=name, **kwargs)
    self.f1score = self.add_weight(name='F1Score', initializer='zeros')
    self.count = self.add_weight(name='F1ScoreCount', initializer='zeros')

  def update_state(self, y_true, y_pred, sample_weight=None):
    y_true = tf.cast(y_true, tf.bool)
    y_pred = tf.cast(y_pred, tf.bool)

    true_positives = tf.logical_and(tf.equal(y_true, True), tf.equal(y_pred, True))
    true_positives = tf.cast(true_positives, self.dtype)
    count_true_positives = tf.reduce_sum(true_positives)

    possible_positives = tf.cast(y_true, self.dtype)
    count_possible_positives = tf.reduce_sum(possible_positives)

    predicted_positives = tf.cast(y_pred, self.dtype)
    count_predicted_positives = tf.reduce_sum(predicted_positives)

    precision = count_true_positives / (count_predicted_positives + K.epsilon())
    recall = count_true_positives / (count_possible_positives + K.epsilon())
    f1_cal = 2*(precision*recall)/(precision + recall + K.epsilon())

    self.count.assign_add(1)
    a = 1.0 / self.count
    b = 1.0 - a
    self.f1score.assign(a*f1_cal+b*self.f1score)

  def result(self):
    return self.f1score

张量流中的自定义 f1_score 指标

Custom f1_score metric in tensorflow

metrics

machine-learning

keras

tensorflow