Custom Metrics and Losses: AttributeError: 'Tensor' object has no attribute 'numpy' raised during training

Question

我正在尝试实现自定义度量函数和自定义损失函数。两种实现都面临着同样的问题，所以我将只关注其中一个 post。

我的目标是在拟合方法期间访问张量的值，以便根据存储在 y_true 和 y_pred 中的所述值进行计算。 无法使用内置的 Keras 后端函数完成这些计算。

例如，我们有以下虚拟代码：

import numpy as np
import tensorflow as tf
from tensorflow.keras.models import Sequential, Model
from tensorflow.keras.layers import Input, LSTM, Dense
from tensorflow.keras.metrics import Metric

x, y = list(), list()
for _ in range(10):
    x.append(np.arange(10))
    y.append(np.random.randint(0, 2))


x = np.reshape(x, (len(x), 1, len(x[0])))
y = np.asarray(y)

class custom_metric(Metric):
    def __init__(self, name = 'custom_metrics', **kwargs):
        super(custom_metric, self).__init__(name = name, **kwargs)
        self.true_positives = self.add_weight(name = 'tp', initializer = 'zeros')

    def update_state(self, y_true, y_pred, sample_weight = None):
        self.test(y_true, y_pred)
        # In a real application, new_metric would be a function that depends on
        # the values stored in both y_true and y_pred 
        new_metric = 0.1 
        self.true_positives.assign_add(tf.reduce_sum(new_metric))

    def result(self):
        return self.true_positives

    def reset_states(self):
        self.true_positives.assign(0.)

    def test(self, y_true, y_pred):
        tf.print(y_true)
        print(y_true.numpy())

model = Sequential([
    LSTM(5,
         input_shape = (np.asarray(x).shape[1], np.asarray(x).shape[2]),
         return_sequences = True,
         recurrent_initializer = 'glorot_uniform',
         activation = 'tanh',
         recurrent_dropout = 0.2,
         dropout = 0.2
        ),
    Dense(2, activation = 'softmax')
])

model.compile(
    optimizer = 'adam',
    loss = 'sparse_categorical_crossentropy',
    metrics = ['sparse_categorical_accuracy', custom_metric()]
)

model.fit(
    x, y,
    epochs = 1,
    batch_size = 1
)

我写这个虚拟函数 test 只是为了说明这个问题。如果仅使用 tf.print，则代码运行并且张量中的值在拟合完成后打印在 stdout 上。但是，我是否尝试 y_true.numpy 或 print(y_true.numpy()) 代码 returns

AttributeError: 'Tensor' object has no attribute 'numpy'

我尝试了来自多个 Whosebug 和 Github 线程的几种方法，包括 sess = tf.Session() 与 .eval()、tf.GradientTape 的组合，但不知何故未能实现其中任何一个成功。

有谁知道如何解决这个问题？

Answer 1

numpy() 当在 tf2.x 下启用急切执行模式时，张量对象应该存在方法。也许这个 link 对你有帮助：https://www.tensorflow.org/guide/eager#object-oriented_metrics

Answer 2

对于 tensorflow<2.0 图形模式是默认的，为了运行在急切模式下你必须在开始时启用它：

import tensorflow as tf  #<--- first import 
tf.enable_eager_execution()   #<-- immidiately followed by this, before you start defining any model
.
.
.
...rest of the code

Eager-tensors 具有 .numpy() 函数。

但即使您这样做，也可能是 tf.keras.Model.fit() 方法在内部某处取消了它。因为：

这个有效：

def test(self, y_true, y_pred):
    if tf.executing_eagerly():  #<--- This is False
        print(y_true.numpy())
    else:
        print(y_pred)

这也是：

def test(self, y_true, y_pred):
    print(y_pred)

但是，这不会：

def test(self, y_true, y_pred):
        tf.print(y_true)
        print(y_true.numpy())

如果您想在 y_true 上做任何进一步的计算，那么您可以在图形模式下使用 tensorflow ops 来完成：

class custom_metric(Metric):
    def __init__(self, name = 'custom_metrics', **kwargs):
        super(custom_metric, self).__init__(name = name, **kwargs)
        self.true_positives = self.add_weight(name = 'tp', initializer = 'zeros')
        self.lol_value = self.add_weight(name = 'lol', initializer = 'zeros')

    def update_state(self, y_true, y_pred, sample_weight = None):
        self.test(y_true, y_pred)
        # In a real application, new_metric would be a function that depends on
        # the values stored in both y_true and y_pred 
        new_metric = 0.1 
        self.true_positives.assign_add(tf.reduce_sum(new_metric))

    def result(self):
        return self.lol_value

    def reset_states(self):
        self.true_positives.assign(0.)
        self.lol_value.assign(0.)

    def test(self, y_true, y_pred):
        print(y_pred)
        self.lol_value.assign_add(100)

或者如果你真的，绝对想要 numpy 然后使用 tf.numpy_function()，它将普通的 numpy 计算转换成它等效的图形代码。

def func_x(varx):
    #print(x)
    return (varx+1).astype(np.uint8)


class custom_metric(Metric):
    def __init__(self, name = 'custom_metrics', **kwargs):
        super(custom_metric, self).__init__(name = name, **kwargs)
        self.true_positives = self.add_weight(name = 'tp', initializer = 'zeros')
        self.res = self.add_weight(name='loop_counter', initializer='zeros', dtype=tf.uint8)

    def update_state(self, y_true, y_pred, sample_weight = None):
        self.test(y_true, y_pred)
        # In a real application, new_metric would be a function that depends on
        # the values stored in both y_true and y_pred 
        new_metric = 0.1 
        self.true_positives.assign_add(tf.reduce_sum(new_metric))

    def result(self):
        return self.res

    def reset_states(self):
        self.true_positives.assign(0.)

    def test(self, y_true, y_pred):
        self.res.assign(tf.numpy_function(func=func_x, inp=[self.res], Tout=[tf.uint8]))

Answer 3

终于找到答案了。我还不知道为什么，但是代码可以使用 tf-nightly 2.2.0-dev 版本。参见 https://github.com/tensorflow/tensorflow/issues/38038

Custom Metrics and Losses: AttributeError: 'Tensor' object has no attribute 'numpy' raised during training

Custom Metrics and Losses: AttributeError: 'Tensor' object has no attribute 'numpy' raised during training

keras

tensorflow

tf.keras

tensorflow2.0