InvalidArgumentError: Inner dimensions of output shape must match inner dimensions of updates shape

Question

我正在尝试在 keras 中实现 SPL 损失。我需要做的很简单，我会用 numpy 来解释我需要什么：

def spl_loss(y_true, y_pred, lmda):
    # compute any arbitrary loss function
    L = categorical_cross_entropy(y_true, y_pred)
    # set to zero those values with an error greater than lambda
    L[L>lmda] = 0
    return L

我正在尝试实现它 following this tutorial 但我在将值设置为零所需的步骤中遇到了问题。

目前我有以下代码：

def spl_loss(lmda, loss_fn):
    def loss(y_true, y_pred):
         # compute an arbitrary loss function, L
        loss_value = loss_fn(y_true, y_pred) # tensor of shape (64,)
        # get the mask of L greater than lmda
        mask = tf.greater( loss_value, tf.constant( float(lmda) ) )    # tensor of shape (64,)
        # compute indexes for the mask
        indexes = tf.reshape(tf.where(mask), [-1])  # tensor of shape (n,); where n<=64
        # set to zero values on indexes
        spl_loss_value = tf.tensor_scatter_nd_update(loss_value, indexes, tf.zeros_like(loss_value, dtype=loss_value.dtype) )  # this line gives the error
        
        return spl_loss_value
    return loss

根据docs，tensor_scatter_nd_update操作应该执行赋值操作，但失败并出现以下错误：

    spl_loss_value = tf.tensor_scatter_nd_update(loss_value, indexes, tf.zeros_like(loss_value, dtype=loss_value.dtype) )
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py:201 wrapper  **
        return target(*args, **kwargs)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/array_ops.py:5512 tensor_scatter_nd_update
        tensor=tensor, indices=indices, updates=updates, name=name)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gen_array_ops.py:11236 tensor_scatter_update
        _ops.raise_from_not_ok_status(e, name)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py:6862 raise_from_not_ok_status
        six.raise_from(core._status_to_exception(e.code, message), None)
    <string>:3 raise_from
        

    InvalidArgumentError: Inner dimensions of output shape must match inner dimensions of updates shape. Output: [64] updates: [64] [Op:TensorScatterUpdate]

我在colab中运行，here你可以试试

我尝试了几种重塑形状，因为我知道这是预期形状与获得形状的问题，但我找不到方法。这是怎么回事？

提前致谢

Answer 1

您收到此错误的原因是 tf.tensor_scatter_nd_update 中的 indices 至少需要 两个轴 ，或者 tf.rank(indices) > = 2 需要被填满。 2D中的indices（在scaler update中）的原因是为了保存两个信息，一个是更新的length（num_updates）和索引向量的长度。有关此内容的详细概述，请查看以下相关答案：.

这是 SPL 损失 在 tensorflow 中的正确实现。

def spl_loss(lmda):
    def loss(y_true, y_pred):
         # compute an arbitrary loss function, L
        loss_value = keras.losses.sparse_categorical_crossentropy(y_true, y_pred)

        # get the mask of L greater than lmda
        mask = tf.greater( loss_value, tf.constant(float(lmda) ) )    

        # compute indexes for the mask
        indexes = tf.where(mask) # tensor of shape (n,); where n<=64
        updates = tf.reshape(tf.zeros_like(indexes, dtype=tf.float32), [-1])

        # scaler update check
        num_updates, index_depth = indexes.shape.as_list()
        assert updates.shape == [num_updates]
        assert index_depth == tf.rank(loss_value)


        # print()
        # print('A', tf.reshape(tf.where(mask), [-1])[:10].numpy()) 
        # print('B', tf.where(mask).numpy()[:10]) 
        # print('Ranks: ', tf.rank(loss_value).numpy(), 
        #                  tf.rank(indices).numpy(), 
        #                   tf.rank(updates).numpy())
        # print('Shape: ', loss_value.shape, indexes.shape, updates.shape)

        # set to zero values on indexes
        spl_loss_value = tf.tensor_scatter_nd_update(loss_value, indexes, updates )

        return spl_loss_value
    return loss

...
model.compile(optimizer="adam", loss=spl_loss(lmda=2.), run_eagerly=True)
...

参考：tf.tensor_scatter_nd_update

InvalidArgumentError: Inner dimensions of output shape must match inner dimensions of updates shape

InvalidArgumentError: Inner dimensions of output shape must match inner dimensions of updates shape

python

tensorflow

keras

tensorflow2.0

loss-function