使用 Keras 和 Tensorflow 的自定义层

Question

我正在使用 Keras 和 Tensorflow 构建自定义层，用于计算一组框之间并集的交集。 a 有两组不同维度的锚点和 gt_boxes，我将计算锚点和 gt_boxes 中每个元素之间并集的交集当我执行代码时出现以下错误：

Use fn_output_signature instead
Traceback (most recent call last):
  File "acceuil.py", line 60, in <module>
    train(train_data[1], val_data[1], dataset_name)
  File "acceuil.py", line 28, in train
    model.train()
  File "/home/imene/APP-M/ROI/mod.py", line 40, in train
    inputs, outputs =   self.worker() 
  File "/home/imene/APP-M/ROI/mod.py", line 135, in worker
    iou_anchors = ioulayer([tf_rois, tf_anchors])
  File "/home/imene/anaconda3/envs/master/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer_v1.py", line 786, in __call__
    outputs = call_fn(cast_inputs, *args, **kwargs)
  File "/home/imene/anaconda3/envs/master/lib/python3.6/site-packages/tensorflow/python/autograph/impl/api.py", line 670, in wrapper
    raise e.ag_error_metadata.to_exception(e)
ValueError: in user code:

    /home/imene/APP-M/ROI/IoULayer.py:25 call  *
        IoU_anchors = tf.map_fn(compute_IoU, (inputs[0], inputs[1] ), dtype=tf.float32)
    /home/imene/anaconda3/envs/master/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py:538 new_func  **
        return func(*args, **kwargs)
    /home/imene/anaconda3/envs/master/lib/python3.6/site-packages/tensorflow/python/ops/map_fn.py:451 map_fn
        tensor.get_shape().with_rank_at_least(1)[0])))
    /home/imene/anaconda3/envs/master/lib/python3.6/site-packages/tensorflow/python/framework/tensor_shape.py:315 merge_with
        self.assert_is_compatible_with(other)
    /home/imene/anaconda3/envs/master/lib/python3.6/site-packages/tensorflow/python/framework/tensor_shape.py:282 assert_is_compatible_with
        (self, other))

    ValueError: Dimensions 3 and 10 are not compatible

这里是我的自定义层的代码，写成python:

import sys
import numpy as np 
import keras
from tensorflow.keras.layers import Layer
from tensorflow.python.keras.backend import map_fn

import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()

class IoULayer(Layer):
    def __init__(self, **kwargs):
        super(IoULayer, self).__init__(**kwargs)
        
        
# -------------------------------------------------------------------------------------------------
    def call(self, inputs):
        '''
        inputs[0]: ground truth boxes
        inputs[1]: anchors boxes
        '''    
        def compute_IoU(inputs):
            
            return IoULayer.multiple_IoU(inputs[0], inputs[1])

        IoU_anchors = tf.map_fn(compute_IoU, (inputs[0], inputs[1] ), dtype=tf.float32)
        
        return IoU_anchors

    def multiple_IoU(gt_boxes, anchors):

        def get_anchor_IoUs(gt_boxes, anchor):
            
            return IoULayer.get_single_IoU(gt_boxes, anchor)

        IoU = tf.map_fn(get_anchor_IoUs, gt_boxes, anchors, dtype=tf.float32)
        return IoU

    def get_single_IoU(gt_boxes, anchors):
        iou_list = []
        
        def single_iou(anchor):
            result = []
            for gt_bbx in gt_boxes:
                
                x_left   = max(gt_bbx[0], anchor[0])
                y_top    = max(gt_bbx[1], anchor[1])
                x_right  = min(gt_bbx[2], anchor[2])
                y_bottom = min(gt_bbx[3], anchor[3])

                bb1_area = (gt_bbx[2]- gt_bbx[0])*(gt_bbx[3]- gt_bbx[1])
                anchor_area = (anchor[2]- anchor[0])*(anchor[3]- anchor[1])

                intersect_area = abs(max((x_right - x_left), 0) * max((y_bottom - y_top),0))
                iou = intersect_area / float(bb1_area + anchor_area - intersect_area)
                result.append(iou)
                return result
        iou_list = tf.stack(single_iou(anchor) for anchor in anchors)
        return iou_list

我不明白这是什么问题？感谢回复

Answer 1

问题可能出在您输入的维度上。来自 tf.map_fn documentation:

If elems is a tuple (or nested structure) of tensors, then those tensors must all have the same outer-dimension size (num_elems); and fn is used to transform each tuple (or structure) of corresponding slices from elems. E.g., if elems is a tuple (t1, t2, t3), then fn is used to transform each tuple of slices (t1[i], t2[i], t3[i]) (where 0 <= i < num_elems).

所以在你的情况下，如果 tf_rois 和 tf_anchors 具有不同的维度（如 3 和 10），fn_map 将失败，因为较小的张量中没有足够的元素来配对具有更大的张量元素。

如果您需要运行您的函数针对两个张量的所有排列，您需要找到这些组合并在 fn_map 中提供两个大小相等的张量作为 elems。

编辑：关于组合的更多细节。

无需将两个原始张量输入 tf.map_fn，您可以找到所有成对组合，因此如果您的张量如下所示：

tensor_1 = [1, 2, 3]
tensor_2 = [100, 200, 300]

成对组合的张量将如下所示：

pairwise_combinations = [
    [1, 100],
    [1, 200],
    [1, 300], 
    [2, 100],
    [2, 200],
    [2, 300],
    [3, 100],
    [3, 200],
    [3, 300]
]

现在您可以将此张量的列作为大小相等的 elems 提供给 map_fn。

关于您可以找到这些组合的实际方法，这里是 with a tensorflow example and here 是如何在 python 标准库中完成的。希望对您有所帮助！

使用 Keras 和 Tensorflow 的自定义层

Custom layer with Keras and Tensorflow

python

layer

deep-learning

keras

tensorflow