使用三元组损失和预训练的 resnet 时形状不兼容

Question

我正在尝试使用预训练的 resnet 并使用三元组损失对其进行微调。我想出的以下代码是我在该主题上找到的教程的组合：

import pathlib
import tensorflow as tf
import tensorflow_addons as tfa


with tf.device('/cpu:0'):
    INPUT_SHAPE = (32, 32, 3)
    BATCH_SIZE = 16
    data_dir = pathlib.Path('/home/user/dataset/')

    base_model = tf.keras.applications.ResNet50V2(
        weights='imagenet',
        pooling='avg',
        include_top=False,
        input_shape=INPUT_SHAPE,
    )

    # following two lines are added after edit, originally it was model = base_model
    head_model = tf.keras.layers.Lambda(lambda x: tf.math.l2_normalize(x, axis=1))(base_model.output)
    model = tf.keras.Model(inputs=base_model.input, outputs=head_model)

    datagen = tf.keras.preprocessing.image.ImageDataGenerator(
        rotation_range=10,
        zoom_range=0.1,
    )

    generator = datagen.flow_from_directory(
        data_dir,
        target_size=INPUT_SHAPE[:2],
        batch_size=BATCH_SIZE,
        seed=42,
    )

    model.compile(
        optimizer=tf.keras.optimizers.Adam(0.001),
        loss=tfa.losses.TripletSemiHardLoss(),
    )

    model.fit(
        generator,
        epochs=5,
    )

不幸的是，在运行代码之后我得到以下错误：

Found 4857 images belonging to 83 classes.
Epoch 1/5
Traceback (most recent call last):
  File "ReID/external_process.py", line 35, in <module>
    model.fit(
  File "/home/user/videolytics/venv_python/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py", line 108, in _method_wrapper
    return method(self, *args, **kwargs)
  File "/home/user/videolytics/venv_python/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py", line 1098, in fit
    tmp_logs = train_function(iterator)
  File "/home/user/videolytics/venv_python/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 780, in __call__
    result = self._call(*args, **kwds)
  File "/home/user/videolytics/venv_python/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 840, in _call
    return self._stateless_fn(*args, **kwds)
  File "/home/user/videolytics/venv_python/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 2829, in __call__
    return graph_function._filtered_call(args, kwargs)  # pylint: disable=protected-access
  File "/home/user/videolytics/venv_python/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 1843, in _filtered_call
    return self._call_flat(
  File "/home/user/videolytics/venv_python/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 1923, in _call_flat
    return self._build_call_outputs(self._inference_function.call(
  File "/home/user/videolytics/venv_python/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 545, in call
    outputs = execute.execute(
  File "/home/user/videolytics/venv_python/lib/python3.8/site-packages/tensorflow/python/eager/execute.py", line 59, in quick_execute
    tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InvalidArgumentError:  Input to reshape is a tensor with 1328 values, but the requested shape has 16
     [[{{node TripletSemiHardLoss/PartitionedCall/Reshape}}]] [Op:__inference_train_function_13749]

Function call stack:
train_function

2020-10-23 22:07:09.094736: W tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated.
     [[{{node PyFunc}}]]

dataset 目录有 83 个子目录，每个 class 一个，每个子目录包含给定 class 的图像。错误输出中的维度 1328 是批量大小 (16) 乘以 classes 的数量 (83)，维度 16 是批量大小（如果我更改 BATCH_SIZE，两个维度都会相应更改）。

老实说，我不太理解这个错误，所以非常感谢任何解决方案，甚至是任何一种洞察力，问题出在哪里。

Answer 1

问题是 TripletSemiHardLoss 期望

labels y_true to be provided as 1-D integer Tensor with shape [batch_size] of multi-class integer labels

但 flow_from_directory 默认生成 categorical 标签；使用 class_mode="sparse" 应该可以解决问题。

使用三元组损失和预训练的 resnet 时形状不兼容

Incompatible shapes while using triplet loss and pre-trained resnet

tensorflow

resnet