使用 Tensorflow 对象检测在 Faster R-CNN 中可视化来自 RPN 头部的提案区域 API

Question

我正在尝试使用 Tensorflow 对象检测 API 调试训练有素的 Faster R-CNN 模型，我想在图像上可视化 RPN 的提议区域。谁能告诉我该怎么做？我找到了一个 post here but it hasn't been answered. I tried to export the model using exporter_main_v2.py with only the RPN head as said 这是我删除 second_stage.

时的消息

Traceback (most recent call last):
  File "exporter_main_v2.py", line 165, in <module>
    app.run(main)
  File "E:\Anaconda\envs\TFOD\lib\site-packages\absl\app.py", line 312, in run
    _run_main(main, args)
  File "E:\Anaconda\envs\TFOD\lib\site-packages\absl\app.py", line 258, in _run_main
    sys.exit(main(argv))
  File "exporter_main_v2.py", line 158, in main
    exporter_lib_v2.export_inference_graph(
  File "E:\Anaconda\envs\TFOD\lib\site-packages\object_detection\exporter_lib_v2.py", line 245, in export_inference_graph
    detection_model = INPUT_BUILDER_UTIL_MAP['model_build'](
  File "E:\Anaconda\envs\TFOD\lib\site-packages\object_detection\builders\model_builder.py", line 1226, in build
    return build_func(getattr(model_config, meta_architecture), is_training,
  File "E:\Anaconda\envs\TFOD\lib\site-packages\object_detection\builders\model_builder.py", line 665, in _build_faster_rcnn_model
    second_stage_box_predictor = box_predictor_builder.build_keras(
  File "E:\Anaconda\envs\TFOD\lib\site-packages\object_detection\builders\box_predictor_builder.py", line 991, in build_keras
    raise ValueError(
ValueError: Unknown box predictor for Keras: None

我再次尝试在不删除 second_stage 的情况下导出模型。这是我收到的消息

INFO:tensorflow:depth of additional conv before box predictor: 0
I0802 20:55:13.930429  1996 convolutional_keras_box_predictor.py:153] depth of additional conv before box predictor: 0
Traceback (most recent call last):
  File "exporter_main_v2.py", line 165, in <module>
    app.run(main)
  File "E:\Anaconda\envs\TFOD\lib\site-packages\absl\app.py", line 312, in run
    _run_main(main, args)
  File "E:\Anaconda\envs\TFOD\lib\site-packages\absl\app.py", line 258, in _run_main
    sys.exit(main(argv))
  File "exporter_main_v2.py", line 158, in main
    exporter_lib_v2.export_inference_graph(
  File "E:\Anaconda\envs\TFOD\lib\site-packages\object_detection\exporter_lib_v2.py", line 271, in export_inference_graph
    concrete_function = detection_module.__call__.get_concrete_function()
  File "E:\Anaconda\envs\TFOD\lib\site-packages\tensorflow\python\eager\def_function.py", line 1299, in get_concrete_function
    concrete = self._get_concrete_function_garbage_collected(*args, **kwargs)
  File "E:\Anaconda\envs\TFOD\lib\site-packages\tensorflow\python\eager\def_function.py", line 1205, in _get_concrete_function_garbage_collected
    self._initialize(args, kwargs, add_initializers_to=initializers)
  File "E:\Anaconda\envs\TFOD\lib\site-packages\tensorflow\python\eager\def_function.py", line 725, in _initialize
    self._stateful_fn._get_concrete_function_internal_garbage_collected(  # pylint: disable=protected-access
  File "E:\Anaconda\envs\TFOD\lib\site-packages\tensorflow\python\eager\function.py", line 2969, in _get_concrete_function_internal_garbage_collected
    graph_function, _ = self._maybe_define_function(args, kwargs)
  File "E:\Anaconda\envs\TFOD\lib\site-packages\tensorflow\python\eager\function.py", line 3361, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "E:\Anaconda\envs\TFOD\lib\site-packages\tensorflow\python\eager\function.py", line 3196, in _create_graph_function
    func_graph_module.func_graph_from_py_func(
  File "E:\Anaconda\envs\TFOD\lib\site-packages\tensorflow\python\framework\func_graph.py", line 990, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "E:\Anaconda\envs\TFOD\lib\site-packages\tensorflow\python\eager\def_function.py", line 634, in wrapped_fn
    out = weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "E:\Anaconda\envs\TFOD\lib\site-packages\tensorflow\python\framework\func_graph.py", line 977, in wrapper
    raise e.ag_error_metadata.to_exception(e)
tensorflow.python.autograph.pyct.error_utils.KeyError: in user code:

    E:\Anaconda\envs\TFOD\lib\site-packages\object_detection\exporter_lib_v2.py:163 call_func  *
        return self._run_inference_on_images(images, true_shapes, **kwargs)
    E:\Anaconda\envs\TFOD\lib\site-packages\object_detection\exporter_lib_v2.py:129 _run_inference_on_images  *
        detections[classes_field] = (

    KeyError: 'detection_classes'

Answer 1

找到解决方案！
在配置文件中添加 number_of_stages: 1
我没有使用 exporter_main_v2.py，而是编写了从检查点文件

构建模型的代码

# Load pipeline config and build a detection model
configs = config_util.get_configs_from_pipeline_file(path_to_config)
model_config = configs['model']
detection_model = model_builder.build(model_config=model_config, is_training=False)

# Restore checkpoint
ckpt = tf.compat.v2.train.Checkpoint(model=detection_model)
ckpt.restore(os.path.join(path_to_ckpt, 'ckpt-0')).expect_partial()

然后我将我需要检查的图像提供给模型，然后我使用 object_detection.utils.visualization_utils.visualize_boxes_and_labels_on_image_array 检查框

使用 Tensorflow 对象检测在 Faster R-CNN 中可视化来自 RPN 头部的提案区域 API

Visualize proposal regions from RPN head in Faster R-CNN with Tensorflow Object Detection API

tensorflow

object-detection-api

faster-rcnn

tensorflow-model-garden