tensorflow openvino ssd-mobilenet coco自定义数据集错误输入层

Question

所以，我正在使用 TensorFlow SSD-Mobilnet V1 coco 数据集。我已经对自己的数据集进行了进一步训练，但是当我尝试使用 Movidius 芯片将其转换为 OpenVino IR 到运行时，它在 Raspberry PI 上。我得到一个错误

➜  utils sudo python3 summarize_graph.py --input_model ssd.pb 
WARNING: Logging before flag parsing goes to stderr.
W0722 17:17:05.565755 4678620608 __init__.py:308] Limited tf.compat.v2.summary API due to missing TensorBoard installation.
W0722 17:17:06.696880 4678620608 deprecation_wrapper.py:119] From ../../mo/front/tf/loader.py:35: The name tf.GraphDef is deprecated. Please use tf.compat.v1.GraphDef instead.

W0722 17:17:06.697348 4678620608 deprecation_wrapper.py:119] From ../../mo/front/tf/loader.py:109: The name tf.MetaGraphDef is deprecated. Please use tf.compat.v1.MetaGraphDef instead.

W0722 17:17:06.697680 4678620608 deprecation_wrapper.py:119] From ../../mo/front/tf/loader.py:235: The name tf.NodeDef is deprecated. Please use tf.compat.v1.NodeDef instead.

1 input(s) detected:
Name: image_tensor, type: uint8, shape: (-1,-1,-1,3)
7 output(s) detected:
detection_boxes
detection_scores
detection_multiclass_scores
detection_classes
num_detections
raw_detection_boxes
raw_detection_scores

当我尝试将 ssd.pb（冻结模型）转换为 OpenVino IR

➜  model_optimizer sudo python3 mo_tf.py --input_model ssd.pb          
Password:
Model Optimizer arguments:
Common parameters:
    - Path to the Input Model:  /opt/intel/openvino_2019.1.144/deployment_tools/model_optimizer/ssd.pb
    - Path for generated IR:    /opt/intel/openvino_2019.1.144/deployment_tools/model_optimizer/.
    - IR output name:   ssd
    - Log level:    ERROR
    - Batch:    Not specified, inherited from the model
    - Input layers:     Not specified, inherited from the model
    - Output layers:    Not specified, inherited from the model
    - Input shapes:     Not specified, inherited from the model
    - Mean values:  Not specified
    - Scale values:     Not specified
    - Scale factor:     Not specified
    - Precision of IR:  FP32
    - Enable fusing:    True
    - Enable grouped convolutions fusing:   True
    - Move mean values to preprocess section:   False
    - Reverse input channels:   False
TensorFlow specific parameters:
    - Input model in text protobuf format:  False
    - Path to model dump for TensorBoard:   None
    - List of shared libraries with TensorFlow custom layers implementation:    None
    - Update the configuration file with input/output node names:   None
    - Use configuration file used to generate the model with Object Detection API:  None
    - Operations to offload:    None
    - Patterns to offload:  None
    - Use the config file:  None
Model Optimizer version:    2019.1.1-83-g28dfbfd
WARNING: Logging before flag parsing goes to stderr.
E0722 17:24:22.964164 4474824128 infer.py:158] Shape [-1 -1 -1  3] is not fully defined for output 0 of "image_tensor". Use --input_shape with positive integers to override model input shapes.
E0722 17:24:22.964462 4474824128 infer.py:178] Cannot infer shapes or values for node "image_tensor".
E0722 17:24:22.964554 4474824128 infer.py:179] Not all output shapes were inferred or fully defined for node "image_tensor". 
 For more information please refer to Model Optimizer FAQ (<INSTALL_DIR>/deployment_tools/documentation/docs/MO_FAQ.html), question #40. 
E0722 17:24:22.964632 4474824128 infer.py:180] 
E0722 17:24:22.964720 4474824128 infer.py:181] It can happen due to bug in custom shape infer function <function tf_placeholder_ext.<locals>.<lambda> at 0x12ab64bf8>.
E0722 17:24:22.964787 4474824128 infer.py:182] Or because the node inputs have incorrect values/shapes.
E0722 17:24:22.964850 4474824128 infer.py:183] Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
E0722 17:24:22.965915 4474824128 infer.py:192] Run Model Optimizer with --log_level=DEBUG for more information.
E0722 17:24:22.966033 4474824128 main.py:317] Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "image_tensor" node. 
 For more information please refer to Model Optimizer FAQ (<INSTALL_DIR>/deployment_tools/documentation/docs/MO_FAQ.html), question #38.

您认为我们应该如何解决这个问题？

Answer 1

当您尝试转换 ssd.pb（您的冻结模型）时，您只是将输入模型参数传递给 mo_tf.py 脚本。要将对象检测模型转换为 IR，请转到到模型优化器目录，运行 mo_tf.py 具有以下必需参数的脚本：

--input_model :
带有预训练模型的文件（冻结后的二进制或文本 .pb 文件）

--tensorflow_use_custom_operations_config : 描述转换特定 TensorFlow* 拓扑的规则的配置文件。对于从 TensorFlow* Object Detection API zoo 下载的模型，您可以在 /deployment_tools/model_optimizer/extensions/front/tf 目录中找到配置文件您可以使用 ssd_v2_support.json / ssd_support.json — 用于模型动物园中的冻结 SSD 拓扑。它将在上述目录中可用。

--tensorflow_object_detection_api_pipeline_config : 描述 TensorFlow 对象检测 API 模型的拓扑超参数和结构的特殊配置文件。对于从 TensorFlow* Object Detection API zoo 下载的模型，配置文件名为 pipeline.config。如果您打算自己训练模型，可以在模型存储库中找到这些文件的模板

--input_shape（可选）：自定义输入图像形状，我们需要根据您使用的预训练模型传递这些值。该模型以 [1 H W C] 格式输入图像，其中参数分别指批量大小、高度、宽度、通道。模型优化器不接受批次、高度、宽度和通道数的负值。因此，如果预先知道模型（SSD mobilenet）的输入图像尺寸，则需要使用 --input_shape 参数传递一组有效的 4 个正数。
如果不可用，则不需要传递输入形状。

使用从模型下载器下载的模型 SSD-MobileNet-v2-COCO 的示例mo_tf.py命令如下所示。

python mo_tf.py  
              --input_model "c:\Program Files (x86)\IntelSWTools\openvino_2019.1.087\deployment_tools\tools\model_downloader\object_detection\common\ssd_mobilenet_v2_coco\tf\ssd_mobilenet_v2_coco.frozen.pb" 
              --tensorflow_use_custom_operations_config  "c:\Program Files (x86)\IntelSWTools\openvino_2019.1.087\deployment_tools\model_optimizer\extensions\front\tf\ssd_v2_support.json"  
              --tensorflow_object_detection_api_pipeline_config  "c:\Program Files (x86)\IntelSWTools\openvino_2019.1.087\deployment_tools\tools\model_downloader\object_detection\common\ssd_mobilenet_v2_coco\tf\ssd_mobilenet_v2_coco.config" 
              --data_type FP16 
              --log_level DEBUG

详情请参考link https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_Object_Detection_API_Models.html
希望对你有帮助。

Answer 2

我将我的 OpenVINO 更新为 OpenVINO 工具包 R2 2019 & 使用以下命令我能够生成 IR 文件

python3 ~/intel/openvino/deployment_tools/model_optimizer/mo_tf.py --input_model frozen_inference_graph.pb --tensorflow_use_custom_operations_config ~/intel/openvino/deployment_tools/model_optimizer/extension/front/tf/ssd_support_api_v1.14.json --tensorflow_object_detection_api_pipeline_config pipeline.config -b 1 --data_type FP16 --reverse_input_channels

Answer 3

要转换 mobilenetv2 ssd，请在原始 ssd_v2_support.json 中添加 "Postprocessor/Cast_1" 并使用以下命令。它应该可以正常工作。

"instances": {
            "end_points": [
                "detection_boxes",
                "detection_scores",
                "num_detections"
            ],
            "start_points": [
                "Postprocessor/Shape",
                "Postprocessor/scale_logits",
                "Postprocessor/Tile",
                "Postprocessor/Reshape_1",
                "Postprocessor/Cast_1"
            ]
        },

然后使用以下命令

#### object detection conversion

import platform
is_win = 'windows' in platform.platform().lower()

mo_tf_path = '/opt/intel/openvino/deployment_tools/model_optimizer/mo_tf.py'
json_file = '/opt/intel/openvino/deployment_tools/model_optimizer/extensions/front/tf/ssd_v2_support.json'
pb_file =          'model/frozen_inference_graph.pb'
pipeline_file =       'model/pipeline.config'
output_dir =       'output/'

img_height = 300
input_shape = [1,img_height,img_height,3]
input_shape_str = str(input_shape).replace(' ','')
input_shape_str

!python3 {mo_tf_path} --input_model {pb_file}  --tensorflow_object_detection_api_pipeline_config {pipeline_file} --tensorflow_use_custom_operations_config {json_file} --output="detection_boxes,detection_scores,num_detections" --output_dir {output_dir} --reverse_input_channels --data_type FP16 --log_level DEBUG

tensorflow openvino ssd-mobilenet coco自定义数据集错误输入层

tensorflow openvino ssd-mobilnet coco custom dataset error input layer

python-3.x

tensorflow

raspberry-pi

openvino

movidius