faster_rcnn_r50 预训练转换为在 Triton 模型服务器中托管的 ONNX

Question

我在此处浏览了 mmdetection 文档以将 pytorch 模型转换为 onnx link

所有安装都是正确的，我正在使用 onnxruntime==1.8.1，ONNX 运行时的自定义运算符 MMCV_WITH_OPS。

我正在使用 configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.py 来获得更快的 rcnn link and using R-5-FPN pretrained model link

我用它来将预训练模型转换为 onnx 并且 成功地 保存了一个名为 fasterrcnn.onnx

的 onnx 文件

python tools/deployment/pytorch2onnx.py \
    configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.py \
    checkpoints/faster_rcnn/faster_rcnn_r50_fpn_mstrain_3x_coco_20210524_110822-e10bd31c.pth \
    --output-file checkpoints/faster_rcnn/fasterrcnn.onnx \
    --input-img demo/demo.jpg \
    --test-img tests/data/color.jpg \
    --shape 608 608 \
    --dynamic-export \
    --cfg-options \
      model.test_cfg.deploy_nms_pre=-1 \

我正在使用该 onnx 文件在 NVIDIA Triton 模型服务器中托管模型。

 fasterrcnn_model | 1       | READY

来自Triton的onnx模型的模型总结如下所示

{
    "name": "fasterrcnn_model",
    "platform": "onnxruntime_onnx",
    "backend": "onnxruntime",
    "version_policy": {
        "latest": {
            "num_versions": 1
        }
    },
    "max_batch_size": 1,
    "input": [
        {
            "name": "input",
            "data_type": "TYPE_FP32",
            "dims": [
                3,
                -1,
                -1
            ]
        }
    ],
    "output": [
        {
            "name": "labels",
            "data_type": "TYPE_INT64",
            "dims": [
                -1
            ]
        },
        {
            "name": "dets",
            "data_type": "TYPE_FP32",
            "dims": [
                -1,
                5
            ]
        }
    ],
    "batch_input": [],
    "batch_output": [],
    "optimization": {
        "priority": "PRIORITY_DEFAULT",
        "input_pinned_memory": {
            "enable": true
        },
        "output_pinned_memory": {
            "enable": true
        },
        "gather_kernel_buffer_threshold": 0,
        "eager_batching": false
    },
    "instance_group": [
        {
            "name": "fasterrcnn_model",
            "kind": "KIND_CPU",
            "count": 1,
            "gpus": [],
            "profile": []
        }
    ],
    "default_model_filename": "model.onnx",
    "cc_model_filenames": {},
    "metric_tags": {},
    "parameters": {},
    "model_warmup": []
}

摘要概述了输出具有类别“labels”和“dets”

在向 triton 发送带有示例图像的推理请求后，我收到以下响应。标签

[[ 0.  1.  2.  3.  4.  5.  6.  7.  8.  9. 10. 11. 12. 13. 14. 15. 16. 17.
  18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35.
  36. 37. 38. 39. 40. 41. 42. 43. 44. 45. 46. 47. 48. 49. 50. 51. 52. 53.
  54. 55. 56. 57. 58. 59. 60. 61. 62. 63. 64. 65. 66. 67. 68. 69. 70. 71.
  72. 73. 74. 75. 76. 77. 78. 79.  0.  1.  2.  3.  4.  5.  6.  7.  8.  9.
  10. 11. 12. 13. 14. 15. 16. 17. 18. 19.]]

dets

[[[-1.0000e+00 -1.0000e+00 -1.0000e+00 -1.0000e+00  0.0000e+00]
  [-3.0000e+02 -3.0000e+02 -3.0000e+02 -3.0000e+02  0.0000e+00]
  [-5.9900e+02 -5.9900e+02 -5.9900e+02 -5.9900e+02  0.0000e+00]
  [-8.9800e+02 -8.9800e+02 -8.9800e+02 -8.9800e+02  0.0000e+00]
  [-1.1970e+03 -1.1970e+03 -1.1970e+03 -1.1970e+03  0.0000e+00]
  [-1.4960e+03 -1.4960e+03 -1.4960e+03 -1.4960e+03  0.0000e+00]
  [-1.7950e+03 -1.7950e+03 -1.7950e+03 -1.7950e+03  0.0000e+00]
  [-2.0940e+03 -2.0940e+03 -2.0940e+03 -2.0940e+03  0.0000e+00]
  [-2.3930e+03 -2.3930e+03 -2.3930e+03 -2.3930e+03  0.0000e+00]
  [-2.6920e+03 -2.6920e+03 -2.6920e+03 -2.6920e+03  0.0000e+00]
  [-2.9910e+03 -2.9910e+03 -2.9910e+03 -2.9910e+03  0.0000e+00]
  [-3.2900e+03 -3.2900e+03 -3.2900e+03 -3.2900e+03  0.0000e+00]
  [-3.5890e+03 -3.5890e+03 -3.5890e+03 -3.5890e+03  0.0000e+00]
  [-3.8880e+03 -3.8880e+03 -3.8880e+03 -3.8880e+03  0.0000e+00]
  [-4.1870e+03 -4.1870e+03 -4.1870e+03 -4.1870e+03  0.0000e+00]
  [-4.4860e+03 -4.4860e+03 -4.4860e+03 -4.4860e+03  0.0000e+00]
  [-4.7850e+03 -4.7850e+03 -4.7850e+03 -4.7850e+03  0.0000e+00]
  [-5.0840e+03 -5.0840e+03 -5.0840e+03 -5.0840e+03  0.0000e+00]
  [-5.3830e+03 -5.3830e+03 -5.3830e+03 -5.3830e+03  0.0000e+00]
  [-5.6820e+03 -5.6820e+03 -5.6820e+03 -5.6820e+03  0.0000e+00]
  [-5.9810e+03 -5.9810e+03 -5.9810e+03 -5.9810e+03  0.0000e+00]
  [-6.2800e+03 -6.2800e+03 -6.2800e+03 -6.2800e+03  0.0000e+00]
  [-6.5790e+03 -6.5790e+03 -6.5790e+03 -6.5790e+03  0.0000e+00]
  [-6.8780e+03 -6.8780e+03 -6.8780e+03 -6.8780e+03  0.0000e+00]
  [-7.1770e+03 -7.1770e+03 -7.1770e+03 -7.1770e+03  0.0000e+00]
  [-7.4760e+03 -7.4760e+03 -7.4760e+03 -7.4760e+03  0.0000e+00]
  [-7.7750e+03 -7.7750e+03 -7.7750e+03 -7.7750e+03  0.0000e+00]
  [-8.0740e+03 -8.0740e+03 -8.0740e+03 -8.0740e+03  0.0000e+00]
  [-8.3730e+03 -8.3730e+03 -8.3730e+03 -8.3730e+03  0.0000e+00]
  [-8.6720e+03 -8.6720e+03 -8.6720e+03 -8.6720e+03  0.0000e+00]
  [-8.9710e+03 -8.9710e+03 -8.9710e+03 -8.9710e+03  0.0000e+00]
  [-9.2700e+03 -9.2700e+03 -9.2700e+03 -9.2700e+03  0.0000e+00]
  [-9.5690e+03 -9.5690e+03 -9.5690e+03 -9.5690e+03  0.0000e+00]
  [-9.8680e+03 -9.8680e+03 -9.8680e+03 -9.8680e+03  0.0000e+00]
  [-1.0167e+04 -1.0167e+04 -1.0167e+04 -1.0167e+04  0.0000e+00]
  [-1.0466e+04 -1.0466e+04 -1.0466e+04 -1.0466e+04  0.0000e+00]
  [-1.0765e+04 -1.0765e+04 -1.0765e+04 -1.0765e+04  0.0000e+00]
  [-1.1064e+04 -1.1064e+04 -1.1064e+04 -1.1064e+04  0.0000e+00]
  [-1.1363e+04 -1.1363e+04 -1.1363e+04 -1.1363e+04  0.0000e+00]
  [-1.1662e+04 -1.1662e+04 -1.1662e+04 -1.1662e+04  0.0000e+00]
  [-1.1961e+04 -1.1961e+04 -1.1961e+04 -1.1961e+04  0.0000e+00]
  [-1.2260e+04 -1.2260e+04 -1.2260e+04 -1.2260e+04  0.0000e+00]
  [-1.2559e+04 -1.2559e+04 -1.2559e+04 -1.2559e+04  0.0000e+00]
  [-1.2858e+04 -1.2858e+04 -1.2858e+04 -1.2858e+04  0.0000e+00]
  [-1.3157e+04 -1.3157e+04 -1.3157e+04 -1.3157e+04  0.0000e+00]
  [-1.3456e+04 -1.3456e+04 -1.3456e+04 -1.3456e+04  0.0000e+00]
  [-1.3755e+04 -1.3755e+04 -1.3755e+04 -1.3755e+04  0.0000e+00]
  [-1.4054e+04 -1.4054e+04 -1.4054e+04 -1.4054e+04  0.0000e+00]
  [-1.4353e+04 -1.4353e+04 -1.4353e+04 -1.4353e+04  0.0000e+00]
  [-1.4652e+04 -1.4652e+04 -1.4652e+04 -1.4652e+04  0.0000e+00]
  [-1.4951e+04 -1.4951e+04 -1.4951e+04 -1.4951e+04  0.0000e+00]
  [-1.5250e+04 -1.5250e+04 -1.5250e+04 -1.5250e+04  0.0000e+00]
  [-1.5549e+04 -1.5549e+04 -1.5549e+04 -1.5549e+04  0.0000e+00]
  [-1.5848e+04 -1.5848e+04 -1.5848e+04 -1.5848e+04  0.0000e+00]
  [-1.6147e+04 -1.6147e+04 -1.6147e+04 -1.6147e+04  0.0000e+00]
  [-1.6446e+04 -1.6446e+04 -1.6446e+04 -1.6446e+04  0.0000e+00]
  [-1.6745e+04 -1.6745e+04 -1.6745e+04 -1.6745e+04  0.0000e+00]
  [-1.7044e+04 -1.7044e+04 -1.7044e+04 -1.7044e+04  0.0000e+00]
  [-1.7343e+04 -1.7343e+04 -1.7343e+04 -1.7343e+04  0.0000e+00]
  [-1.7642e+04 -1.7642e+04 -1.7642e+04 -1.7642e+04  0.0000e+00]
  [-1.7941e+04 -1.7941e+04 -1.7941e+04 -1.7941e+04  0.0000e+00]
  [-1.8240e+04 -1.8240e+04 -1.8240e+04 -1.8240e+04  0.0000e+00]
  [-1.8539e+04 -1.8539e+04 -1.8539e+04 -1.8539e+04  0.0000e+00]
  [-1.8838e+04 -1.8838e+04 -1.8838e+04 -1.8838e+04  0.0000e+00]
  [-1.9137e+04 -1.9137e+04 -1.9137e+04 -1.9137e+04  0.0000e+00]
  [-1.9436e+04 -1.9436e+04 -1.9436e+04 -1.9436e+04  0.0000e+00]
  [-1.9735e+04 -1.9735e+04 -1.9735e+04 -1.9735e+04  0.0000e+00]
  [-2.0034e+04 -2.0034e+04 -2.0034e+04 -2.0034e+04  0.0000e+00]
  [-2.0333e+04 -2.0333e+04 -2.0333e+04 -2.0333e+04  0.0000e+00]
  [-2.0632e+04 -2.0632e+04 -2.0632e+04 -2.0632e+04  0.0000e+00]
  [-2.0931e+04 -2.0931e+04 -2.0931e+04 -2.0931e+04  0.0000e+00]
  [-2.1230e+04 -2.1230e+04 -2.1230e+04 -2.1230e+04  0.0000e+00]
  [-2.1529e+04 -2.1529e+04 -2.1529e+04 -2.1529e+04  0.0000e+00]
  [-2.1828e+04 -2.1828e+04 -2.1828e+04 -2.1828e+04  0.0000e+00]
  [-2.2127e+04 -2.2127e+04 -2.2127e+04 -2.2127e+04  0.0000e+00]
  [-2.2426e+04 -2.2426e+04 -2.2426e+04 -2.2426e+04  0.0000e+00]
  [-2.2725e+04 -2.2725e+04 -2.2725e+04 -2.2725e+04  0.0000e+00]
  [-2.3024e+04 -2.3024e+04 -2.3024e+04 -2.3024e+04  0.0000e+00]
  [-2.3323e+04 -2.3323e+04 -2.3323e+04 -2.3323e+04  0.0000e+00]
  [-2.3622e+04 -2.3622e+04 -2.3622e+04 -2.3622e+04  0.0000e+00]
  [-1.0000e+00 -1.0000e+00 -1.0000e+00 -1.0000e+00  0.0000e+00]
  [-3.0000e+02 -3.0000e+02 -3.0000e+02 -3.0000e+02  0.0000e+00]
  [-5.9900e+02 -5.9900e+02 -5.9900e+02 -5.9900e+02  0.0000e+00]
  [-8.9800e+02 -8.9800e+02 -8.9800e+02 -8.9800e+02  0.0000e+00]
  [-1.1970e+03 -1.1970e+03 -1.1970e+03 -1.1970e+03  0.0000e+00]
  [-1.4960e+03 -1.4960e+03 -1.4960e+03 -1.4960e+03  0.0000e+00]
  [-1.7950e+03 -1.7950e+03 -1.7950e+03 -1.7950e+03  0.0000e+00]
  [-2.0940e+03 -2.0940e+03 -2.0940e+03 -2.0940e+03  0.0000e+00]
  [-2.3930e+03 -2.3930e+03 -2.3930e+03 -2.3930e+03  0.0000e+00]
  [-2.6920e+03 -2.6920e+03 -2.6920e+03 -2.6920e+03  0.0000e+00]
  [-2.9910e+03 -2.9910e+03 -2.9910e+03 -2.9910e+03  0.0000e+00]
  [-3.2900e+03 -3.2900e+03 -3.2900e+03 -3.2900e+03  0.0000e+00]
  [-3.5890e+03 -3.5890e+03 -3.5890e+03 -3.5890e+03  0.0000e+00]
  [-3.8880e+03 -3.8880e+03 -3.8880e+03 -3.8880e+03  0.0000e+00]
  [-4.1870e+03 -4.1870e+03 -4.1870e+03 -4.1870e+03  0.0000e+00]
  [-4.4860e+03 -4.4860e+03 -4.4860e+03 -4.4860e+03  0.0000e+00]
  [-4.7850e+03 -4.7850e+03 -4.7850e+03 -4.7850e+03  0.0000e+00]
  [-5.0840e+03 -5.0840e+03 -5.0840e+03 -5.0840e+03  0.0000e+00]
  [-5.3830e+03 -5.3830e+03 -5.3830e+03 -5.3830e+03  0.0000e+00]
  [-5.6820e+03 -5.6820e+03 -5.6820e+03 -5.6820e+03  0.0000e+00]]]

labels 响应看起来像普通的 COCO 类 (80) 但我很难解码 dets 回复。这看起来像边界框坐标 4 和置信度阈值 1。制作形状 (1,100,5)。知道 dets 类别应该代表什么吗？输出通常取决于模型本身，但我认为 onnx 转换正在将输出更改为 labels 和 dets

Answer 1

看转换脚本好像dets是box加score的组合

boxes (Tensor): The bounding boxes of shape [N, num_boxes, 4]
scores (Tensor): The detection scores of shape [N, num_boxes, num_classes]
tuple[Tensor, Tensor]: dets of shape [N, num_det, 5]
            and class labels of shape [N, num_det].
dets = torch.cat([boxes, scores], dim=2)

https://github.com/open-mmlab/mmdetection/blob/master/mmdet/core/export/onnx_helper.py#L197

faster_rcnn_r50 预训练转换为在 Triton 模型服务器中托管的 ONNX

faster_rcnn_r50 pretrained converted to ONNX hosted in Triton model server

nvidia

onnx

onnxruntime

tritonserver