Tensorflow DrawBoundingBoxes C++

Question

我正在使用 C++ 中的 Tensorflow 检测对象。它确实运行良好，我想绘制方框以获得一些视觉反馈。
有一个操作 tensorflow::ops::DrawBoundingBoxes 可以让我这样做，但问题是：

我不明白这些框的输入值应该是多少。这是什么意思：

boxes: 3-D with shape [batch, num_bounding_boxes, 4] containing bounding boxes.

我在任何地方都找不到在 C++ 中使用此操作的示例，就好像此操作不存在一样。

C++ 中是否有使用此操作的示例？这听起来是教程或调试的基本操作。

Answer 1

下面是python中的一个小用法例子，在image.png上画了2个矩形，保存为outout.png，相信应该对你有帮助：

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import tensorflow as tf
import numpy as np
import PIL.Image as pimg

if __name__ == '__main__':
    image = tf.convert_to_tensor(np.array(pimg.open('image.png'), np.float), tf.float32)
    bbox = tf.convert_to_tensor([[0.1, 0.1, 0.4, 0.4], [0.5, 0.5, 0.6, 0.7]])
    with tf.Session() as s:
        s.run(tf.global_variables_initializer())
        output = s.run(tf.image.draw_bounding_boxes(tf.expand_dims(image, 0), tf.expand_dims(bbox, 0)))
        pimg.fromarray(np.uint8(output[0])).save('output.png')

boxes 是每个图像的矩形数组数组，其中矩形由四个标准化浮点数定义 [min_y、min_x、max_y、max_x]。

Answer 2

我设法让它工作。
太糟糕了，您无法真正控制盒子的外观，也无法控制乐谱或标签文本等其他信息。

Status CreateBoxedTensor(Tensor &input_image, Tensor &input_boxes, Tensor *output) {
    auto root = Scope::NewRootScope();

    // First OP is to convert uint8 image tensor to float for the drawing op to not loose its s*
    Input imgin(input_image);
    auto cast_op = Cast(root, imgin, DT_FLOAT);

    // Next one is the drawing itself
    Input boxin(input_boxes);
    auto draw_op = DrawBoundingBoxes(root, cast_op, boxin);

    // And we convert back to uint8 because it's RGB after all that we want
    auto cast_back_op = Cast(root, draw_op, DT_UINT8);

    ClientSession session(root);
    std::vector<Tensor> out_tensors;
    TF_RETURN_IF_ERROR(session.Run({cast_back_op}, &out_tensors));

    *output = out_tensors[0];
    return Status::OK();
}

Answer 3

如果您仍然对这个问题有疑问，我已经使用 OpenCV 基本方法编写了我自己的此操作的实现。它还支持使用相应的 class 标签为框添加字幕。

/** Draw bounding box and add caption to the image.
 *  Boolean flag _scaled_ shows if the passed coordinates are in relative units (true by default in tensorflow detection)
 */
void drawBoundingBoxOnImage(Mat &image, double yMin, double xMin, double yMax, double xMax, double score, string label, bool scaled=true) {
    cv::Point tl, br;
    if (scaled) {
        tl = cv::Point((int) (xMin * image.cols), (int) (yMin * image.rows));
        br = cv::Point((int) (xMax * image.cols), (int) (yMax * image.rows));
    } else {
        tl = cv::Point((int) xMin, (int) yMin);
        br = cv::Point((int) xMax, (int) yMax);
    }
    cv::rectangle(image, tl, br, cv::Scalar(0, 255, 255), 1);

    // Ceiling the score down to 3 decimals (weird!)
    float scoreRounded = floorf(score * 1000) / 1000;
    string scoreString = to_string(scoreRounded).substr(0, 5);
    string caption = label + " (" + scoreString + ")";

    // Adding caption of type "LABEL (X.XXX)" to the top-left corner of the bounding box
    int fontCoeff = 12;
    cv::Point brRect = cv::Point(tl.x + caption.length() * fontCoeff / 1.6, tl.y + fontCoeff);
    cv::rectangle(image, tl, brRect, cv::Scalar(0, 255, 255), -1);
    cv::Point textCorner = cv::Point(tl.x, tl.y + fontCoeff * 0.9);
    cv::putText(image, caption, textCorner, FONT_HERSHEY_SIMPLEX, 0.4, cv::Scalar(255, 0, 0));
}

/** Draw bounding boxes and add captions to the image.
 *  Box is drawn only if corresponding score is higher than the _threshold_.
 */
void drawBoundingBoxesOnImage(Mat &image,
                              tensorflow::TTypes<float>::Flat scores,
                              tensorflow::TTypes<float>::Flat classes,
                              tensorflow::TTypes<float,3>::Tensor boxes,
                              map<int, string> labelsMap, double threshold=0.5) {
    for (int j = 0; j < scores.size(); j++)
        if (scores(j) > threshold)
            drawBoundingBoxOnImage(image, boxes(0,j,0), boxes(0,j,1), boxes(0,j,2), boxes(0,j,3), scores(j), labelsMap[classes(j)]);
}

完整的例子是here。

Tensorflow DrawBoundingBoxes C++

Tensorflow DrawBoundingBoxes C++

c++

bounding-box

tensorflow