Tensorflow DrawBoundingBoxes C ++

Question

我在C ++中使用Tensorflow检测对象。它确实很好用，我想绘制框以获得一些视觉反馈。 有一个操作tensorflow :: ops :: DrawBoundingBoxes可以让我这样做，但问题是：

• 我不明白框的输入值应该是什么。这是什么意思 ： boxes: 3-D with shape [batch, num_bounding_boxes, 4] containing bounding boxes.
• 我无法在任何地方找到一个在C ++中使用此操作的示例，就像这个ops不存在一样。

C ++中是否有一个使用此操作的示例？对于教程或调试来说，这听起来很基本。

Answer 1

如果你仍然在这个问题上，我已经使用OpenCV基本方法编写了我自己的这个操作的实现。它还支持使用相应的类标签为方框添加字幕。

/** Draw bounding box and add caption to the image.
 *  Boolean flag _scaled_ shows if the passed coordinates are in relative units (true by default in tensorflow detection)
 */
void drawBoundingBoxOnImage(Mat &image, double yMin, double xMin, double yMax, double xMax, double score, string label, bool scaled=true) {
    cv::Point tl, br;
    if (scaled) {
        tl = cv::Point((int) (xMin * image.cols), (int) (yMin * image.rows));
        br = cv::Point((int) (xMax * image.cols), (int) (yMax * image.rows));
    } else {
        tl = cv::Point((int) xMin, (int) yMin);
        br = cv::Point((int) xMax, (int) yMax);
    }
    cv::rectangle(image, tl, br, cv::Scalar(0, 255, 255), 1);

    // Ceiling the score down to 3 decimals (weird!)
    float scoreRounded = floorf(score * 1000) / 1000;
    string scoreString = to_string(scoreRounded).substr(0, 5);
    string caption = label + " (" + scoreString + ")";

    // Adding caption of type "LABEL (X.XXX)" to the top-left corner of the bounding box
    int fontCoeff = 12;
    cv::Point brRect = cv::Point(tl.x + caption.length() * fontCoeff / 1.6, tl.y + fontCoeff);
    cv::rectangle(image, tl, brRect, cv::Scalar(0, 255, 255), -1);
    cv::Point textCorner = cv::Point(tl.x, tl.y + fontCoeff * 0.9);
    cv::putText(image, caption, textCorner, FONT_HERSHEY_SIMPLEX, 0.4, cv::Scalar(255, 0, 0));
}

/** Draw bounding boxes and add captions to the image.
 *  Box is drawn only if corresponding score is higher than the _threshold_.
 */
void drawBoundingBoxesOnImage(Mat &image,
                              tensorflow::TTypes<float>::Flat scores,
                              tensorflow::TTypes<float>::Flat classes,
                              tensorflow::TTypes<float,3>::Tensor boxes,
                              map<int, string> labelsMap, double threshold=0.5) {
    for (int j = 0; j < scores.size(); j++)
        if (scores(j) > threshold)
            drawBoundingBoxOnImage(image, boxes(0,j,0), boxes(0,j,1), boxes(0,j,2), boxes(0,j,3), scores(j), labelsMap[classes(j)]);
}

完整的例子是here。

Answer 2

这是python中的一个小用法示例，它在image.png上绘制2个矩形并将其保存为outout.png，我相信它应该可以帮助您：

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import tensorflow as tf
import numpy as np
import PIL.Image as pimg

if __name__ == '__main__':
    image = tf.convert_to_tensor(np.array(pimg.open('image.png'), np.float), tf.float32)
    bbox = tf.convert_to_tensor([[0.1, 0.1, 0.4, 0.4], [0.5, 0.5, 0.6, 0.7]])
    with tf.Session() as s:
        s.run(tf.global_variables_initializer())
        output = s.run(tf.image.draw_bounding_boxes(tf.expand_dims(image, 0), tf.expand_dims(bbox, 0)))
        pimg.fromarray(np.uint8(output[0])).save('output.png')

boxes是每个图像的矩形阵列数组，其中矩形由四个标准化浮点数[min_y，min_x，max_y，max_x]定义。

Answer 3

我设法让它使用它。 太糟糕了，你无法真正控制盒子的外观，也不能控制分数或标签文本等其他信息。

Status CreateBoxedTensor(Tensor &input_image, Tensor &input_boxes, Tensor *output) {
    auto root = Scope::NewRootScope();

    // First OP is to convert uint8 image tensor to float for the drawing op to not loose its s*
    Input imgin(input_image);
    auto cast_op = Cast(root, imgin, DT_FLOAT);

    // Next one is the drawing itself
    Input boxin(input_boxes);
    auto draw_op = DrawBoundingBoxes(root, cast_op, boxin);

    // And we convert back to uint8 because it's RGB after all that we want
    auto cast_back_op = Cast(root, draw_op, DT_UINT8);

    ClientSession session(root);
    std::vector<Tensor> out_tensors;
    TF_RETURN_IF_ERROR(session.Run({cast_back_op}, &out_tensors));

    *output = out_tensors[0];
    return Status::OK();
}