从NumPy数组到Mat(OpenCV)的C ++转换

问题描述 投票:1回答:2

我正在围绕ArUco增强现实库(基于OpenCV)编写一个薄的包装器。我想要构建的接口非常简单:

  • Python将图像传递给C ++代码;
  • C ++代码检测标记并将其位置和其他信息作为dicts元组返回给Python。

但是,我无法弄清楚如何在Python中表示图像以将其传递给C ++。对于GUI和相机管理,我将使用PyQt,所以最初它将是QImage,但我不能简单地将它传递给OpenCV(或者我可以?)。起初,我尝试使用嵌套元组来表示每个像素的行,列和颜色,所以我最终得到了这个示例代码:

using namespace cv;
namespace py = boost::python;

void display(py::tuple pix)
{
    /*
        Receive image from Python and display it.
    */
    Mat img(py::len(pix), py::len(pix[0]), CV_8UC3, Scalar(0, 0, 255));
    for (int y = 0; y < py::len(pix); y++)
        for (int x = 0; x < py::len(pix[y]); x++)
        {
            Vec3b rgb;
            for (int i = 0; i < 3; i++)
                rgb[i] = py::extract<int>(pix[y][x][i]);
            img.at<Vec3b>(Point(x, y)) = rgb;
        }
    imshow("Image", img);
    waitKey(0);
}

BOOST_PYTHON_MODULE(aruco)
{
    py::def("display", display);
}

结果是痛苦的慢(单帧几秒钟),所以我去google搜索并找到了应该更快的解决方案:使用NumPy数组,所以代码看起来像这样:

void display(py::object array)
{
    Mat img;
    // ... some magic here to convert NumPy array to Mat ...
    imshow("Image", img);
    waitKey(0);
}

但是,我不知道如何将NumPy数组(在C ++级别只是一个Python对象)转换为OpenCV Mat。我很感激这里的任何帮助。

或者,也许真的不需要NumPy,所以我可以直接将QImage Python对象传递给C ++层?或者可能有不同的方法解决这个问题?任何建议表示赞赏!

python c++ opencv numpy boost-python
2个回答
4
投票

您遇到的最佳解决方案是为cv :: Mat对象使用自定义boost :: python转换器。 OpenCV有Python包装器,当你使用这个包装器时,你在Numpy数组上运行 - 你甚至不需要知道那些数组在“穿越c ++ < - > python border”时被转换为cv :: Mat对象。为简单类型编写这样的转换器非常容易,但是为cv :: Mat创建转换器并不简单。幸运的是其他人已经这样做了 - here是OpenCV 2.x和here for 3.x的版本。如果你不熟悉boost :: python转换器,this article应该会帮助你。 希望它有所帮助,如果您有任何问题,请告诉我们。


2
投票

或者,如果您不喜欢使用包装器,并且想要使用本机python扩展模块,则可以这样做。

python3:

my_image = cv.imread("my_image.jpg", 1)  # reads colorfull image in python
dims = my_image.shape  # get image shape (h, w, c)
my_image = my_image.ravel()  # flattens 3d array into 1d
cppextenionmodule.np_to_mat(dims, my_image)

C ++:

static PyObject *np_to_mat(PyObject *self, PyObject *args){
    PyObject *size;
    PyArrayObject *image;

    if (!PyArg_ParseTuple(args, "O!O!", &PyTuple_Type, &size, &PyArray_Type, &image)) {
        return NULL;
    }
    int rows = PyLong_AsLong(PyTuple_GetItem(size ,0));
    int cols = PyLong_AsLong(PyTuple_GetItem(size ,1));
    int nchannels = PyLong_AsLong(PyTuple_GetItem(size ,2));
    char my_arr[rows * nchannels * cols];

    for(size_t length = 0; length<(rows * nchannels * cols); length++){
        my_arr[length] = (*(char *)PyArray_GETPTR1(image, length));
    }

    cv::Mat my_img = cv::Mat(cv::Size(cols, rows), CV_8UC3, &my_arr);

    ... whatever with the image
}

1
投票

我写了这个例子,谁不知道有Boost Numpy模块。您可以看到如何将Mat转换为NDArray,反之亦然。它会让你了解转换ndarray的方式。

#define BOOST_PYTHON_STATIC_LIB
#define BOOST_LIB_NAME "boost_numpy35"
//#include <boost/config/auto_link.hpp>
#include <boost/python.hpp>
#include <boost/python/numpy.hpp>
#include <iostream>
#include <opencv2/opencv.hpp>

namespace py = boost::python;
namespace np = boost::python::numpy;

void Init() {
    // set your python location.
    wchar_t str[] = L"D:\\Anaconda3\\envs\\tensorflow_vision";

    Py_SetPythonHome(str);

    Py_Initialize();
    np::initialize();
}

np::ndarray ConvertMatToNDArray(const cv::Mat& mat) {
    py::tuple shape = py::make_tuple(mat.rows, mat.cols, mat.channels());
    py::tuple stride = py::make_tuple(mat.channels() * mat.cols * sizeof(uchar), mat.channels() * sizeof(uchar), sizeof(uchar));
    np::dtype dt = np::dtype::get_builtin<uchar>();
    np::ndarray ndImg = np::from_data(mat.data, dt, shape, stride, py::object());

    return ndImg;
}

cv::Mat ConvertNDArrayToMat(const np::ndarray& ndarr) {
    //int length = ndarr.get_nd(); // get_nd() returns num of dimensions. this is used as a length, but we don't need to use in this case. because we know that image has 3 dimensions.
    const Py_intptr_t* shape = ndarr.get_shape(); // get_shape() returns Py_intptr_t* which we can get the size of n-th dimension of the ndarray.
    char* dtype_str = py::extract<char *>(py::str(ndarr.get_dtype()));

    // variables for creating Mat object
    int rows = shape[0];
    int cols = shape[1];
    int channel = shape[2];
    int depth;

    // you should find proper type for c++. in this case we use 'CV_8UC3' image, so we need to create 'uchar' type Mat.
    if (!strcmp(dtype_str, "uint8")) {
        depth = CV_8U;
    }
    else {
        std::cout << "wrong dtype error" << std::endl;
        return cv::Mat();
    }

    int type = CV_MAKETYPE(depth, channel); // CV_8UC3

    cv::Mat mat = cv::Mat(rows, cols, type);
    memcpy(mat.data, ndarr.get_data(), sizeof(uchar) * rows * cols * channel);

    return mat;
}

int main()
{
    using namespace std;

    try
    {
        // initialize boost python and numpy
        Init();

        // import module
        py::object main_module = py::import("__main__");
        py::object print = main_module.attr("__builtins__").attr("print"); // this is for printing python object

        // get image
        cv::Mat img;
        img = cv::imread("Lenna.jpg", cv::IMREAD_COLOR);
        if (img.empty())
        {
            std::cout << "can't getting image" << std::endl;
            return -1;
        }

        // convert Mat to NDArray
        cv::Mat cloneImg = img.clone(); // converting functions will access to same data between Mat and NDArray. so we should clone Mat object. This may important in your case.
        np::ndarray ndImg = ConvertMatToNDArray(cloneImg);

        // You can check if it's properly converted.
        //print(ndImg);

        // convert NDArray to Mat
        cv::Mat matImg = ConvertNDArrayToMat(ndImg); // also you can convert ndarray to mat.

        // add 10 brightness to converted image
        for (int i = 0; i < matImg.rows; i++) {
            for (int j = 0; j < matImg.cols; j++) {
                for (int c = 0; c < matImg.channels(); c++) {
                    matImg.at<cv::Vec3b>(i, j)[c] += 10;
                }
            }
        }

        // show image
        cv::imshow("original image", img);
        cv::imshow("converted image", matImg);
        cv::waitKey(0);
        cv::destroyAllWindows();
    }
    catch (py::error_already_set&)
    {
        PyErr_Print();
        system("pause");
    }

    system("pause");
    return 0;
}

推荐问答