How to convert OpenCV image data from Python to C++?

Question

How can I pass an OpenCV 3 image from Python 3 to C++? In technical terms, how do I convert a NumPy array to a cv::Mat object and back?

To make my question more specific, let's assume:

1. I have a Python program that uses OpenCV to capture images from the webcam and display them. (In real life, I'm prototyping further functions there, such as some web server features.)

   cam.py:

   import cv2
   
   def main():
       cam = cv2.VideoCapture(0)
       while True:
           _, img = cam.read()
           cv2.imshow('cam', img)
   
           # TODO: Call the C++ function from the sample below 
           # img = helloWorld(img) 
   
           cv2.waitKey(16) # let imshow do its work (16 ms -> 60 FPS)
       cv2.destroyAllWindows()
   
   if __name__ == '__main__':
       main()
   

   Run python3 cam.py to try it out.

2. I would like to use an existing helloWorld function from some C++ library that processes the OpenCV images. (In real life, it does some signal processing and extracts information from the images.) For the sake of simplicity, my sample function below just mirrors the image horizontally. The main function is there to check that helloWorld works as expected.

   cam.cpp:

   #include <opencv2/highgui/highgui.hpp>
   
   using namespace cv;
   using namespace std;
   
   void helloWorld(Mat img) {
       flip(img, img, 1);
   }
   
   int main() {
       VideoCapture stream1(0);
   
       while (true) {
           Mat img;
           stream1.read(img);
   
           helloWorld(img);
   
           imshow("cam", img);
           waitKey(16);
       }
   
       return 0;
   }
   

   To compile this code, run g++ -std=c++11 -lopencv_core -lopencv_highgui -lopencv_videoio cam.cpp -o cam, then execute ./cam to try it out.

Now what changes should I make to my code so that I can call the helloWorld C++ function out of my Python app? Luckily, I have access to the C++ source code, so that I can extend and recompile it if necessary.

Answer 1

See Artanis's gist Python C Extension Hello World for getting started with the creation of a module allowing you to call C code out of Python.

For converting between the Python 3 NumPy array and the C++ Mat, there are at least 2 places where you can find sample C++ code:

1. The file cv2.cpp on the OpenCV repository contains functions that are not exposed in its API:

   • bool pyopencv_to(PyObject* o, UMat& um, const char* name) and

   • PyObject* pyopencv_from(const Mat& m).

2. pyboostcvconverter provides similar standalone converter functions

   • PyObject* fromMatToNDArray(const Mat& m) and
   • Mat fromNDArrayToMat(PyObject* o).

Before calling any of these, make sure you call the NumPy function import_array() once to avoid a Segmentation Fault: 11 at runtime.

I've put it all together in a code sample on GitHub. Python grabs an image from the camera, passes it to the C++ module, then C++ mirrors the image and passes it back to Python, and finally Python displays the image.

Answer 2

Here you have to do inter process comunication, since you have the python process and the c++ process running simultaneoulsy, so, to achive what you want, instead of calling a function in the C++ program, just read from a shared memory pagefile where you put the mat.data() and you retrieve that in your C++ program, just make sure that the shared memory zone is created before trying to access it.

If you really want to call a c++ function as it, you may have to consider compiling your C++ source function/code into a dll, load that dll at runtime in python, and calling that function after that.