Converting an opencv image cv::Mat Format to a C# BitmapImage

Question

I wrote this sample code to explain my problem. I have a solution in VS 2013, contains one C# project and a C++ project. I try to read an image with OpenCV in C++ (x86). and want to pass in to a C# x86 project (used CLR mode) to a Bitmap Object and Then BitmapImage Object to use as a WPF ImageSource.
My C++ Code:

Bitmap^ SomeClass::Test(System::String^ imgFileName)
{
    auto fileName = msclr::interop::marshal_as<string>(imgFileName);
    Mat img = imread(fileName);
    //Do something
    auto bmp = gcnew Bitmap(img.cols, img.rows, img.step, Imaging::PixelFormat::Format24bppRgb, (IntPtr)img.data);
    bmp->Save("InC++Side.png");     
    return bmp;
}

My C# Code:

private void ImageTester(object sender, RoutedEventArgs e)
{
    var image = testClass.Test("test.png");
    image.Save("InC#Side.png");
    bg.Source = ConvertToBitmapImageFromBitmap(image);
}
public static BitmapImage ConvertToBitmapImageFromBitmap(Bitmap image)
{
    using(var ms = new MemoryStream())
    {
        image.Save(ms, System.Drawing.Imaging.ImageFormat.Png);
        BitmapImage bImg = new BitmapImage();
        bImg.BeginInit();
        bImg.StreamSource = new MemoryStream(ms.ToArray());
        bImg.EndInit();
        return bImg;
    }
}

Problem is that the file saved by C++ (InC++Side.png) is perfect; but the other one which is presented the Bitmap object in C# is just a gray rectangle with that image's Height and Width.
Where is the problem?
How I can pass the Image to my C# project?

What do you mean with "pass the image to C# project"? Do you create one single application (that is a C# project) and use the OpenCV C++ API in order to load the image from file? — enzom83, Jul 20 '15 at 15:26
@enzom83: Yes. I have a solution which is has one C# project to display an image on the UI and a C++ project to use OpenCV. I do not use OpenCV just for loading an image from file in my original project; but in this sample one, yes. I just load the image. — Zhr Saghaie, Jul 21 '15 at 04:14
You could convert `Mat` to `byte` array (see [this answer](http://answers.opencv.org/question/33596/convert-mat-to-byte-in-c/?answer=33603#post-id-33603)), then try to use the procedure described in [this answer](http://stackoverflow.com/a/10214278). Moreover, have you tried to change `(IntPtr)img.data` to `img.data.ptr`? — enzom83, Jul 21 '15 at 07:21

score 2 · Answer 1 · answered Jul 08 '16 at 11:23

I see it's quite an old question at this point, but I just encountered a similar task. Rather than keeping the unmanaged resource alive and use it in managed code, I believe it's better to copy data to the managed heap and let CLR handle it. Here is my approach:

System::Windows::Media::ImageSource^ ManagedCppClass::GetLatestImage()
{
    cv::Mat frame;
    if (!videoSrc->read(frame))
        return nullptr;

    int totSize = frame.total() * frame.elemSize();
    auto managedArray = gcnew array<System::Byte>(totSize);
    System::Runtime::InteropServices::Marshal::Copy(System::IntPtr((void*)frame.data), 
                                                    managedArray, 0, totSize); //assume isContinuous is always true at this point (?)

    return System::Windows::Media::Imaging::BitmapImage::Create(
                frame.cols, frame.rows, 96, 96, System::Windows::Media::PixelFormats::Bgr24, //Rgb24,
                System::Windows::Media::Imaging::BitmapPalettes::WebPalette, managedArray, frame.step);
}

Maybe it will help someone who stumbles upon it in the future.

score 0 · Accepted Answer · answered Jul 25 '15 at 05:40

It seems that the problem is about the shared memory.
Mat and Bitmap are share the memory in C++. So, when the Mat object destroys, the Bitmap object can't access to the data. That's why its data is correct in C++ side but have nothing inside in C# side.
To solving this problem, I used static Mat. It'll never release, but can solve my problem.

Converting an opencv image cv::Mat Format to a C# BitmapImage

2 Answers2

Linked