Decode H264 frame to Bitmap

Question

I am receiving H264 encoded frame but when I convert it into bitmap I just get a black screen. The resolution is right. I have tried a lot of things and couldnt find a working way. Thank you! Here is my code

        public System.Drawing.Bitmap CopyDataToBitmap(byte[] data)
        {
            //Here create the Bitmap to the know height, width and format
            System.Drawing.Bitmap bmp = new System.Drawing.Bitmap((int)2592, (int)1936, System.Drawing.Imaging.PixelFormat.Format24bppRgb);

            //Create a BitmapData and Lock all pixels to be written 
            System.Drawing.Imaging.BitmapData bmpData = bmp.LockBits(
                                 new System.Drawing.Rectangle(0, 0, bmp.Width, bmp.Height),
                                 System.Drawing.Imaging.ImageLockMode.WriteOnly, bmp.PixelFormat);


            //Copy the data from the byte array into BitmapData.Scan0
            Marshal.Copy(data, 0, bmpData.Scan0, data.Length);


            //Unlock the pixels
            bmp.UnlockBits(bmpData);


            //Return the bitmap 
            return bmp;
        }
        public async void ListenVideo()
        {

            byte[] data = new byte[1024];
            IPEndPoint ipep = new IPEndPoint(IPAddress.Any, 11111);
            UdpClient newsock = new UdpClient(ipep);
            IPEndPoint sender = new IPEndPoint(IPAddress.Any, 11111);
            data = newsock.Receive(ref sender);
            string message = Encoding.UTF8.GetString(data, 0, data.Length);
            while (true)
            {

                data = newsock.Receive(ref sender);
                message = Encoding.UTF8.GetString(data, 0, data.Length);
                MemoryStream stream = new MemoryStream(data);
                panel1.BackgroundImage = CopyDataToBitmap(data);

                await Task.Delay(2000);

            }
        }

Where is the code that does the decoding? All I can see in the code is copying raw bytes from another process into the bitmap. — Neil, May 22 '20 at 14:13
If you open the file with Notepad there is an ascii header that will give version of h264 that is being used. Wiki says there is a number of versions. See : https://en.wikipedia.org/wiki/Advanced_Video_Coding. See the profile paragraph. — jdweng, May 22 '20 at 14:41
Can you open the file on machine with a viewer? This will indicate if the graphic card on machine will support the graphics. — jdweng, May 22 '20 at 14:45
The data comes from my drone. I have a drone app for my phone on there it works fine. It uses H264 decoder — MASTER GT, May 22 '20 at 14:56
"but when I convert it into bitmap" - because you do not DECODE it. This is like saying "I got a tibetan poem. WHen i type it on a typewrite, it is not in english". Sure it is not - it is in whatever langauge they speak in tibet. The bytes of a H264 frame will not magically transform into a bitmap when you use Marhsal.COPY (!). — TomTom, May 28 '20 at 10:01

score 1 · Answer 1 · answered May 28 '20 at 09:58

H.264 is encoding for elementary video stream, it is not an encoding for a separate image.

This means that decoding process involves a setup for video encoding. There is no single function (at least it is not supposed to work this way) to take bitstream and output a video frame.

You would typically setup stream decoding context, then feed input and pull output when it is ready.

Windows comes with a stock H.264 encoder and a typical API to consume such decoding services is Media Foundation. It is not the API for .NET so hence Media Foundation with C#

There is also Media Foundation .NET project with C# wrappers over native Media Foundation API.

Decode H264 frame to Bitmap

1 Answers1