How to avoid an explicit call to GC.Collect() for this case that uses BitmapSource to continuously update a camera image

Question

I need to include a camera image in my UI. The solution that I came up with for the View and ViewModel is given below. In the ViewModel I am using a BitmapSource which hold the camera image and that is continuously update, whenever the camera signals a new frame. To this BitmapSource I bind the View.

This code works find for the most part. However I have the issue that the application periodically consumes very large amounts of memory. When the video has not started up yet, the consumption is ~245MB. But when the video starts, it will quickly climb to ~1GB within ~4 seconds, which is when the Garbage-Collector kicks in and reduces that value back down to ~245MB. At this point the video will briefly stutter (I suspect due to CPU taxation). This happens periodically, every 4 seconds or so. Sometime, when the GC does not kick in after 4 seconds, memory usage can even reach 2GB and has also caused an out-of-memory exception.

The way I found to remedy this, is to explicitly call the GC each time a the frame is updated. See the two commented lines. When doing this, the memory will continue to hover at ~245MB after the video starts.

However this causes a significant increase in CPU usage from ~20% to ~35%.

I do not understand very well how the GC works, but I suspect, the reason that the GC kicks in so late, is that the thread that updates the BitmapSource is busy with updating the video (which runs at 25FPS) and therefore does not have time to run GC unless it explicitly told to do so.

So my question is: What is the reason for this behavior and is there a better way to achieve, what I am trying to do, while avoiding the explicit call to the GC?

I have tried wrapping this in a using-statement, but BitmapSource does not implement IDisponsable and from what I understand using is not created for this case, but for when you are accessing external/unmanaged resources.

Here is the code:

CameraImageView:

<Image Source="{Binding CameraImageSource}"/>

CameraImageViewModel:

public class CameraImageViewModel : ViewModelBase
{
    private ICamera camera;
    private UMat currentImage;

    public BitmapSource CameraImageSource
    {
        get { return cameraImageSource; }
        set
        {
            cameraImageSource = value;
            RaisePropertyChanged("CameraImageSource");
        }
    }
    private BitmapSource cameraImageSource;

    public CameraImageViewModel(ICamera camera)
    {
        this.camera = camera;
        camera.EventFrame += new EventHandler(UpdateCameraImage);
    }

    private void UpdateCameraImage(object s, EventArgs e)
    {
        camera.GetMatImage(out currentImage);
        // commenting from here on downward, will also remove the described memory usage, but then we do not have an image anymore
        BitmapSource tmpBitmap = ImageProcessing.UMatToBitmapSource(currentImage);
        tmpBitmap.Freeze();
        DispatcherHelper.CheckBeginInvokeOnUI(() => CameraImageSource = tmpBitmap);
        //GC.Collect(); // without these lines I have the memory issue
        //GC.WaitForPendingFinalizers();
    }
}

ImageProcessing.UMatToBitmapSource:

    public static BitmapSource UMatToBitmapSource(UMat image)
    {
        using (System.Drawing.Bitmap source = image.Bitmap)
        {
            return Convert(source);
        }
    }

    /*
     * REF for implementation of 'Convert': http://stackoverflow.com/questions/30727343/fast-converting-bitmap-to-bitmapsource-wpf/30729291#30729291
     */
    public static BitmapSource Convert(System.Drawing.Bitmap bitmap)
    {
        var bitmapData = bitmap.LockBits(
            new System.Drawing.Rectangle(0, 0, bitmap.Width, bitmap.Height),
            System.Drawing.Imaging.ImageLockMode.ReadOnly, bitmap.PixelFormat);

        var bitmapSource = BitmapSource.Create(
            bitmapData.Width, bitmapData.Height, 96, 96, PixelFormats.Gray8, null,
            bitmapData.Scan0, bitmapData.Stride * bitmapData.Height, bitmapData.Stride);

        bitmap.UnlockBits(bitmapData);
        return bitmapSource;
    }

Answer 1

Replace creating new BitmapSource every UpdateCameraImage with with usage of single WriteableBitmap that you create once and update frequently. This way you will avoid creating copies of image in memory.

This code assumes that ImageWidth and ImageHeight do not change with time and are known in advance. If that is not the case, you will have to recreate the image dynamically when dimensions change.

1. Replace your cameraImageSource with:

   private cameraImageSource = new WriteableBitmap(
           ImageWidth,
           ImageHeight,
           96,
           96,
           PixelFormats.Gray8,
           null);
   

2. Change your UpdateCameraImage to:

   private void UpdateCameraImage(object s, EventArgs e)
   {
       camera.GetMatImage(out currentImage);
       System.Drawing.Bitmap bitmap = currentImage.Bitmap;
   
       var bitmapData = bitmap.LockBits(
           new System.Drawing.Rectangle(0, 0, ImageWidth, ImageHeight),
           System.Drawing.Imaging.ImageLockMode.ReadOnly, bitmap.PixelFormat);
   
   
       cameraImageSource.CopyPixels(new Int32Rect(0, 0, ImageWidth, 
           ImageHeight), bitmapData.Scan0, bitmapData.Stride * bitmapData.Height, bitmapData.Stride);
   
       bitmap.UnlockBits(bitmapData);
   }
   

It's also possible that calling currentImage.Bitmap on UMat is not necessary. I don't know EmguCV library which you seem to be using, but UMat has also other properties like Ptr that could be passed directly to CopyPixels.