.Net app using 1.3GB on system with 16GB RAM throws OutOfMemoryException

Question

I have a process dump from a Windows 10 64-bit .Net Winforms application that suffered from a System.OutOfMemoryException. The dump file is 1.3GB. A managed profiler (dotMemory) says 220MB of heap is allocated (of which 108MB is used).

The app is compiled as AnyCPU, prefer 32-bit is off. It also contains CLI/C++ projects that target x64, so it just won't run in a 32-bit environment. The app happily uses more than 1.3GB in other circumstances.

It is running on a system with 16GB of RAM, so why does it go out of memory?

The exception stack trace:

System.OutOfMemoryException: Out of memory.
   at System.Drawing.Graphics.FromHdcInternal(IntPtr hdc)
   at System.Drawing.Graphics.FromHdc(IntPtr hdc)
   at DevExpress.XtraBars.Docking2010.DocumentsHost.DoPaint(Message& m)
   at DevExpress.XtraBars.Docking2010.DocumentsHost.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

Heap fragmentation could be a thing. This is the report from dotMemory (managed mem), nothing to worry about as far as I can see:

WinDbg gives me this for '!address -summary'. Although a lot of , the majority is only 'reserve' and not 'commit'.

0:000> !address -summary
--- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
Free                                    405     7ffe`8db96000 ( 127.994 TB)          100.00%
<unknown>                              1515        1`3f3b3000 (   4.988 GB)  86.22%    0.00%
Image                                  2261        0`25f26000 ( 607.148 MB)  10.25%    0.00%
Heap                                    120        0`08324000 ( 131.141 MB)   2.21%    0.00%
Stack                                   234        0`04bc0000 (  75.750 MB)   1.28%    0.00%
Other                                    39        0`00200000 (   2.000 MB)   0.03%    0.00%
TEB                                      78        0`0009c000 ( 624.000 kB)   0.01%    0.00%
PEB                                       1        0`00001000 (   4.000 kB)   0.00%    0.00%

--- Type Summary (for busy) ------ RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_PRIVATE                            1882        1`452fe000 (   5.081 GB)  87.82%    0.00%
MEM_IMAGE                              2261        0`25f26000 ( 607.148 MB)  10.25%    0.00%
MEM_MAPPED                              105        0`07236000 ( 114.211 MB)   1.93%    0.00%

--- State Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_FREE                                405     7ffe`8db96000 ( 127.994 TB)          100.00%
MEM_RESERVE                             681        1`22426000 (   4.535 GB)  78.39%    0.00%
MEM_COMMIT                             3567        0`50034000 (   1.250 GB)  21.61%    0.00%

--- Protect Summary (for commit) - RgnCount ----------- Total Size -------- %ofBusy %ofTotal
PAGE_READWRITE                         1835        0`23d12000 ( 573.070 MB)   9.67%    0.00%
PAGE_EXECUTE_READ                       195        0`15090000 ( 336.563 MB)   5.68%    0.00%
PAGE_READONLY                           854        0`13fde000 ( 319.867 MB)   5.40%    0.00%
PAGE_WRITECOPY                          484        0`01633000 (  22.199 MB)   0.37%    0.00%
PAGE_EXECUTE_READWRITE                   92        0`012db000 (  18.855 MB)   0.32%    0.00%
PAGE_READWRITE|PAGE_WRITECOMBINE          5        0`00830000 (   8.188 MB)   0.14%    0.00%
PAGE_READWRITE|PAGE_GUARD                78        0`0015e000 (   1.367 MB)   0.02%    0.00%
PAGE_NOACCESS                            24        0`00018000 (  96.000 kB)   0.00%    0.00%

--- Largest Region by Usage ----------- Base Address -------- Region Size ----------
Free                                    213`79810000     7de1`2f130000 ( 125.880 TB)
<unknown>                              7ff4`a8af0000        1`00020000 (   4.000 GB)
Image                                  7ffd`06181000        0`03b43000 (  59.262 MB)
Heap                                    213`6b332000        0`0095d000 (   9.363 MB)
Stack                                    52`c8600000        0`000fb000 (1004.000 kB)
Other                                   213`311e0000        0`00181000 (   1.504 MB)
TEB                                      52`c8000000        0`00002000 (   8.000 kB)
PEB                                      52`c8158000        0`00001000 (   4.000 kB)

Excessive use of GDI handles is a common cause for trouble, but the application has a watchdog for that: it fails if GDI/user handles reach 8000, well below the 10000 OS limit.

I also found this bug report but it was fixed ages ago.

In this post a typical cause is having Bitmap instances that are not disposed, so their unmanaged memory piles up. But that should show up in the WinDbg output then. Anyway, I've compared the typical use of the application with the faulty behavior, and they both have around 1000 Bitmap instances around (many icons etc). It could still be caused by a few very large undisposed Bitmap's, but that seems unlikely. And it still doesn't explain why it goes out of memory at 1.3GB on a system with 16GB RAM.

What else can be the cause? Memory fragmentation of the unmanaged memory? What can I do to further investigate this issue?

What does your code do? In 99% of cases, OOMs are thrown because of memory fragmentation caused by inefficient application code, not lack of RAM. Adding items to a list 1 by 1 for example results in log2(N) reallocations of the internal buffer. GDI resources have their own limits, so trying for example to "render" 1000 documents as images by using GDI methods can end up eating up all GDI resources. — Panagiotis Kanavos, Jan 12 '23 at 11:05
`The app happily uses more than 1.3GB in other circumstances.` that almost certainly means it's leaking. Why would any application *need* 1GB in the first place? Somehow, somewhere, the application is allocating 1GB of orphaned objects in RAM that have to be garbage-collected eventually. This results in a *lot* of CPU time wasted to allocate and GC the orphans. Fixing the leak could result in performance gains similar or better than parallelizing the application — Panagiotis Kanavos, Jan 12 '23 at 11:06
@PanagiotisKanavos: using all GDI resources would not result in a OOM exception, would it? IMHO a lot of strange effects happen, like black boxes, screen not updating, etc. Why would an app not need 1 GB? I worked on several larger project in which the memory usage without the user having done anything is 700 MB+. — Thomas Weller, Jan 12 '23 at 11:12
It is important to stop being misled by the exact text of the exception message. The stack trace tells the tale, this is a problem induced by graphics handles. Nothing to do with RAM and extremely common in .NET programs when too many programmers forget to Dispose() their drawing objects. If you have a "watchdog" for it then we know that is a real issue with the program, it remains to be seen by nobody here from this post how reliable that code actually is. — Hans Passant, Jan 12 '23 at 13:15
Fragmentation should not be the case (managed mem, see description, unmanaged mem, see answer of Thomas) With 'it uses more than 1.3GB' I intended to avoid the 32-bit discussion. There are reasons for apps to use that amount of memory, I'm with Thomas here ;-) — Stijn, Jan 12 '23 at 13:44
Good point from Hans about the 'watchdog' :-) Can you see the # allocated handles from a dump file? — Stijn, Jan 12 '23 at 13:48
Does this happen when you debug the application? Visual Studio's [Diagnostics Window](https://learn.microsoft.com/en-us/visualstudio/profiling/running-profiling-tools-with-or-without-the-debugger?view=vs-2022) displays the current memory usage as a graph and allows you to take and compare memory snapshots. In a leak scenario the graph will look like a saw, increasing and then dropping on any GC. — Panagiotis Kanavos, Jan 12 '23 at 15:13
As for GDI exhaustion, it *can* result in an OOM exception, because the number of allowed handles was exhausted. You can use [Process Explorer](https://techcommunity.microsoft.com/t5/windows-blog-archive/pushing-the-limits-of-windows-user-and-gdi-objects-8211-part-2/ba-p/723897) from the SysInternals tool to check the GDI object count for a process. You can load a process dump into Visual Studio and check call stacks etc. You can use [ProcDump](https://learn.microsoft.com/en-us/sysinternals/downloads/procdump) to generate dumps — Panagiotis Kanavos, Jan 12 '23 at 15:18
Which .NET version are you using? There are [some GitHub issues](https://github.com/search?q=org%3Adotnet+fromhdcinternal+outofmemoryexception&type=issues) for GDI related OOMs that *aren't* memory related at all. [In this issue](https://github.com/dotnet/winforms/issues/3662) an OOM was thrown for a null argument. This was fixed in .NET 5 to actually throw an ArgumentNullException. — Panagiotis Kanavos, Jan 12 '23 at 15:29
Latest .Net framework is being used. I think some GDI bug is a likely explanation. As for the sawtooth memory usage pattern, that just means you're creating garbage, not necessarily a leak? — Stijn, Jan 12 '23 at 15:37

Thomas Weller · Answer 1 · 2023-01-12T12:06:39.653

1

RAM is not interesting, generally. RAM refers to physical memory and your application does not work with physical memory, it works with virtual memory. You typically have more virtual memory than physical memory, because the page file counts as virtual memory, too. As you can see, WinDbg reports 127 TB of free memory.

However, this does not mean that you can actually get that much memory. You would need a 127 TB of free storage on your hard disk, which is unlikely. The typical memory you can use is somewhere around

your RAM size 
- kernel stuff 
- RAM in use by other applications
+ size of the page file
- page file use by other applications

Calculating that amount after the fact is quite difficult, so we can't really know how much virtual memory was available at the time of the OOM.

And it still doesn't explain why it goes out of memory at 1.3GB on a system with 16GB RAM.

You can OOM with even less memory used. Try

#include <exception>
#include <limits>

int main()
{
   auto const p = malloc(std::numeric_limits<size_t>::max());
   if (!p) throw std::exception();
}

Memory fragmentation of the unmanaged memory?

No. Largest Region by Usage says there is a contiguous block of 125 TB. A new block would be requested to fulfill the request. You run into fragmentation issues when the largest available block is smaller than what was requested.

it fails if GDI/user handles reach 8000, well below the 10000 OS limit.

8000 is pretty much, IMHO. Please note that there is not only a limit of 10000 per process but also 65536 per session.

What can you do?

Make sure you're not having a OOM due to GDI handles (thanks to Panagiotis Kanavos for the reference)
Follow the instructions of How to use WinDBG to track down .net out of memory exceptions?
Make Proper use of the IDisposable interface. The top answer has an example explicitly for Bitmaps. I'd even consider it ok to call GC.Collect() to indicate to the GC that it might be able to free a lot of memory now.
Check the size of your page file (control panel) and the free disk space for that page file to expand.
Check if there are Updates to your 12 year old DXperience component and you use the latest version of it.

edited Jan 12 '23 at 12:06

answered Jan 12 '23 at 11:00

Thomas Weller

55,411
20
125
222

Just the fact that 1.3GB is used means there are huge leaks. I suspect the application is doing something "clever" like rendering controls into Bitmaps as a quick&dirty way to print or generate files. – Panagiotis Kanavos Jan 12 '23 at 11:12
1

@PanagiotisKanavos see my comment above. I worked on several larger projects in which the memory usage without the user having done anything is 700 MB+. Just at the start screen. Opening a single document easily doubles that. – Thomas Weller Jan 12 '23 at 11:15
@PanagiotisKanavos Visual Studio uses 1.2 GB even with a single 5 liner like above C++ code. Does VS have a leak? Most certainly it has leaks somewhere, but not in that amount. Download Paint.NET and create a 30000x30000 bitmap. It will use 3.6 GB of memory. Does it indicate a leak? Probably not. – Thomas Weller Jan 12 '23 at 11:19
I've worked on projects where lazy object management *did* cause 1GB of extra RAM. There are a *lot* of SO questions already where OOMs are caused by adding lots of items to a List<> one by one. Fixing the inefficient code didn't just reduce RAM, it resulted in 4-8x faster performance – Panagiotis Kanavos Jan 12 '23 at 11:23
1

As for GDI resulting in an OOM, yes it does. [System.OutOfMemoryException: Out of memory (GDI)](https://stackoverflow.com/questions/771819/system-outofmemoryexception-out-of-memory-gdi) – Panagiotis Kanavos Jan 12 '23 at 11:23
`Does VS have a leak?` VS does far more than 90% of desktop applications. It doesn't show 1.3GB of RAM taken with only 300MB used either. It *can* show if the application has such a leak though - the memory usage in the `Diagnostic Tools` window will look like a sawtooth, dropping on every GC – Panagiotis Kanavos Jan 12 '23 at 11:26
@PanagiotisKanavos The [link to OOM & GDI](https://stackoverflow.com/questions/771819/system-outofmemoryexception-out-of-memory-gdi) may be related to the [bug report](https://support.microsoft.com/en-us/topic/fix-outofmemoryexception-exception-when-you-use-a-graphics-object-to-paint-in-a-net-framework-3-5-based-windows-forms-application-fd2c4d42-6137-a793-a17f-3e8f2cf46fb7) I mentioned, fixed long ago. The error occurred in a finalization method, causing OOM to be thrown incorrectly. – Stijn Jan 12 '23 at 14:28
@ThomasWeller: thanks for the suggestions. I in fact did all those things except checking the page file. The component is up to date, 2010 is when it was first conceived I believe. Can you see the number of GDI handles from a dump file? I don't have many of the typical ones like Pen's, Brushes, Graphics etc., but there might be others of course. – Stijn Jan 12 '23 at 14:36

.Net app using 1.3GB on system with 16GB RAM throws OutOfMemoryException

1 Answers1