27

I heard that there is a hard limit on the size of .Net Array. It is said that the maximum amount of memory that can be allocated to any single instance of an Array object ( regardless of whether it's int[], double[] or your own array) is 2GB. And no, if you have a 64 bit machine, the 2GB limit is still there.

I'm not sure whether my impression is correct or not. Anyone can confirm?

Graviton
  • 81,782
  • 146
  • 424
  • 602
  • Wow, you started an interesting discussion. – uriDium Mar 10 '10 at 08:47
  • If it makes you feel any better: the 2GB limit exists for unmanaged code as well. This is a constraint in the x64 instruction set, indexed offset addressing still has a 32-bit limit for the offset. It is not that it can't be overcome, it is just very inefficient to do so. – Hans Passant Mar 10 '10 at 09:18
  • 1
    Hans Passant: I am using C++ unmanaged code in my x64 app to allocate an array of 600E6 doubles using malloc. That is 4.8E9 bytes. – Michael Fitzpatrick Aug 27 '14 at 00:26

6 Answers6

24

In versions of .NET prior to 4.5, the maximum object size is 2GB. From 4.5 onwards you can allocate larger objects if gcAllowVeryLargeObjects is enabled. Note that the limit for string is not affected, but "arrays" should cover "lists" too, since lists are backed by arrays.

Marc Gravell
  • 1,026,079
  • 266
  • 2,566
  • 2,900
14

That is correct. No single object can be larger than 2 GB.

As with 32-bit Windows operating systems, there is a 2GB limit on the size of an object you can create while running a 64-bit managed application on a 64-bit Windows operating system.

This question has additional details and some useful links: Single objects still limited to 2 GB in size in CLR 4.0?

Community
  • 1
  • 1
Brian Rasmussen
  • 114,645
  • 34
  • 221
  • 317
  • 1
    Is there also a limit on the number of elements in the array? – uriDium Mar 10 '10 at 08:38
  • 1
    @uriDium: effectively yes, since an array is one object. – Brian Rasmussen Mar 10 '10 at 08:39
  • This question has additional info on the size of arrays in .NET http://stackoverflow.com/questions/1589669/overhead-of-a-net-array/1589759#1589759 – Brian Rasmussen Mar 10 '10 at 08:42
  • Hi, I meant not memory wise but how many elements an array can hold. But I suppose maybe the number of elements it can hold is restricted by memory first and not the fact that the index is an integer value? – uriDium Mar 10 '10 at 08:43
  • 1
    @Brain, just did the maths, i know in java the minimum class size is 8 bytes. At 8 bytes 2147483648 is almost 17 gig. So i guess we don't even get to the index being an integer as a problem. Yet... – uriDium Mar 10 '10 at 08:46
  • 6
    This restriction is lifted as of .NET 4.5: http://msdn.microsoft.com/en-us/library/hh285054(v=vs.110).aspx. – jhclark Aug 05 '13 at 22:17
4

You will run into a practical limit first - it is pretty impossible to get a 2gb array allocated. Practical limits I have encountered are around the 800mb mark AT PROGRAM START - going down drastically after that.

Anything larger than 64mb is a luck gamble on 32 bit - the large object heap is not defragmented, so you need 65mb free in one piece or allocation fails.

Theoretical limits are:

  • usable memory, especially under 32 bit.
  • 32 bit number space for index (0 upward - no negative numbers for arrays UNLESS YOU PLAY SMART IN CREATION). You can create arrays allowing negative numbers, but not with C# standard syntax - only with reflection.
  • 2gb per object.

But seriously, the practical implications are larger.

For .NET 4.0.... consider using memory mapped files ;)

Michal Ciechan
  • 13,492
  • 11
  • 76
  • 118
TomTom
  • 61,059
  • 10
  • 88
  • 148
1

I would have thought that the limit might be on the index. I thought that index used has to be an integer so anything bigger than integer wouldn't work unless they have some way around that. So that would be 4294967296 elements. Not sure if this is even half true. I would like to know the answer myself.

EDIT: As tomtom pointed out, integer is usually signed unless they using a non signed integer. So half of 4294967296 or 2147483648 roughly.

uriDium
  • 13,110
  • 20
  • 78
  • 138
  • it is pretty much double true. As arrays are integer indexed.... and start at 0 - only the positive part of the index is usable, roughly half your number ;) So, your answer is double true ;) – TomTom Mar 10 '10 at 08:41
  • If the index is an unsigned integer it would be your number. – Aurril Mar 10 '10 at 08:42
  • @TomTom: No it isn't. Each array is one object and since objects are restricted to 2 GB it depends on the type of elements the array holds. – Brian Rasmussen Mar 10 '10 at 08:43
  • 1
    @uriDium: The 2 GB limit will restrict this number significantly. You cannot have 2147483648 elements in an array. – Brian Rasmussen Mar 10 '10 at 08:55
  • @Brian Rasmussen: 2147483648 is the theoretical maximum number of elements, if each element would be 1 Byte long. – Aurril Mar 10 '10 at 09:00
  • @Aurril: You're right I should have stated that in most cases the 2 GB will limit the number of elements. Byte[] can get close to the theoretical maximum, but as arrays store a few additional pieces of data it is a little less than that. – Brian Rasmussen Mar 10 '10 at 09:08
  • The CLI specification allows Array indices to be either `int` or `long`. Microsoft chose to limit them to `int` in .NET, but Mono uses `long`, because Mono is used extensively in scientific high-performance cluster computing, and they frequently deal with arrays of much more than 2 billion elements. – Jörg W Mittag Mar 10 '10 at 17:13
1

Since .NET 6, the maximum number of elements an array can hold is defined by Array.MaxLength. It is currently 0x7FFFFFC7.

While strings are array-like, they have a lower limit. Currently the longest you cam make a string is 0x3FFFFFDF.

Austin
  • 2,771
  • 1
  • 18
  • 14
0

Hope this help: http://blogs.msdn.com/joshwil/archive/2005/08/10/450202.aspx

i.e.

  1. It uses int as index, which has max value = 2,147,483,647 (2GB)
  2. Its by design. 2.
Alex Essilfie
  • 12,339
  • 9
  • 70
  • 108
ata
  • 8,853
  • 8
  • 42
  • 68