how properly discover the maximun size allowed for an array

Question

From one side I am afraid my question will downvote soon and close. On another hand I really couldn't find the correct answer. This question is very close to mine but the answer accepted is fae away to be true in my case.

Do Java arrays have a maximum size?

To exemplify, if I run

public static void main(String args[]) {
    String[] sa = new String[Integer.MAX_VALUE - 100];
}

with my current Java 9, I will get:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at...

It must exist a good explanation instead of a Integer.MAX_VALUE - magic number but I just can't find it.

*** edited

C:\>java -Xms4G -Xmx4G one.ex5
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at one.ex5.main(ex5.java:5)

There is no maximum size of an array, it depends on your JVM memory. (e.g. If you only have one array and the maximum size is N, if you have two array then this value maybe less than N/2) — zhh, Jan 10 '18 at 14:36
Jvm has a memory limit that is configurable, it seems you have reached its limit. — Yan.F, Jan 10 '18 at 14:36
Java array is indexed by ints. That's why Integer.MAX_VALUE is the maximum array index. Depending on the VM there might be changes to that magic number (for example 3-4 indexes less than max value might give the error). In your case -100 it might be due to the memory you have given to the java VM which also gives a limit to the allocated size — Veselin Davidov, Jan 10 '18 at 14:37
@zhh - there is a limit in java even if you have memory. You can't have array with index more than max int value because by design Java array indexes are integers - from java specification: "Arrays must be indexed by int values... An attempt to access an array component with a long index value results in a compile-time error.[22]" — Veselin Davidov, Jan 10 '18 at 14:38
@Veselin Davidov Yes, the maximum size has an upper bound which is ```Integer.MAX_VALUE```. — zhh, Jan 10 '18 at 14:41
Please, don't get me wrong but my question clear ask "how discover". So, how can I discover the limit of my current VM? Or how can I setup this "magic number"? Regard long limitation, it is very clear on pasted question above but it is not exactly what I asked for. — Jim C, Jan 10 '18 at 14:42
You might be able to discover by trial and error how large a `String[]` you can create, but once you figure it out, there won't be any memory left in which to create any `String`s to store into your maximally large `String[]`. Just saying... — Kevin Anderson, Jan 10 '18 at 15:02
I think you can't. Even if you can get the current available memory, you can't get the max number of elements you can create because different types of array have different overhead and the overhead may change according to the number of elements. Besides that, get the current available memory is not an easy work. — zhh, Jan 10 '18 at 15:03
If you are trying to discover the max possible size of an array for any reason other than pure intellectual curiosity or sheer entertainment, you need to seriously rethink your motivation. Again, just saying... ;-) — Kevin Anderson, Jan 10 '18 at 15:12
In fact I am really trying to understand waht is going behind the scene. I totally agree with "once you figure it out, there won't be any memory left in which to create any Strings to store into your maximally large String[]". I am not planing to do this in real scenario but I want to understand the reason. At this point might be useless but, thinking in long term, it will help me by sure. — Jim C, Jan 10 '18 at 15:17
I would consider a final answer for my question something like: "You might be able to discover by trial and error how large a String[]" and "Even if you can get the current available memory, you can't get the max number of elements you can create because different types of array have different overhead and the overhead may change according to the number of elements". And I would revel accept the answer from https://stackoverflow.com/questions/3038392/do-java-arrays-have-a-maximum-size if I was the question owner. — Jim C, Jan 10 '18 at 15:18

score 2 · Accepted Answer · answered Jan 10 '18 at 20:47

Maksimum size of array is determined by JVM max heap size

With default JVM settings on my machine

Runtime.getRuntime().maxMemory() returns 3790077952 or 3614.5 MiB

By trial and error I got following limits for array of 4 byte reference type or 4 byte primitive( I assume that string references/pointers are 4 byte long)

    int numOfBytes = 4;  // or 8 64bit JVM maybe ?
    int GiB = (1024*1024*1024)/numOfBytes;
    int MiB = GiB/1024;
    int KiB = MiB/1024;

    String[] a = new String[2*GiB+ 662*MiB +1023*KiB+252 /* 1008 bytes*/ ]; 
    // after first allocation JVM allowed me smaller ones
    String[] b = new String[59*MiB +613*KiB+ 220/* 880 bytes*/];
    String[] c = new String[4*MiB];
    //   String[] d = new String[1]; <-- fails here

So exact answer on your question in my environment is

   MagicNumber = Integer.MAX_VALUE-2*GiB+ 662*MiB +1023*KiB+252 
   // MagicNumber == 1784414203

additionally initial maximum size (on my machine) for array of long type is 355335999 leaving only about 60 MiB of free heap space

score 1 · Answer 2 · answered Jan 10 '18 at 14:45

This depends on the JVM you use, cf. this answer.

Apart from that, the reason for your error is not the maximum allowed length of an array, but the fact that the Java heap space available to your application at runtime was not sufficient to initialize your array.

Use the

-Xmx4G

option (where 4G stands for the heap space in gigabytes that you want to make available to the application. Just use other values, for example 2G or 8G or even values in megabytes, e.g. 3192M or 1024M etc.). More details on memory relevant JVM flags in this answer.

jpmottin · Answer 3 · 2018-01-10T16:03:24.443

You are confusing two notions... JVM (Java Virtual Machine) reserved space VS max allowed space for an array.

You got this error because your JVM are not properly configured to create an array of such size. To do that, you can launch you JVM with some parameters like :

-Xms4G -Xmx4G

More infos : https://docs.oracle.com/cd/E13150_01/jrockit_jvm/jrockit/jrdocs/refman/optionX.html

With these parameters, you will increase the size of your JVM, and so now you are able create very big arrays.

For information, the error text for max size array is (not the same as yours):

java.lang.OutOfMemoryError:
  Requested array size exceeds VM limit

You can have this error you exceed the limit size of an array. And this is "int" not "Integer", so theoretically it is 2^31-1 = 2147483647, which is Integer.MAX_VALUE. But some JVM has defined max array value as Integer.MAX_VALUE - 5

Hope my answer is somehow clear...

Edited: And if it's still not working, maybe you can figure it out with these lines :

// Get current size of heap in bytes
long heapSize = Runtime.getRuntime().totalMemory();

// Get maximum size of heap in bytes. The heap cannot grow beyond this size.
// Any attempt will result in an OutOfMemoryException.
long heapMaxSize = Runtime.getRuntime().maxMemory();

// Get amount of free memory within the heap in bytes. This size will increase // after garbage collection and decrease as new objects are created.
long heapFreeSize = Runtime.getRuntime().freeMemory();

Or these tools: Jconsole, VisualVM.

Please, take a note in my edited question that there was no improvement at all and I am using Integer.MAX_VALUE - 100 — Jim C, Jan 10 '18 at 15:12
@jim-c I have edited my answer... (this add is too big for a comment) — jpmottin, Jan 10 '18 at 16:05

how properly discover the maximun size allowed for an array

3 Answers3

Linked