0

I want to add 5000000 elements to an ArrayList and write that ArrayList to a file, but I encounter an OutOfMemoryException. I use a for-loop to add the objects to the ArrayList.

martijnn2008
  • 3,552
  • 5
  • 30
  • 40
we22world
  • 1
  • 4
    possible duplicate of [How to add more than 50,000,000 records in arraylist from mysql database](http://stackoverflow.com/questions/21159988/how-to-add-more-than-50-000-000-records-in-arraylist-from-mysql-database) – Raman Shrivastava Jun 21 '15 at 14:33
  • Where are the records coming from? – Robert Moskal Jun 21 '15 at 14:33
  • 1
    could you describe your record.. also are you doing this in a production application? – Jos Jun 21 '15 at 14:34
  • 2
    Can't you write the records to the file in chunks instead of all loading them into a list to write the datas from the list just after? – Alexis C. Jun 21 '15 at 14:36
  • I mean records are objects that i create and add to arraylist. – we22world Jun 21 '15 at 14:37
  • How much memory does a single object take ? Refer to this answer for cost model : http://stackoverflow.com/questions/320980/steps-in-the-memory-allocation-process-for-java-objects – John Jun 21 '15 at 14:38
  • If you are instantiating objects by yourself, do you really need to add them all to a list before you write them to a file ? Can't you just write them to file as you instantiate them? – Alp Jun 21 '15 at 14:40
  • I only know that when i test it with 1M objects it's okay and the size of the file is about 56MB. – we22world Jun 21 '15 at 14:41
  • cuz i use objectoutputstream. – we22world Jun 21 '15 at 15:21
  • 1
    If you're adding 5M records to an ArrayList your design is flawed. What are you trying to do since this might be an [XY Problem](http://meta.stackexchange.com/questions/66377/what-is-the-xy-problem) – JNYRanger Jun 23 '15 at 16:03

1 Answers1

0

If you're in control of creating the ArrayList, use the constructor which takes the initial capacity as argument. This way you prevent unnecessary copy operations too.

// This alone takes about 20 MB of RAM with 4 byte addresses
List<Object> l = new ArrayList<>(5000000);

If this doesn't help, don't buffer all you objects, write them directly to a file.

martijnn2008
  • 3,552
  • 5
  • 30
  • 40
Brian
  • 872
  • 8
  • 16