0

I'm always getting GC overhead limit errors during random times in a for loop, where I go through 200K+ records I queried from Pgres. How do I prevent this error?

Here's when I get the OutOfMemoryError, going through 200K+ records:

 final List<CustomerCSV> customerCSVs = new ArrayList<>();
 for (Record record : results) {
     final Iterable<String> tags = 
     Iterables.transform(record.get(CUSTOMER.TAGS), (part) -> StringUtils.chop(part.toString().substring(1)));
     customerCSVs.add(new CustomerCSV(record.get("name"), record.get("email"), Joiner.on(",").join(tags));
}
SamHoque
  • 2,978
  • 2
  • 13
  • 43
P_equals_NP_2021
  • 627
  • 1
  • 9
  • 27
  • Possible duplicate of [How to deal with "java.lang.OutOfMemoryError: Java heap space" error (64MB heap size)](https://stackoverflow.com/questions/37335/how-to-deal-with-java-lang-outofmemoryerror-java-heap-space-error-64mb-heap) – SamHoque Nov 01 '18 at 22:46
  • GC overhead limit exceeded doesn't (necessarily) mean you're running out of heap memory. It means the GC is working too hard. This usually means you're allocating objects at an enormous rate, and your young generation space is too small, so too many objects are getting tenured. – Daniel Pryden Nov 01 '18 at 23:21
  • What version of Java is this, and what options are you passing on the `java` command line? – Daniel Pryden Nov 01 '18 at 23:22
  • Java8 and no options. – P_equals_NP_2021 Nov 01 '18 at 23:41

1 Answers1

0

Have you tried increasing the heap memory size?

Try add the below parameters while running: java -Xmx**m

I believe the above is just a temporary solution. Alternatively I suggest you fetch the records in sequence.

https://www.postgresql.org/docs/9.5/static/sql-createsequence.html

Rohit Vincent
  • 66
  • 1
  • 1
  • 6