0

I am running a Bulk API in Java (Spring boot Framework). I see in kubernetes memory and CPU usage is continuously increasing.

My code is like this:

  function(list <ids>){
       for each id in ids{

          expensiveFunction(id);

      }
  }

  expensiveFunction(id){
   //bring lot of data 
    // process data
   //save data in db(transactional)
     }

So, I believe when expensive function is completed for one ID, memory(lot of data objects) should be released(Java's automatic Garbage Collection).

So, why is memory increasing continuously? How can I avoid it, as it causing throttling of memory resulting in termination of application.

Is this is something to do with hiberante like caching data or something?

qingy2019
  • 536
  • 7
  • 23
Deepak Aggarwal
  • 1
  • 1
  • 3
  • It's hard to say without your actual code ( kindly see https://stackoverflow.com/help/minimal-reproducible-example) but if for example, that "expensive operation" does something like open a new db connection each time that could be a problem. – bdehmer May 05 '21 at 17:41
  • i am bringing data from external service through api. Then doing some operations with data (manipulating data to usable form) and then saving a data in multiple tables which happens in one transaction. In my case no new connection is made to db. But how opening a new connection can increase memory? – Deepak Aggarwal May 05 '21 at 17:51
  • [related](https://stackoverflow.com/questions/61506136/kubernetes-pod-memory-java-gc-logs/61512521#61512521) – Eugene May 05 '21 at 18:00
  • I'm guessing this is not a problem with the Garbage Collector. Have you tried your application outside of Kubernetes? If so, does the application gets terminated as well? Also, remember that the Garbage Collector only collect objects which are not referenced in your code anymore, so if you're referencing this 'lot of data objects' outside the loop, they will not be collected until the the end of the method you called 'function'. – Matheus May 05 '21 at 20:07
  • I am not referencing any object outside loop. All objects are created inside loop. I have not tried – Deepak Aggarwal May 05 '21 at 21:37
  • I have not tried without kubernates because it's a heavy job. But i will try it now. Thanks – Deepak Aggarwal May 05 '21 at 21:59
  • Note that the GC does not clean up objects instantly but when it needs to. The GC may be triggered e.g. when the memory is full. You can use a heap dump in order to see the problem or run `System.gc()` (this is just a hint for the GC to run, you cannot rely on that) for debugging. – dan1st May 06 '21 at 04:44
  • Also, saving data to the DB might cause memory consumption as frameworks like JPA (I don't know what you are using) might cache objects. – dan1st May 06 '21 at 04:48

0 Answers0