Memory not freed up, after closing java InputStream

Question

In this snippet of code, I load an excel file of size 10MB using Apache POI library. This consumes almost 2GB of Memory. Having iterated over all of the rows, I finally call close method. However, it seems GC does not free up spaces consumed by this stream and object. And still using 2GB + 400MB of memory.

Any ideas?

Here is my Code:

public List<Meter> loadFile(File excelFile) throws IOException, InvalidFormatException {
    List<Meter> allMeters = new ArrayList<>();
    InputStream inputStream = new FileInputStream(excelFile);
    XSSFWorkbook workbook = new XSSFWorkbook(inputStream);
    Sheet sheet1 = workbook.getSheetAt(0);
    Iterator<Row> rows_sheet1 = sheet1.iterator();
    if (rows_sheet1.hasNext()) {
        rows_sheet1.next(); //skip header
    }

    while (rows_sheet1.hasNext()) {
        try {
            Row currentRow = rows_sheet1.next();
            Cell meterNoCell = currentRow.getCell(0);
            Cell startPeriodCell = currentRow.getCell(1);
            Cell endPeriodCell = currentRow.getCell(2);
            Cell previousConsumption = currentRow.getCell(3);
            Cell currentConsumption = currentRow.getCell(4);
            Cell periodConsumptionCell = currentRow.getCell(5);

            meterNoCell.setCellType(CellType.STRING);
            startPeriodCell.setCellType(CellType.STRING);
            endPeriodCell.setCellType(CellType.STRING);

            //Reading values from above_defined cells and filling allMeters list (defined at the begining of the function).
            //......
            //Done
        }
        catch (Exception ex) {
            Logger.getLogger(MetersList.class.getName()).log(Level.SEVERE, null, ex);
        }
    }
    workbook.close();
    inputStream.close();
    return allMeters;
}

@assylias Using Activity Monitor in Mac / or Task Manager in Windows — Arman, Aug 27 '18 at 09:32
take a memory dump and analyse it to know what type of object is remaining in your memory — Fran Montero, Aug 27 '18 at 09:34
Please note that GC is not invoked automatically after each `close()` operation, it's invoked by JVM and the invokation depends on its GC settings. You can use `jvisualvm` to monitor your Java process, there you can find 'Perform GC' button. Please try it out and you'll see the less heap usage after GC. — Vladimir Vagaytsev, Aug 27 '18 at 09:43
@Arman Task manager gives you no information on GC activity - the JVM will reserve some space in the OS memory and grow it as necessary. But the memory used by the java process includes used and unused heap space (i.e. it won't decrease after a GC). You need to monitor GC differently, for example with jvisualvm or flight recorder. — assylias, Aug 27 '18 at 09:47
Dereferencing inputStream and workbook can trigger GC earlier, give it a shot. — arkantos, Aug 27 '18 at 09:51
@assylias Thank you very much, my friend. I figuy red out the problem. What Task Manager shows is the reserved heap space. Now, could you please let me know how I can decrease the reserved heap space in the idle time of my program? I've had some google search; however, it didn't give me a definite solution. — Arman, Aug 27 '18 at 11:19
@Arman You can use [the `-Xmx` flag](https://stackoverflow.com/questions/14763079/what-are-the-xms-and-xmx-parameters-when-starting-jvms) to limit the total heap size. If you want to reduce the heap size while the program is running, this may give you pointers: https://stackoverflow.com/q/4952568/829571 — assylias, Aug 27 '18 at 13:45

score 3 · Answer 1 · edited Aug 28 '18 at 09:26

3

First of all, I noticed that monitoring using Task Manager (Windows) or Activity Monitor (Mac) is a stupid job. What these tools show is the reserved heap space (not the used heap space). Thus, when I monitored my application's memory usage using NetBeans profiling, I noticed that GC works very well and frees up heap memory.

Besides, dereferencing Workbook and InputStream objects (with =null;) accelerated GC execution.

After that, my problem had changed.

After I close these streams, GC works well and used heap space decreases. However, the reserved heap space will remain unchanged and does not decrease, as shown in the below picture:

I had a look at this article. In summary, you need to use the following JVM parameters:

-XX:MinHeapFreeRatio
-XX:MaxHeapFreeRatio

I set -XX:MaxHeapFreeRatio=40 and after a while, the reserved heap space was released.

edited Aug 28 '18 at 09:26

Didier L

18,905
10
61
103

answered Aug 27 '18 at 14:17

Arman

99
9

1

It is quite normal: it depends on the JVM, and they don't immediately release memory to the OS once GCed. i.e. see https://www.quora.com/Will-Java-JVM-release-memory-back-to-OS-Is-the-memory-usage-of-a-Java-program-monotonically-incremental Please also note that depending on the jvm type you may also use a powerful profiler called mission control flight recorder https://wiki.openjdk.java.net/display/jmc/Overview – Davide Cavestro Aug 27 '18 at 14:45
Thanks @DavideCavestro – Arman Aug 27 '18 at 14:46

score -1 · Answer 2 · answered Aug 27 '18 at 09:43

-1

Try two of the suggested solutions and let us know if any of these helped you.

Use finalize() instead of close() for workbook and inputStream.
After closing workbook and inputStream set them both to null, and then call System.gc();

answered Aug 27 '18 at 09:43

Robert Gardziński

9

3

No need to use `finalize()`, moreover its usage should be avoided. See J. Bloch's "Effective Java" book. The `close()` invokation is enough here. The `System.gc()` should not be called in production code, it can be used to debugging just to make sure the GC eliminates dead objects. It can be triggered from the instrumentation tools like `jvisualvm`, so no need to pollute your code with it. – Vladimir Vagaytsev Aug 27 '18 at 09:46
5

Also `finalize()` is protected, and [deprecated since Java 9](https://docs.oracle.com/javase/9/docs/api/java/lang/Object.html#finalize--). It is not intended to be called from user code. – Didier L Aug 27 '18 at 09:50
@DidierL even better, `XSSFWorkbook` doesn’t override `finalize` (which is a good thing as it would be broken otherwise), so calling `finalize` on it wouldn’t close it at all. – Holger Aug 28 '18 at 08:29

Memory not freed up, after closing java InputStream

2 Answers2