0

I have a file and i read data from this and put them in a jTable. The problem is that when the file has many data (e.g 300.000 rows), my application needs a lot of memory (350MB). Is there any efficient way in order to load many rows in a JTable?

I created a Default Model and a Jtable like that:

DefaultTableModel model = new DefaultTableModel(array, colNames);
JTable data_table = new JTable();
data_table.setModel(model);

The array 'array' contains the data and the array 'colNames' the names of the Columns.

mKorbel
  • 109,525
  • 20
  • 134
  • 319
user1005633
  • 625
  • 3
  • 9
  • 23
  • If you're loading files that large into tables, it's probably inevitable that you consume a lot of memory. Is it necessary to keep all that data in memory at once? How is the data being loaded from file? – Surveon Sep 18 '13 at 13:27
  • I want to make search e.g for a record in the JTable. If data are not in memory how can i do this? – user1005633 Sep 18 '13 at 13:30
  • There are some alternatives available to you, but the fastest way to access that data is through volatile memory. It might be unavoidable that your application leans towards larger memory footprints. How are you loading the data in from the file? – Surveon Sep 18 '13 at 13:35
  • i open the file i read the first line, split it and i store it in the two dimensional array 'array' – user1005633 Sep 18 '13 at 13:39
  • 2
    e.g 300.000 rows is beyond/overloading human possibility, loading can take more that 2-3minutes, sorting/filtering is about how to spent free time, use paginations logics .... – mKorbel Sep 18 '13 at 13:39

3 Answers3

3

If you want to reduce the memory consuption of your program. You could use java.nio.channels.FileChannel and map parts of the file to your main memory.

But you would have to reload/change current mapping, depending on your view.
Take a look at the javadoc of the map method.

I don' know what exactly your file is, but if you just use it as a simplistic replacement for an actual database. You're better off with using a real one.

mike
  • 4,929
  • 4
  • 40
  • 80
3

Use an embedded database, and store only the record ID in the model and then only if you need to filter/sort rows.

Joop Eggen
  • 107,315
  • 7
  • 83
  • 138
3

Absent enough information to offer a particular solution, you may be able to identify a reasonable partition function for your data's primary key, e.g. a String prefix or a Date range. Use an adjacent control to update the TableModel based on the selection. In this example, buttons are used to change a chart's data model. To minimize latency, Use SwingWorker. Aside from the memory problem, you may want to filter the table or load the file into an in-memory database such as H2 Database.

Community
  • 1
  • 1
trashgod
  • 203,806
  • 29
  • 246
  • 1,045
  • 1
    I like the idea about identifying a function for the primary key. If the key had information encoded. One could do simple operations with out the actual data and only load if when it is demanded. – mike Sep 18 '13 at 14:34