0

Someone knows way to write and read from file (xml, json) in java where java object will be able to store in file in reactive way.

For now I use com.fasterxml.jackson.core.JsonGenerator to write objects to json file.

For reading from file I'm using com.fasterxml.jackson.core.JsonParser

but I'm not able to write and read from file in the same time. I don't want to have many open/close file descriptor operations.

It's way to do it in java?

My business scenario:

I read records from one big file ( more then couple of GB) and I need to create relations between records. Records in input file unfortunetlly are not sorted and it's imposible to write to sort file when all records are in perfect order.

I iterate over input file and find parent record and want to write the record to second file when I was created realations between records.

In second step I want to write child record but I need to iterate over record relations file (read file operation with open write descriptor) and I need to find find parent record for my child record and update that parent record when I fill child structure where insert child_1 element example structure used by me right now:

{
"parent_1": {
...
childs: [{
"child_1": { ... } 
}]
}
}
G. Sz.
  • 55
  • 1
  • 2
  • 4
  • 1
    Do you need to actually read for changes? Most programs just read the file once, store it in memory, edit the in-memory content, then when it's modified save it – Zachary Craig Jan 24 '19 at 15:30
  • @zack6849 I cannot store all my records in memory because it will produce out of memory (my team make that tests and it's real scenario). – G. Sz. Jan 25 '19 at 07:30
  • is it possible to change the application to NOT be flatfile based? this seems like exactly the sort of thing you'd want a database or key value store for – Zachary Craig Jan 25 '19 at 07:37

3 Answers3

0

The description of your problem is not very clear, we don't know why you want to do that and what you want to achieve. But i'll have a try:

If you try to do something like React.js does in JavaScript, you'd need your program to monitor if the file has changed and react to that by updating the corresponding object. But this won't be very performant (nor make much sense) - maybe just using static fields in you object's class to maintain state between instances would do the trick? If that applies to your scenario at all...

bkis
  • 2,530
  • 1
  • 17
  • 31
  • Storing data in memory it is not an option. I use small object amout the objects is to hight to store them in memory. – G. Sz. Jan 24 '19 at 15:43
0

I would load and save a file in its entirety with their file modification times (or a running version number/UUID in the json).

For multiple files, and possibly multiple accesses to the same file one needs a managing class.

The class Files is usefull for loading/saving.

Path path = Paths.get("...");
FileTime t = Files.getLastModified(path);
byte[] data = Files.readAllBytes(path);
FileTime tcheck = Files.getLastModified(path);
ByteArrayInputStream bais = ...

In this way access is as atomic/momentarily as possible.

Ideally one could use FileLock.lock(), but that is more for random-access files, file channels, especially with buffers. For instance if the json files were really huge. However then the json format is inadequate itself.

Joop Eggen
  • 107,315
  • 7
  • 83
  • 138
  • I heared about c# file streams but I didn't found any implementation in java https://stackoverflow.com/questions/11627276/read-and-write-to-file-at-the-same-time – G. Sz. Jan 24 '19 at 15:46
  • Look at Channel. One can open a Channel immediately, or do a RandomAccessFile.getChannel. A Channel one can lock. _I hope I remember correctly._ Locking a file for a longer time is not that ideal. One mechanism is to create a .lock file to signal to the software for exclusivity. – Joop Eggen Jan 24 '19 at 16:19
0

I think the best solution for my case will be:

  • read parent record and find for him all child records (store in memory) and after reaching end-of-file flush my parent record + child records to relations_file
  • for reading and writing best approach will be use of FileChannel
G. Sz.
  • 55
  • 1
  • 2
  • 4