2

I am using MappedByteBuffer to load a file I wrote into memory. This is supposed to be a fast way of loading a file. My file contains a bunch of ints and doubles. There are a lot of tutorials online that show you how to write and then read the file. This is my test:

private static int count = (500*4) + (500*8);

public static void main(String[] args) throws IOException {
    //writing two arrays to file
    int[] a = new int[500];
    double [] b = new double[500];

    for (int i = 0; i < a.length; i++){
        a[i] = i;
        b[i] = (double)i;
    }

    RandomAccessFile memoryMappedFile = new RandomAccessFile("f.txt","rw");
    MappedByteBuffer out = memoryMappedFile.getChannel().map(FileChannel.MapMode.READ_WRITE, 0, count);
    for(int i = 0 ; i < a.length; i++) {
        out.putInt(a[i]);
        out.putDouble(b[i]);
    }

    //reading back the ints and doubles
    int c = 0;
    out.position(c);
    while (out.hasRemaining()){
        System.out.println(out.getInt(c));
        c += 4;
        out.position(c);
        System.out.println(out.getDouble(c));
        c += 8;
        out.position(c);

    }
}

This is all well and good but the problem with these tutorials is that they are only printing the contents of the file to console. However, If I want to use the values of the file to do more calculations, later on, I need to store them to some variables (or arrays), essentially making a new copy of them which defeats the purpose of the memory mapped file.

The only other workaround is to call out.position() on demand but I need to know at what position is my desired value I want to use which I think is impossible to know if I am not traversing them sequentially.

in c/c++ you can create an int variable that will point to the memory mapped int so You will not copy the value but in Java, this is not possible.

So Essentially I would like to access the memory mapped file in random access rather than sequential.

Is this maybe a disadvantage of using MemoryMappedFiles in java?

bcsta
  • 1,963
  • 3
  • 22
  • 61
  • 1
    *in c/c++ you can create an int variable that will point to the memory mapped int* Not without [violating strict aliasing](https://stackoverflow.com/questions/98650/what-is-the-strict-aliasing-rule) and invoking undefined behavior in the process. – Andrew Henle Jun 19 '19 at 11:15
  • The disadvantage of Java not having pointers has nothing to do with MMapped files. There are workarounds for this but they depend on the structure of the data you have. – Torben Jun 19 '19 at 11:15
  • @Torben so you need to know the byte position to use memory mapped files? to me, that's impossible since I have a very large file with multiple primitive data types. – bcsta Jun 19 '19 at 11:19
  • Copying data around in memory is much faster than reading the data from a disk file. How do you know the position of the desired data in any case: on disk or in memory? – NormR Jun 19 '19 at 12:28
  • @NormR I don't. That's not the point. The point is that if I have a 2GB memory mapped file, copying that memory twice is obviously bad. so knowing the position in memory would be very memory efficient. – bcsta Jun 19 '19 at 12:31
  • 1
    From what I get from this, the fundamental question seems to be "are you supoposed to know the binary structure of the data when you read it". I think you are. And as Andrew Henle points out, this applies to C and C++ too. – Torben Jun 19 '19 at 13:34
  • How can you find the position of the desired data in the file? Searching data that is in memory has to be faster than searching for it by continual reading of a file. – NormR Jun 19 '19 at 15:09

0 Answers0