Difference between ByteBuffer.allocateDirect() and MappedByteBuffer.load()

Question

I was trying to implement a sort of shared cache between two or more JVMs by memory mapping a particular file using MappedByteBuffer. From the specifications I see that when we use MappedByteBuffer.load() it should load the data into a direct buffer. I have a couple of questions on this.

My code snippet::

RandomAccessFile file = new RandomAccessFile("file.txt","rw");
FileChannel fc = file.getChannel();
MappedByteBuffer buf5 = fc.map(MapMode.READ_WRITE, 0, fc.size());

//ByteBuffer buf6 = ByteBuffer.allocateDirect(100000000);

buf5.load();

try
{
    Class c = Class.forName("java.nio.Bits");
    Field f = c.getDeclaredField("reservedMemory");
    f.setAccessible(true);
    long reservedMemory = f.getLong(null);
    f = c.getDeclaredField("maxMemory");
    f.setAccessible(true);
    System.out.println(
            "Direct Memory Usage: "+ reservedMemory +"/"+ f.getLong(null)+"\n");
}
catch (Throwable t)
{
}

1. The output of the above code is 0 byte for the Direct Memory Usage (File.txt is 1 GB). But if I uncomment the line ..

   ByteBuffer buf6 = ByteBuffer.allocateDirect(100000000);
   

   I get a Direct Memory Usage of 100MB . Not able to understand why is this so, as to why I am not getting any direct memory usage in the first place ( i.e. when the line is commented out )

2. Although the Direct Memory Usage is 0 B for the above code, I do see that the resident memory ( using unix top ) of the process increase by 1 GB. But if I do a "free -m" on the box, I do not see any increase in memory usage.

In both the cases, I am a bit confused as to where the memory is ending up.

Thanks!

Answer 1

Direct ByteBuffers (those allocated using ByteBuffer.allocateDirect) are different to MappedByteBuffers in that they represent different sections of memory and are allocated differently. Direct ByteBuffers are a way to access a block of memory allocated outside of the JVM generally allocated with a malloc call (although most implementations will probably use an efficient slab allocator). I.e. it's just a pointer to a block of memory.

A MappedByteBuffer represents a section of memory allocated using mmap call, which is used to perform memory mapped I/O. Therefore MappedByteBuffers won't register their use of memory in the same way a Direct ByteBuffer will.

So while both are "direct" in that they represent memory outside of the JVM their purposes are different.

As an aside, in order to get the reservedMemory value you are reflectively calling to an internal method of the JVM, whose implementation is not covered by any specification, therefore there are no guarantees as to what that value returns. Direct ByteBuffers can be allocated from within JNI using NewDirectByteBuffer call from C/C++ (MappedByteBuffers likely use this) and this probably doesn't affect the reservedMemory value, which is may only changed when using the Java ByteBuffer.allocateDirect.