JNA/ByteBuffer not getting freed and causing C heap to run out of memory

Question

Let me start by saying that my understanding of how JNA and Java direct native memory allocations is visceral at best, so I'm trying to describe my understanding of what's going on. Any corrections in addition to responses would be great...

I'm running an application that mixes Java and C native code using JNA and am running accross a reproducible issue with the Java Garbage Collector failing to free references to direct native memory allocations, resulting in the C heap running out of memory.

I'm positive that my C application is not the source of the allocation issue, as I'm passing a java.nio.ByteBuffer into my C code, modifying the buffer, and then accessing the result in my Java function. I have a single malloc and a single corresponding free during each function call, but after repeatedly running the code in Java the malloc will eventually fail.

Here's a somewhat trivialized set of code that exhibits the issue -- realistically I'm trying to allocate about 16-32MB on the C heap during the function call.

My Java code does something like:

public class MyClass{
    public void myfunction(){
        ByteBuffer foo = ByteBuffer.allocateDirect(1000000);
        MyDirectAccessLib.someOp(foo, 1000000);
        System.out.println(foo.get(0));
    }
}

public MyDirectAccessLib{
    static {
        Native.register("libsomelibrary");
    }
    public static native void someOp(ByteBuffer buf, int size);
}

Then my C code might be something like:

#include <stdio.h>
#include <stdlib.h>
void someOp(unsigned char* buf, int size){
    unsigned char *foo;
    foo = malloc(1000000);
    if(!foo){
        fprintf(stderr, "Failed to malloc 1000000 bytes of memory\n");
        return;
    }
    free(foo);

    buf[0] = 100;
}

Trouble is after calling this function repeatedly the Java heap is somewhat stable (it grows slowly), but the C function eventually cannot allocate any more memory. At a high level I believe this is because Java is allocating memory to the C heap, but not cleaning up the ByteBuffer that points at this memory because the Java ByteBuffer object is relatively small.

Thus far I've found running the GC manually in my function will provide the required cleanup, but this seems like both a poor idea and a poor solution.

How can I manage this problem better so that the ByteBuffer space is appropriately freed and my C heap space is controlled?

Is my understanding of the problem incorrect (is there something I'm running improperly)?

Edit: adjusted buffer sizes to be more reflective of my actual application, I'm allocating for images approximately 3000x2000...

Answer 1

You are actually facing a known bug in the Java VM. The best workaround listed in the bug report is:

• "The -XX:MaxDirectMemorySize= option can be used to limit the amount of direct memory used. An attempt to allocate direct memory that would cause this limit to be exceeded causes a full GC so as to provoke reference processing and release of unreferenced buffers."

Other possible workarounds include:

• Insert occasional explicit System.gc() invocations to ensure that direct buffers are reclaimed.
• Reduce the size of the young generation to force more frequent GCs.
• Explicitly pool direct buffers at the application level.

If you really want to rely on direct byte buffers, then I would suggest pooling at the application level. Depending on the complexity of your application, you might even simply cache and reuse the same buffer (beware of multiple threads).