Memory usage of byte array in Java

Question

For a heuristic precomputed table i need a byte array with 1504935936 entries. This should take about 1.5 GB of Memory.

public class Main{
    public static void main(String[] args){
        byte[] arr = new byte[1504935936];
    }
}

Why do I have a "OutOfMemoryError: Java heap space"-Exception, if I give the program 2 GB of RAM with

java -Xmx2048M Main

With

java -Xmx2153M Main

it works. Why does it need that much RAM?

Answer 1

Probably because the Java heap is being used and fragmented by other data in your program.

That byte array needs to be allocated as one contiguous 1.5 GB chunk of memory within the Java heap space. (This isn't required by the Java language spec, but AFAIK is how all current JVM implementations actually work.) Some of your heap space is being consumed and – probably more importantly – fragmented by the other memory allocations that happen in your program prior to allocating that big byte array. That java -Xmx2153M Main may be how big you have to make the overall heap for there to be a contiguous 1.5 GB space left by the time you get to the allocation.

If you chop up that big array into 100 smaller arrays 1/100th the size, it may fit in to a smaller heap because it's not as sensitive to heap fragmentation.