Exceeding byte[] array length (over int upper limit) - java.lang.ArrayIndexOutOfBoundsException
I have a ByteArrayOutputStream object that I'm getting the following error for:
java.lang.ArrayIndexOutOfBoundsException at
java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:113)
I am trying to load a file that is several gigs into it by writing byte[] chunks of 250mb one at a time.
I can watch the byte grow in size and as soon as it hits length 2147483647, the upper limit of int, it blows up on the following line:
stream.write(buf);
stream is the ByteArrayOutputStream, buf is what I'm writing to the stream in 250mb chunks.
I was planning to do
byte result[] = stream.toByteArray();
At the end. Is there some other method I can try that will support byte array sizes greater than the int upper limit?
Answer
Arrays in Java simply can't exceed the bounds of int.
From the JLS section 15.10:
The type of each dimension expression within a DimExpr must be a type that is convertible (§5.1.8) to an integral type, or a compile-time error occurs. Each expression undergoes unary numeric promotion (§). The promoted type must be int, or a compile-time error occurs; this means, specifically, that the type of a dimension expression must not be long.
Likewise in the JVM spec for arraylength:
The arrayref must be of type reference and must refer to an array. It is popped from the operand stack. The length of the array it references is determined. That length is pushed onto the operand stack as an int.
That basically enforces the maximum size of arrays.
It's not really clear what you were going to do with the data after loading it, but I'd attempt not to need to load it all into memory to start with.