java: read large binary file

Question

I need to read out a given large file that contains 500000001 binaries. Afterwards I have to translate them into ASCII.

My Problem occurs while trying to store the binaries in a large array. I get the warning at the definition of the array ioBuf:

"The literal 16000000032 of type int is out of range."

I have no clue how to save these numbers to work with them! Has somebody an idea?

Here is my code:

public byte[] read(){
    try{
        BufferedInputStream in = new BufferedInputStream(new FileInputStream("data.dat"));
        ByteArrayOutputStream bs = new ByteArrayOutputStream();
        BufferedOutputStream out = new BufferedOutputStream(bs);
        byte[] ioBuf = new byte[16000000032];       
        int bytesRead;
        while ((bytesRead = in.read(ioBuf)) != -1){
            out.write(ioBuf, 0, bytesRead);
        }
          out.close();
          in.close();
          return bs.toByteArray();
}

Answer 1

The maximum Index of an Array is Integer.MAX_VALUE and 16000000032 is greater than Integer.MAX_VALUE

Integer.MAX_VALUE = 2^31-1 = 2147483647

2147483647 < 16000000032

You could overcome this by checking if the Array is full and create another and continue reading. But i'm not quite sure if your approach is the best way to perform this. byte[Integer_MAX_VALUE] is huge ;) Maybe you can split the input file in smaller chunks process them.

EDIT: This is how you could read a single int of your file. You can resize the buffer's size to the amount of data you want to read. But you tried to read the whole file at once.

//Allocate buffer with 4byte = 32bit = Integer.SIZE
byte[] ioBuf = new byte[4];       
int bytesRead;
while ((bytesRead = in.read(ioBuf)) != -1){
   //if bytesRead == 4 you read 1 int
   //do your stuff
}