Java: InputStream too slow to read huge files

pflz picture pflz · May 6, 2012 · Viewed 28k times · Source

I have to read a 53 MB file character by character. When I do it in C++ using ifstream, it is completed in milliseconds but using Java InputStream it takes several minutes. Is it normal for Java to be this slow or am I missing something?

Also, I need to complete the program in Java (it uses servlets from which I have to call the functions which process these characters). I was thinking maybe writing the file processing part in C or C++ and then using Java Native Interface to interface these functions with my Java programs... How is this idea?

Can anyone give me any other tip... I seriously need to read the file faster. I tried using buffered input, but still it is not giving performance even close to C++.

Edited: My code spans several files and it is very dirty so I am giving the synopsis

import java.io.*;

public class tmp {
    public static void main(String args[]) {
        try{
        InputStream file = new BufferedInputStream(new FileInputStream("1.2.fasta"));
        char ch;        
        while(file.available()!=0) {
            ch = (char)file.read();
                    /* Do processing */
            }
        System.out.println("DONE");
        file.close();
        }catch(Exception e){}
    }
}

Answer

Marko Topolnik picture Marko Topolnik · May 6, 2012

I ran this code with a 183 MB file. It printed "Elapsed 250 ms".

final InputStream in = new BufferedInputStream(new FileInputStream("file.txt"));
final long start = System.currentTimeMillis();
int cnt = 0;
final byte[] buf = new byte[1000];
while (in.read(buf) != -1) cnt++;
in.close();
System.out.println("Elapsed " + (System.currentTimeMillis() - start) + " ms");