Making file transfer more efficient Java

David picture David · Sep 21, 2011 · Viewed 10.7k times · Source

I have two wireless computers connected to an N wireless router. Each of these PCs are connected at between 108-150Mbps.

Theoretically, I should be able to transfer at 13.5MB/s to 18.75MB/s, under the absolute best of conditions.

The first computer (that is sending), uses a very fast SSD, which is around 100MB/s if I remember correctly. CPU usage also stays below 20%.

It sent 1960273535 bytes (1.8GB) in 656367ms. That's 2.8MB/s (22 out of 108 Megabits). When I open up task manager, I see that only 25-27% of the network connection is being used.

I am looking for any ideas, suggestions, or improvements that can make the transfer faster (over a network). I was thinking of buffering the file from the disk on a thread and sending the buffered data from another thread but I'm not sure if it's a good idea. Here is the SSCCE:

Host:

import java.io.*;
import java.net.*;


public class Host {


    public static void main(String[] args) throws IOException {


        ServerSocket servsock = new ServerSocket(15064);
        Socket sock = servsock.accept();
            long time = System.currentTimeMillis();

        OutputStream out = sock.getOutputStream();
        FileInputStream fileInputStream = new FileInputStream("C:\\complete.rar");

        byte [] buffer = new byte[64*1024]; 
        int bytesRead = 0;
        long totalSent = 0;

        while ( (bytesRead = fileInputStream.read(buffer)) != -1)
        {
            if (bytesRead > 0)
            {   
                out.write(buffer, 0, bytesRead);
                totalSent += bytesRead;
                System.out.println("sent " + totalSent);
            }   
        }

        sock.close();

        System.out.println("Sent " + totalSent + " bytes in "
                + (System.currentTimeMillis() - time) + "ms.");

    }
}

Client:

import java.io.*;
import java.net.*;

public class Client {

    public static void main(String[] args) throws Exception {
        Socket sock = new Socket("127.0.0.1", 15064);
        InputStream in = sock.getInputStream();
        FileOutputStream fileOutputStream = new FileOutputStream("output.rar");

        byte [] buffer = new byte[64*1024]; 
        int bytesRead = 0;

        while ( (bytesRead = in.read(buffer)) != -1)
            fileOutputStream.write(buffer, 0, bytesRead);
        sock.close();
        fileOutputStream.close();
    }
}

Edit: I tried mapping a network drive and sending the file over that, and windows did even worse - 2.35MB/s. According to this article http://tinyurl.com/634qaqg mapping a network drive is faster than FTP, and I also don't have the time to stay playing around and setting up the FTP server.

Edit2: After changing the timer, turns out it was transferring at 3MB/s over WiFi. I hate the "theoretical" throughput. When I buy something, I want to know it's REAL performance. It turns out the code is indeed limited by WiFi speeds. I am still open to suggestions though.

Edit 3: After running the program on 100Mbps LAN, it managed to transfer the file at 11.8MB/s. That's pretty good considering that the maximum transfer rate is 12.5MB/s.

Answer

NPE picture NPE · Sep 21, 2011

At 2.8MB/s, it is unlikely that the slowness has anything to do with your code. It is almost certainly due to the wireless network not being able to achieve the theoretical throughput (possibly due to environmental conditions).

It's easy to test whether this is the case: simply time a large ftp or scp file transfer between the same two computers and see what kind of throughput you're seeing.