Calculating Time Remaining on File Copy

Question

I have an app that copies a large amount of files across the network to a file server (not web). I am trying to display a half decent estimation of the time remaining.

I have looked at a number of articles on SO an while the problem is addressed none that I have tried really do what I want. I want the estimated time remaining to be relatively stable I.E. not jump around all over the place depending on fluctuating transfer speeds.

So the first solution I looked at was to calculate the transfer speed in bytes per second

double bytePerSec = totalBytesCopied / TimeTaken.TotalSeconds;

And then divide the total byte remaining by the transfer rate.

double secRemain = (totalFileSizeToCopy - totalBytesCopied) / bytePerSec;

I figured that the time remaining would become more stable once a few MB had been copied (although expecting it to change . It doesn't, its erratic and jumps around all over the place.

Then I tried one of the solutions on SO....

double secRemain = (TimeTaken.TotalSeconds / totalBytesCopied) * (totalFileSizeToCopy - totalBytesCopied);

Which is a similar calculation but hoped it might make a difference!

So now I am kind of thinking I need to approach this from a different angle. IE Use averages? Use some kind of countdown timer and reset the time to go every so often? Just looking for opinions or preferably advice from anyone that has already had this problem.

Answer 1

Here's a working example of how you would asynchronously copy a file D:\dummy.bin to D:\dummy.bin.copy, with a timer taking snapshots of the transfer rate every second.

From that data, I simply take the average transfer rate from up to 30 snapshots (newest first). From that I can calculate a rough estimate of how long it will take to transfer the rest of the file.

This example is provided as-is and does not support copying multiple files in 1 operation. But it should give you some ideas.

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading;

public class Program
{
    public static void Main(string[] args)
    {
        var sourcePath = @"D:\dummy.bin";
        var destinationPath = @"D:\dummy.bin.copy";
        var sourceFile = new FileInfo(sourcePath);
        var fileSize = sourceFile.Length;
        var currentBytesTransferred = 0L;
        var totalBytesTransferred = 0L;
        var snapshots = new Queue<long>(30);
        var timer = new System.Timers.Timer(1000D);
        timer.Elapsed += (sender, e) =>
        {
            // Remember only the last 30 snapshots; discard older snapshots
            if (snapshots.Count == 30)
            {
                snapshots.Dequeue();
            }

            snapshots.Enqueue(Interlocked.Exchange(ref currentBytesTransferred, 0L));
            var averageSpeed = snapshots.Average();
            var bytesLeft = fileSize - totalBytesTransferred;
            Console.WriteLine("Average speed: {0:#} MBytes / second", averageSpeed / (1024 * 1024));
            if (averageSpeed > 0)
            {
                var timeLeft = TimeSpan.FromSeconds(bytesLeft / averageSpeed);
                var timeLeftRounded = TimeSpan.FromSeconds(Math.Round(timeLeft.TotalSeconds));
                Console.WriteLine("Time left: {0}", timeLeftRounded);
            }
            else
            {
                Console.WriteLine("Time left: Infinite");
            }
        };

        using (var inputStream = sourceFile.OpenRead())
        using (var outputStream = File.OpenWrite(destinationPath))
        {
            timer.Start();
            var buffer = new byte[4096];
            var numBytes = default(int);
            var numBytesMax = buffer.Length;
            var timeout = TimeSpan.FromMinutes(10D);
            do
            {
                var mre = new ManualResetEvent(false);
                inputStream.BeginRead(buffer, 0, numBytesMax, asyncReadResult =>
                {
                    numBytes = inputStream.EndRead(asyncReadResult);
                    outputStream.BeginWrite(buffer, 0, numBytes, asyncWriteResult =>
                    {
                        outputStream.EndWrite(asyncWriteResult);
                        currentBytesTransferred = Interlocked.Add(ref currentBytesTransferred, numBytes);
                        totalBytesTransferred = Interlocked.Add(ref totalBytesTransferred, numBytes);
                        mre.Set();
                    }, null);
                }, null);
                mre.WaitOne(timeout);
            } while (numBytes != 0);
            timer.Stop();
        }
    }
}