Serving large files with C# HttpListener

Question

I'm trying to use HttpListener to serve static files, and this works well with small files. When file sizes grow larger (tested with 350 and 600MB files), the server chokes with one of the following exceptions:

HttpListenerException: The I/O operation has been aborted because of either a thread exit or an application request, or:
HttpListenerException: The semaphore timeout period has expired.

What needs to be changed to get rid of the exceptions, and let it run stable/reliable (and fast)?

Here's some further elaboration: This is basically a follow-up question to this earlier question. The code is slightly extended to show the effect. Content writing is in a loop with (hopefully reasonable) chunk sizes, 64kB in my case, but changing the value didn't make a difference except speed (see the mentioned older question).

using( FileStream fs = File.OpenRead( @"C:\test\largefile.exe" ) ) {

    //response is HttpListenerContext.Response...
    response.ContentLength64 = fs.Length;
    response.SendChunked = false;
    response.ContentType = System.Net.Mime.MediaTypeNames.Application.Octet;
    response.AddHeader( "Content-disposition", "attachment; filename=largefile.EXE" );

    byte[] buffer = new byte[ 64 * 1024 ];
    int read;
    using( BinaryWriter bw = new BinaryWriter( response.OutputStream ) ) {
        while( ( read = fs.Read( buffer, 0, buffer.Length ) ) > 0 ) {
            Thread.Sleep( 200 ); //take this out and it will not run
            bw.Write( buffer, 0, read );
            bw.Flush(); //seems to have no effect
        }

        bw.Close();
    }

    response.StatusCode = ( int )HttpStatusCode.OK;
    response.StatusDescription = "OK";
    response.OutputStream.Close();
}

I'm trying the download in a browser and also in a C# program using HttpWebRequest, it makes no difference.

Based on my research, I suppose that HttpListener is not really able to flush contents to the client or at least does so at its own pace. I have also left out the BinaryWriter and wrote directly to the stream - no difference. Introduced a BufferedStream around the base stream - no difference. Funny enough, if a Thread.Sleep(200) or slightly larger is introduced in the loop, it works on my box. However I doubt it is stable enough for a real solution. This question gives the impression that there's no chance at all to get it running correctly (besides moving to IIS/ASP.NET which I would resort to, but more likely stay away from if possible).

Answer 1

You didn't show us the other critical part how you initialized HttpListener. Therefore I tried your code with the one below and it worked

HttpListener listener = new HttpListener();
listener.Prefixes.Add("http://*:8080/");
listener.Start();
Task.Factory.StartNew(() =>
{
    while (true)
    {
        HttpListenerContext context = listener.GetContext();
        Task.Factory.StartNew((ctx) =>
        {
            WriteFile((HttpListenerContext)ctx, @"C:\LargeFile.zip");
        }, context,TaskCreationOptions.LongRunning);
    }
},TaskCreationOptions.LongRunning);

WriteFile is your code where Thread.Sleep( 200 ); is removed.

If you want to see the full code of it.

---

void WriteFile(HttpListenerContext ctx, string path)
{
    var response = ctx.Response;
    using (FileStream fs = File.OpenRead(path))
    {
        string filename = Path.GetFileName(path);
        //response is HttpListenerContext.Response...
        response.ContentLength64 = fs.Length;
        response.SendChunked = false;
        response.ContentType = System.Net.Mime.MediaTypeNames.Application.Octet;
        response.AddHeader("Content-disposition", "attachment; filename=" + filename);

        byte[] buffer = new byte[64 * 1024];
        int read;
        using (BinaryWriter bw = new BinaryWriter(response.OutputStream))
        {
            while ((read = fs.Read(buffer, 0, buffer.Length)) > 0)
            {
                bw.Write(buffer, 0, read);
                bw.Flush(); //seems to have no effect
            }

            bw.Close();
        }

        response.StatusCode = (int)HttpStatusCode.OK;
        response.StatusDescription = "OK";
        response.OutputStream.Close();
    }
}