How to upload a large file through an Azure function?

Question

I am exploring Azure Functions. The scenarios I have tested so far work great.

I am at a point where I am trying to figure out a way to upload files (20MB+) through an Azure Function.

The idea is that the Azure Function would first validate whether or not the authenticated user is allowed to upload the file before getting a hold on the request's stream and saving it to the BLOB storage.

Here is the code from the client side which creates a StreamContent to beam the bytes to the server:

using (Stream fileStream = ...)
{
    var streamContent = new StreamContent(fileStream);

    streamContent.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
    streamContent.Headers.ContentLength = fileStream.Length;
    streamContent.Headers.Add("FileId", fileId);

    var responseMessage = await m_httpClient.PutAsync(<validURI>, streamContent);

    responseMessage.EnsureSuccessStatusCode();

    succeeded = true;
}

Here is the code on the server side.

[FunctionName("upload-data")]
public static async Task<HttpResponseMessage> Run([HttpTrigger(AuthorizationLevel.Function, "put")]HttpRequestMessage req, TraceWriter log)
{
    try
    {
         //  Initialize stuff.

         //  Validate authenticated user & privileges.  

         //  Get the content stream of the request and 
         //  save it in the BLOB storage.

         return req.CreateResponse(HttpStatusCode.OK);
    }
    catch (Exception exc)
    {
        return req.CreateResponse(HttpStatusCode.InternalServerError, exc);
    }
}

I put a breakpoint right at the beginning of the method. I was expecting the breakpoint to be hit right after the client side sent the request, no matter how big the file is. However it does not.

I am guessing that the Azure Function is somehow trying to get all the content of the request's body before calling the method. I also think that I am sending a file that may exceed the 4 MB limit of the underlying Web Job but I did not see a way to configure that.

Is it possible to upload a large file to an Azure Function by streaming it? Is there a way to make this work?

Answer 1

I have found another way of doing things. Here is the solution that works for me.

When a client needs to upload a file, it calls the Azure Function to be authenticated (using the Identity provided by the Framework) & authorized (it can be a simple targeted check in a Table Storage, meaning is (s)he allowed to do such operation).

The Azure Function will ask for a Shared Access Signature to access a specific Blob. The SAS will give the client access to the Blob storage with Write-only privileges for a limited time (watch out of the clock's skew on Azure).

The client will then use the returned SAS to upload the file directly to the Blob storage. That way, it avoids the long term communication with the client as mentioned by Afzaal Ahmad Zeeshan and reduces the overall cost even more as the Azure Function is no more dependent on the connection speed of the client.