Stream uploaded file to Azure blob storage with Node

Question

Using Express with Node, I can upload a file successfully and pass it to Azure storage in the following block of code.

app.get('/upload', function (req, res) {
    res.send(
    '<form action="/upload" method="post" enctype="multipart/form-data">' +
    '<input type="file" name="snapshot" />' +
    '<input type="submit" value="Upload" />' +
    '</form>'
    );
});

app.post('/upload', function (req, res) {
    var path = req.files.snapshot.path;
    var bs= azure.createBlobService();
    bs.createBlockBlobFromFile('c', 'test.png', path, function (error) { });
    res.send("OK");
});

This works just fine, but Express creates a temporary file and stores the image first, then I upload it to Azure from the file. This seems like an inefficient and unnecessary step in the process and I end up having to manage cleanup of the temp file directory.

I should be able to stream the file directly to Azure storage using the blobService.createBlockBlobFromStream method in the Azure SDK, but I am not familiar enough with Node or Express to understand how to access the stream data.

app.post('/upload', function (req, res) {

    var stream = /// WHAT GOES HERE ?? ///

    var bs= azure.createBlobService();
    bs.createBlockBlobFromStream('c', 'test.png', stream, function (error) { });
    res.send("OK");
});

I have found the following blog which indicates that there may be a way to do so, and certainly Express is grabbing the stream data and parsing and saving it to the file system as well. http://blog.valeryjacobs.com/index.php/streaming-media-from-url-to-blob-storage/

vjacobs code is actually downloading a file from another site and passing that stream to Azure, so I'm not sure if it can be adapted to work in my situation.

How can I access and pass the uploaded files stream directly to Azure using Node?

Answer 1

SOLUTION (based on discussion with @danielepolencic)

Using Multiparty(npm install multiparty), a fork of Formidable, we can access the multipart data if we disable the bodyparser() middleware from Express (see their notes on doing this for more information). Unlike Formidable, Multiparty will not stream the file to disk unless you tell it to.

app.post('/upload', function (req, res) {
    var blobService = azure.createBlobService();
    var form = new multiparty.Form();
    form.on('part', function(part) {
        if (part.filename) {

            var size = part.byteCount - part.byteOffset;
            var name = part.filename;

            blobService.createBlockBlobFromStream('c', name, part, size, function(error) {
                if (error) {
                    res.send({ Grrr: error });
                }
            });
        } else {
            form.handlePart(part);
        }
    });
    form.parse(req);
    res.send('OK');
});

Props to @danielepolencic for helping to find the solution to this.