"Header content contains invalid characters" error when piping multipart upload part into a new request

Question

My express server receives file uploads from browsers. The uploads are transferred as multipart/form-data requests; I use multiparty to parse the incoming entity body.

Multiparty allows you to get a part (roughly, a single form field like an <input type="file">) as a readable stream. I do not want to process or store the uploaded file(s) on my web server, so I just pipe the uploaded file part into a request made to another service (using the request module).

app.post('/upload', function(req, res) {
    var form = new multiparty.Form();

    form.on('part', function(part) {

        var serviceRequest = request({
            method: 'POST',
            url: 'http://other-service/process-file',
            headers: {
                'Content-Type': 'application/octet-stream'
            }
        }, function(err, svcres, body) {
            // handle response
        });

        part.pipe(serviceRequest);
    });

    form.parse(req);
});

This works correctly most of the time. node automatically applies chunked transfer encoding, and as the browser uploads file bytes, they are correctly sent to the backend service as a raw entity body (without the multipart formatting), which ultimately gets the complete file and returns successfully.

However, sometimes the request fails and my callback gets called with this err:

TypeError: The header content contains invalid characters 
    at ClientRequest.OutgoingMessage.setHeader (_http_outgoing.js:360:11) 
    at new ClientRequest (_http_client.js:85:14) 
    at Object.exports.request (http.js:31:10) 
    at Object.exports.request (https.js:199:15) 
    at Request.start (/app/node_modules/request/request.js:744:32) 
    at Request.write (/app/node_modules/request/request.js:1421:10) 
    at PassThrough.ondata (_stream_readable.js:555:20) 
    at emitOne (events.js:96:13) 
    at PassThrough.emit (events.js:188:7) 
    at PassThrough.Readable.read (_stream_readable.js:381:10) 
    at flow (_stream_readable.js:761:34) 
    at resume_ (_stream_readable.js:743:3) 
    at _combinedTickCallback (internal/process/next_tick.js:80:11) 
    at process._tickDomainCallback (internal/process/next_tick.js:128:9) 

I'm unable to explain where that error is coming from since I only set the Content-Type header and the stack does not contain any of my code.

Why do my uploads occasionally fail?

Answer 1

This example shows how to send file as an attachment with national symbols in the filename.

const http = require('http');
const fs = require('fs');
const contentDisposition = require('content-disposition');
...

// req, res - http request and response
let filename='totally legit 😈.pdf';
let filepath = 'D:/temp/' + filename;               

res.writeHead(200, {
    'Content-Disposition': contentDisposition(filename), // Mask non-ANSI chars
    'Content-Transfer-Encoding': 'binary',
    'Content-Type': 'application/octet-stream'
});

var readStream = fs.createReadStream(filepath);
readStream.pipe(res);
readStream.on('error', (err) => ...);