POST request in Gatling

Question

I'm running into an issue with the POST of a pdf file. After recording a HAR file in Gatling 2.1.7, here's what I have:

.exec(http("request_10")
        .post("/api/data/files?revisionId=e9af2c93-d8df-4424-b307-df4c4abbaad1&uploadType=read_only_file&fileType=application%2Fpdf&fileName=testdocument.pdf&fileSize=10080&copyToEditable=true")
        .headers(Map(
            "Accept-Encoding" -> "gzip, deflate",
            "Content-Type" -> "multipart/form-data; boundary=----WebKitFormBoundaryawCJ4mjL1imiO7Ye"
            "Origin" -> url))
        .body(RawFileBody("RecordedSimulation_0010_request.txt")))

With the contents of RecordedSimulation_0010_request.txt:

------WebKitFormBoundaryawCJ4mjL1imiO7Ye
Content-Disposition: form-data; name="resumableChunkNumber"

1
------WebKitFormBoundaryawCJ4mjL1imiO7Ye
Content-Disposition: form-data; name="resumableChunkSize"

1048576
------WebKitFormBoundaryawCJ4mjL1imiO7Ye
Content-Disposition: form-data; name="resumableCurrentChunkSize"

10080
------WebKitFormBoundaryawCJ4mjL1imiO7Ye
Content-Disposition: form-data; name="resumableTotalSize"

10080
------WebKitFormBoundaryawCJ4mjL1imiO7Ye
Content-Disposition: form-data; name="resumableType"

application/pdf
------WebKitFormBoundaryawCJ4mjL1imiO7Ye
Content-Disposition: form-data; name="resumableIdentifier"

66dc65bf-265d-4363-96fd-7fc13f8ceda4
------WebKitFormBoundaryawCJ4mjL1imiO7Ye
Content-Disposition: form-data; name="resumableFilename"

testdocument.pdf
------WebKitFormBoundaryawCJ4mjL1imiO7Ye
Content-Disposition: form-data; name="resumableRelativePath"

testdocument.pdf
------WebKitFormBoundaryawCJ4mjL1imiO7Ye
Content-Disposition: form-data; name="resumableTotalChunks"

1
------WebKitFormBoundaryawCJ4mjL1imiO7Ye
Content-Disposition: form-data; name="file"; filename="blob"
Content-Type: application/octet-stream


------WebKitFormBoundaryawCJ4mjL1imiO7Ye--

This doesn't work when I try to play it back (presumably due to the unique IDs), so (after sticking testdocument.pdf in the bodies folder) I've done the following:

val documentFeeder = Iterator.continually(Map(
        "documentBoundary" -> (Random.alphanumeric.take(16).mkString),
        "documentUuid" -> ((Random.alphanumeric.take(8).mkString + "-" +
                           Random.alphanumeric.take(4).mkString + "-" + 
                           Random.alphanumeric.take(4).mkString + "-" +
                           Random.alphanumeric.take(4).mkString + "-" +
                           Random.alphanumeric.take(12).mkString).toLowerCase)))

...

.feed(documentFeeder)
// a previous exec POST with a check to grab the documentRevisionId

.exec(http("Post document: upload the file")
            .post("/api/data/files")
            .queryParamMap(Map(
                "revisionId" -> "${documentRevisionId}",
                "uploadType" -> "read_only_file",
                "fileType" -> "application%2Fpdf",
                "fileName" -> "testdocument.pdf",
                "fileSize" -> "10080",
                "copyToEditable" -> "true"))
            .headers(Map(
                "Accept-Encoding" -> "gzip, deflate",
                "Content-Type" -> "multipart/form-data; boundary=----WebKitFormBoundary${documentBoundary}"
                "Origin" -> url))

            .body(StringBody("""------WebKitFormBoundary${documentBoundary}
                                Content-Disposition: form-data; name="resumableChunkNumber"

                                1
                                ------WebKitFormBoundary${documentBoundary}
                                Content-Disposition: form-data; name="resumableChunkSize"

                                1048576
                                ------WebKitFormBoundary${documentBoundary}
                                Content-Disposition: form-data; name="resumableCurrentChunkSize"

                                10080
                                ------WebKitFormBoundary${documentBoundary}
                                Content-Disposition: form-data; name="resumableTotalSize"

                                10080
                                ------WebKitFormBoundary${documentBoundary}
                                Content-Disposition: form-data; name="resumableType"

                                application/pdf
                                ------WebKitFormBoundary${documentBoundary}
                                Content-Disposition: form-data; name="resumableIdentifier"

                                ${documentUuid}
                                ------WebKitFormBoundary${documentBoundary}
                                Content-Disposition: form-data; name="resumableFilename"

                                testdocument.pdf
                                ------WebKitFormBoundary${documentBoundary}
                                Content-Disposition: form-data; name="resumableRelativePath"

                                testdocument.pdf
                                ------WebKitFormBoundary${documentBoundary}
                                Content-Disposition: form-data; name="resumableTotalChunks"

                                1
                                ------WebKitFormBoundary${documentBoundary}
                                Content-Disposition: form-data; name="file"; filename="blob"
                                Content-Type: application/octet-stream
                                Content-Transfer-Encoding: BINARY


                                ------WebKitFormBoundary${documentBoundary}--""")))

Lastly, here's the Node.js code to POST dummy data in our app (I'm using this as reference since I know it works):

var resumableData = {
        resumableChunkNumber: 1,
        resumableChunkSize: 1048576,
        resumableCurrentChunkSize: file.size,
        resumableTotalSize: file.size,
        resumableType: guessBestMimeType(file.name, file.type),
        resumableIdentifier: genUuid(),
        resumableFilename: file.name,
        resumableRelativePath: file.name,
        resumableTotalChunks:1
    };

    var boundaryKey = Math.random().toString(16); // random string

    // the header for the one and only part (need to use CRLF here)
    var resumableBody = '';

    for(var resumablePart in resumableData){
        if(resumableData.hasOwnProperty(resumablePart)){
            resumableBody +=
                '--' + boundaryKey + '\r\n' +
                'Content-Disposition: form-data; name="' + resumablePart + '"\r\n\r\n' +
                resumableData[resumablePart] + '\r\n';
        }
    }
    resumableBody +=
        '--' + boundaryKey + '\r\n' +
        'Content-Disposition: form-data; name="file"; filename="blob"\r\n' +
            // use your file's mime type here, if known
        'Content-Type: application/octet-stream\r\n' +
        'Content-Transfer-Encoding: BINARY\r\n\r\n';

    var resumableEnd = '\r\n--' + boundaryKey + '--';

    var request = https.request({
        method : 'POST',
        host : config.API_HOST + config.BASE_URL,
        port : config.API_PORT,
        path : generateUrl(documentRevision, file, fileType, convertEditable, copyToEditable),
        headers : {
            'Content-Type': 'multipart/form-data; boundary='+boundaryKey,
            'Content-Length' : file.size + Buffer.byteLength(resumableBody + resumableEnd, 'utf-8')
        }
    }, function (response) {
        var data = '';
        response.on('data', function(chunk) {
            data += chunk.toString();
        });
        response.on('end', function() {
            resolve(JSON.parse(data));
        });
        response.on('error', function(err){
            console.error(err);
            reject(err);
        });
    });

    request.write(resumableBody);
    fs.createReadStream(file.path, { bufferSize: 4 * 1024})
        .on('end', function() {
            request.end(resumableEnd);
        })
        .pipe(request, { end: false });

I've been working on troubleshooting this for a couple days and this is my first foray into Scala and Gatling. What am I missing here to get this POST?

One issue (although it could be a red herring) that stands out is that my app uses Content-Length in the header - is this necessary, given that Gatling has omitted it? If it is necessary, am I able to insert the number in Gatling without hardcoding it?

Edit

After reading this post I tried the following:

.exec(http("test post")
            .post("/api/data/files")
            .headers(Headers.headers_6)
            .formParamMap(Map(
                "revisionId" -> "${documentRevisionId}",
                "uploadType" -> "read_only_file",
                "fileType" -> "application%2Fpdf",
                "fileName" -> "testdocument.pdf",
                "fileSize" -> "10080",
                "copyToEditable" -> "true"))
            .bodyPart(StringBodyPart("""{ "resumableChunkNumber": "1",
                "resumableChunkSize": "1048576",
                "resumableCurrentChunkSize": "10080",
                "resumableTotalSize": "10080",
                "resumableType": "application/pdf",
                "resumableIdentifier": "${documentUuid}",
                "resumableFilename": "testdocument.pdf",
                "resumableRelativePath": "testdocument.pdf",
                "resumableTotalChunks": "1" }""")).asJSON
            .bodyPart(RawFileBodyPart("file", "testdocument.pdf")
                .fileName("testdocument.pdf")
                .transferEncoding("binary")).asMultipartForm)

But I still haven't had success. Any suggestions on moving forward?

Answer 1

In case it helps anyone else, I was trying to set a custom boundary when Gatling already does this. Here's what solved my problem:

.exec(http("Post document: upload the file")
            .post("/api/data/files?revisionId=${documentRevisionId}&uploadType=read_only_file&fileType=application%2Fpdf&fileName=testdocument.pdf&fileSize=10080&copyToEditable=true") // ensure testdocument.pdf is in user-files/bodies
            .headers(Headers.headers_6)
            .formParamMap(Map(
                "resumableChunkNumber" -> "1",
                "resumableChunkSize" -> "1048576",
                "resumableCurrentChunkSize" -> "10080",
                "resumableTotalSize" -> "10080",
                "resumableType" -> "application/pdf",
                "resumableIdentifier" -> "${documentUuid}",
                "resumableFilename" -> "testdocument.pdf",
                "resumableRelativePath" -> "testdocument.pdf",
                "resumableTotalChunks" -> "1"))
            .bodyPart(RawFileBodyPart("file", "testdocument.pdf")
                .fileName("testdocument.pdf")
                .transferEncoding("binary")).asMultipartForm)