POST InputStream with RestTemplate

Question

I have a client that needs to POST a large number of large json files to a server. I've been able to get it working by reading each of the files into memory and posting the entire file with RestTemplate. However, the client quickly runs out of memory dealing with the large json files. I want to switch to a streaming approach but can't figure out how to use a FileInputStream with the RestTemplate properly. I found this question and used the code given in the accepted answer but I'm still seeing memory usage and OutOfMemory exceptions that lead me to believe that it is not streaming the files but still reading them into memory entirely. What am I doing wrong? Here is what I have currently:

final InputStream fis = ApplicationStore.class.getResourceAsStream(path);

final RequestCallback requestCallback = new RequestCallback() {
    @Override
    public void doWithRequest(final ClientHttpRequest request) throws IOException {
        request.getHeaders().add("Content-type", "application/json");
        IOUtils.copy(fis, request.getBody());
    }
};

final RestTemplate restTemplate = new RestTemplate();
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
requestFactory.setBufferRequestBody(false);     
restTemplate.setRequestFactory(requestFactory);     
final HttpMessageConverterExtractor<String> responseExtractor =
         new HttpMessageConverterExtractor<String>(String.class, restTemplate.getMessageConverters());

restTemplate.execute("http://" + host + ":8080/upads-data-fabric" + "/ruleset", httpMethod, requestCallback, responseExtractor);

Answer 1

Don't. Use a Resource in combination with an appropriate RestTemplate#exchange method.

Create an HttpEntity with the Resource as the body. There's ClassPathResource to represent class path resources. The RestTemplate, by default, registers a ResourceHttpMessageConverter.

Internally, the ResourceHttpMessageConverter streams the request content to the opposite end of the connection with StreamUtils#copy(InputStream, OutputStream) with a buffer size that's currently set to 4096.