Spring WebClient: How to stream large byte[] to file?

java spring spring-webflux project-reactor spring-webclient
Dave L. picture Dave L. · May 19, 2019 · Viewed 8.2k times · Source

It seems like it the Spring RestTemplate isn't able to stream a response directly to file without buffering it all in memory. What is the proper to achieve this using the newer Spring 5 WebClient? 

WebClient client = WebClient.create("https://example.com");
client.get().uri(".../{name}", name).accept(MediaType.APPLICATION_OCTET_STREAM)
                    ....?

I see people have found a few workarounds/hacks to this issue with RestTemplate, but I am more interested in doing it the proper way with the WebClient.

There are many examples of using RestTemplate to download binary data but almost all of them load the byte[] into memory.

Answer

Z4- picture Z4- · Mar 17, 2020

With recent stable Spring WebFlux (5.2.4.RELEASE as of writing):

final WebClient client = WebClient.create("https://example.com");
final Flux<DataBuffer> dataBufferFlux = client.get()
        .accept(MediaType.TEXT_HTML)
        .retrieve()
        .bodyToFlux(DataBuffer.class); // the magic happens here

final Path path = FileSystems.getDefault().getPath("target/example.html");
DataBufferUtils
        .write(dataBufferFlux, path, CREATE_NEW)
        .block(); // only block here if the rest of your code is synchronous

For me the non-obvious part was the bodyToFlux(DataBuffer.class), as it is currently mentioned within a generic section about streaming of Spring's documentation, there is no direct reference to it in the WebClient section.

Related questions

spring webclient: retry with backoff on specific error

i'd like to retry the request 3 times after waiting 10sec when response is 5xx. but i don't see a method that I can use. On object WebClient.builder() .baseUrl("...").build().post() .retrieve().bodyToMono(...) i can see methods: retrying on condition …

Read More
How to correctly read Flux<DataBuffer> and convert it to a single inputStream

I'm using WebClient and custom BodyExtractorclass for my spring-boot application WebClient webLCient = WebClient.create(); webClient.get() .uri(url, params) .accept(MediaType.APPLICATION.XML) .exchange() .flatMap(response -> { return response.body(new BodyExtractor()); }) BodyExtractor.java @Override public Mono<T&…

Read More
Spring WebFlux, how can I debug my WebClient POST exchange?

I am having trouble understanding what I've done wrong in constructing my WebClient request. I would like to understand what the actual HTTP request looks like. (e.g., dumping the raw request to console) POST /rest/json/send HTTP/1.1 Host: …

Read More