Is it possible to cache multi-stage docker builds?

docker containers dockerfile google-cloud-build
Matthew Goslett picture Matthew Goslett · Oct 4, 2018 · Viewed 7.5k times · Source

I recently switched to multi-stage docker builds, and it doesn't appear that there's any caching on intermediate builds. I'm not sure if this is a docker limitation, something which just isn't available or whether I'm doing something wrong.

I am pulling down the final build and doing a --cache-from at the start of the new build, but it always runs the full build.

Answer

Matthew Goslett picture Matthew Goslett · Oct 4, 2018

This appears to be a limitation of docker itself and is described under this issue - https://github.com/moby/moby/issues/34715

The workaround is to:

  1. Build the intermediate stages with a --target
  2. Push the intermediate images to the registry
  3. Build the final image with a --target and use multiple --cache-from paths, listing all the intermediate images and the final image
  4. Push the final image to the registry
  5. For subsequent builds, pull the intermediate + final images down from the registry first

Related questions

Using docker-compose to set containers timezones

I have a docker-compose file running a few Dockerfiles to create my containers. I don't want to edit my Dockerfiles to set timezones because they could change at any time by members of my team and I have a docker-compose.…

Read More
Difference between Docker ENTRYPOINT and Kubernetes container spec COMMAND?

Dockerfile has a parameter for ENTRYPOINT and while writing Kubernetes deployment YAML file, there is a parameter in Container spec for COMMAND. I am not able to figure out what's the difference and how each is used?

Read More
Docker filling up storage on macOS

(Post created on Oct 05 '16) I noticed that every time I run an image and delete, my system doesn't return to the original amount of available space. The lifecycle I'm applying to my containers is: > docker build ... > docker …

Read More