How to cache package manager downloads for docker builds?

Question

If I run composer install from my host, I hit my local composer cache:

  - Installing deft/iso3166-utility (1.0.0)
    Loading from cache

Yet when building a container having in its Dockerfile:

RUN composer install -n -o --no-dev

I download all the things, e.g.:

  - Installing deft/iso3166-utility (1.0.0)
    Downloading: 100%         

It's expected, yet I like to avoid it. As even on a rebuilt, it would also download everything again.

I would like to have a universal cache for composer that I could also reshare for other docker projects.

I looked into this and found the approach to define a volume in the Dockerfile:

ENV COMPOSER_HOME=/var/composer
VOLUME /var/composer

I added that to my Dockerfile, and expected to only download the files once, and hit the cache afterwards.

Yet when I modify my composer, e.g. remove the -o flag, and rerun docker build ., I expected to hit the cache on build, yet I still download the vendors again.

How are volumes supposed to work to have a data cache inside a docker container?

Answer 1

Use the experimental feature : Docker buildkit (Supported Since docker 18.09, docker-compose 1.25.4)

In your dockerfile

# syntax=docker/dockerfile:experimental
FROM ....
# ......  
RUN --mount=type=cache,target=/var/composer composer install -n -o --no-dev

Now before building, make sure the env var is exported:

export DOCKER_BUILDKIT=1
docker build ....

If you are using docker-compose, make sure to export also COMPOSE_DOCKER_CLI_BUILD :

export COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1
docker-compose build ...

If it does not work with docker-compose, make sure your docker-compose version is above 1.25.4

docker-compose version