Role of docker-in-docker (dind) service in gitlab ci

pkaramol picture pkaramol · Nov 14, 2017 · Viewed 16.8k times · Source

According to the official gitlab documentation, one way to enable docker build within ci pipelines, is to make use of the dind service (in terms of gitlab-ci services).

However, as it is always the case with ci jobs running on docker executors, the docker:latest image is also needed.

Could someone explain:

  • what is the difference between the docker:dind and the docker:latest images?
  • (most importantly): why are both the service and the docker image needed (e.g. as indicated in this example, linked to from the github documentation) to perform e.g. a docker build whithin a ci job? doesn't the docker:latest image (within which the job will be executed!) incorporate the docker daemon (and I think the docker-compose also), which are the tools necessary for the commands we need (e.g. docker build, docker push etc)?

Unless I am wrong, the question more or less becomes:

Why a docker client and a docker daemon cannot reside in the same docker (enabled) container

Answer

saraedum picture saraedum · Feb 11, 2018

what is the difference between the docker:dind and the docker:latest images?

  • docker:latest contains everything necessary to connect to a docker daemon, i.e., to run docker build, docker run and such. It also contains the docker daemon but it's not started as its entrypoint.
  • docker:dind builds on docker:latest and starts a docker daemon as its entrypoint.

So, their content is almost the same but through their entrypoints one is configured to connect to tcp://docker:2375 as a client while the other is meant to be used for a daemon.

why are both the service and the docker image needed […]?

You don't need both. You can just use either of the two, start dockerd as a first step, and then run your docker build and docker run commands as usual like I did here; apparently this was the original approach in gitlab at some point. But I find it cleaner to just write service: docker:dind instead of having a before_script to setup dockerd. Also you don't have to figure out how to start & install dockerd properly in your base image (if you are not using docker:latest.)

Declaring the service in your .gitlab-ci.yml also lets you swap out the docker-in-docker easily if you know that your runner is mounting its /var/run/docker.sock into your image. You can set the protected variable DOCKER_HOST to unix:///var/run/docker.sock to get faster builds. Others who don't have access to such a runner can still fork your repository and fallback to the dind service without modifying your .gitlab-ci.yml.