I can't seem to get the cache or the artifacts to carry over between jobs in gitlab CI. I suspect it's something to do with my configuration, but I'm not sure what. I'm running gitlab and gitlab-ci-multirunner, both in docker, using the following docker-compose config. I've left out the database config and some environment variables for brevity:
version: '2'
services:
gitlab:
image: sameersbn/gitlab:8.5.1
links:
- redis:redisio
- postgresql:postgresql
ports:
- "10080:80"
- "10022:22"
environment:
...
volumes:
- gitlab_data:/home/git/data
gitlab-ci-runner:
restart: always
image: gitlab/gitlab-runner
volumes:
- gitlab_runner_config_data:/etc/gitlab-runner
- /var/run/docker.sock:/var/run/docker.sock
- /etc/nginx/ssl/gitlab.crt:/etc/gitlab-runner/certs/ca.crt
- /etc/ssh:/ssh
links:
- gitlab:gitlab
redis:
...
postgresql:
...
volumes:
postgresql_data:
redis_data:
gitlab_data:
gitlab_runner_config_data:
The runner configuration (config.toml
) is:
concurrent = 1
[[runners]]
name = "docker"
url = <public gitlab url>/ci
token = <gitlab token>
tls-ca-file = "/etc/gitlab-runner/certs/ca.crt"
executor = "docker"
[runners.docker]
image = "docker-bash"
volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
The docker-bash
image referred to is just the official docker:1.10 image with bash installed.
My build process consists of 3 steps:
npm install
and the tests in the official node:5 image. For now, I've left this step out in order to test the deploy.The .gitlab-ci.yml
file looks like this:
variables:
FULL_IMAGE_TAG: deploy-$CI_BUILD_REF_NAME:$CI_BUILD_ID-$CI_BUILD_REF
IMAGE_FILE: deploy-$CI_BUILD_REF_NAME.tar.gz
cache:
paths:
- $IMAGE_FILE
build:
stage: build
script:
- docker build -t $FULL_IMAGE_TAG .
- docker save $FULL_IMAGE_TAG | gzip -cf - > $IMAGE_FILE
artifacts:
paths:
- $IMAGE_FILE
deploy:
stage: deploy
image: ansible-ssh
script:
- ls
- ansible-playbook -e image_file=$IMAGE_FILE -e branch=$CI_BUILD_REF_NAME -e full_image_name=$FULL_IMAGE_TAG deploy-playbook.yml
only:
- develop
- master
As you can see, the compressed docker image is referred to here both in the cache and artifacts sections, but is not actually available in the deploy step, where ansible is supposed to copy it to the remote machine. I have tried including an ls
command so check the folder contents and the file is clearly not there, but it is definitely built and I can download it from the gitlab UI. Here is the log from the deploy job:
gitlab-ci-multi-runner 1.0.4 (014aa8c)
Using Docker executor with image ansible-ssh ...
Pulling docker image ansible-ssh ...
WARNING: Cannot pull the latest version of image ansible-ssh : Error: image library/ansible-ssh not found
WARNING: Locally found image will be used instead.
Running on runner-59d43cf3-project-8-concurrent-0 via 381c2ea97744...
Fetching changes...
Removing artifacts.zip
Removing deploy-develop.tar.gz
HEAD is now at 6009bd0 test
Checking out 6009bd0f as develop...
HEAD is now at 6009bd0... test
$ ls
Dockerfile
deploy-playbook.yml
server
$ ansible-playbook -e image_file=$IMAGE_FILE -e branch=$CI_BUILD_REF_NAME -e full_image_name=$FULL_IMAGE_TAG deploy-playbook.yml
Using /etc/ansible/ansible.cfg as config file
1 plays in deploy-playbook.yml
PLAY ***************************************************************************
TASK [setup] *******************************************************************
ok: [deploy-host]
TASK [copy docker image] *******************************************************
task path: /builds/test/test/deploy-playbook.yml:44
fatal: [deploy-host]: FAILED! => {"changed": false, "failed": true, "msg": "could not find src=/builds/test/test/deploy-develop.tar.gz"}
NO MORE HOSTS LEFT *************************************************************
to retry, use: --limit @deploy-playbook.retry
PLAY RECAP *********************************************************************
deploy-host : ok=1 changed=0 unreachable=0 failed=1
ERROR: Build failed with: exit code 1
I suspect that I am not setting up or using the runner correctly, but I can't find much in the documentation for anything beyond really simple cases and I don't know the tool well enough to know how it all fits together under the hood.
Caching is not designed to pass files between stages of a build.
From the doc
cache : Define list of files that should be cached between subsequent runs
I think what you need is in fact in progress : WIP: Download build artifacts from previous stages and restore them in context of the build (Technology Preview)