Getting GitLab CI to clone private repositories

Question

I have GitLab & GitLab CI set up to host and test some of my private repos. For my composer modules under this system, I have Satis set up to resolve my private packages.

Obviously these private packages require an ssh key to clone them, and I have this working in the terminal - I can run composer install and get these packages, so long as I have the key added with ssh-add in the shell.

However, when running my tests in GitLab CI, if a project has any of these dependencies the tests will not complete as my GitLab instance needs authentication to get the deps (obviously), and the test fails saying Host key verification failed.

My question is how do I set this up so that when the runner runs the test it can authenticate to gitlab without a password? I have tried putting a password-less ssh-key in my runners ~/.ssh folder, however the build wont even add the key, "eval ssh-agent -s" followed by ssh-add seems to fail saying the agent isn't running...

Answer 1

See also other solutions:

• git submodule permission (see Marco A.'s answer)
• job token and override repo in git config (see a544jh's answer)

---

Here a full howto with SSH keys:

General Design

• generating a pair of SSH keys
• adding the private one as a secure environment variable of your project
• making the private one available to your test scripts on GitLab-CI
• adding the public one as a deploy key on each of your private dependencies

Generating a pair of public and private SSH keys

Generate a pair of public and private SSH keys without passphrase:

ssh-keygen -b 4096 -C "<name of your project>" -N "" -f /tmp/name_of_your_project.key

Adding the private SSH key to your project

You need to add the key as a secure environment variable to your project as following:

• browse https://<gitlab_host>/<group>/<project_name>/variables
• click on "Add a variable"
• fill the text field Key with SSH_PRIVATE_KEY
• fill the text field Value with the private SSH key itself
• click on "Save changes"

Exposing the private SSH key to your test scripts

In order to make your private key available to your test scripts you need to add the following to your .gitlab-ci.yml file:

before_script:
  # install ssh-agent
  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
  # run ssh-agent
  - eval $(ssh-agent -s)
  # add ssh key stored in SSH_PRIVATE_KEY variable to the agent store
  - ssh-add <(echo "$SSH_PRIVATE_KEY")
  # disable host key checking (NOTE: makes you susceptible to man-in-the-middle attacks)
  # WARNING: use only in docker container, if you use it with shell you will overwrite your user's ssh config
  - mkdir -p ~/.ssh
  - echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config

Code Snippet comes from GitLab documentation

Adding the public SSH key as a deploy key to all your private dependencies

You need to register the public SSH key as deploy key to all your private dependencies as following:

• browse https://<gitlab_host>/<group>/<dependency_name>/deploy_keys
• click on "New deploy key"
• fill the text field Title with the name of your project
• fill the text field Key with the public SSH key itself
• click on "Create deploy key"