How to specify wildcard artifacts subdirectories in .gitlab-ci.yml?

swalex picture swalex · Jun 24, 2016 · Viewed 30.9k times · Source

I'm using GitLab CI to build a C# solution and try to pass some build artifacts from one build stage to another.

The problem is, that the artifacts are not located in a single directory but in different subdirectories, which however all have the same names bin/ or obj/.

My .gitlab-ci.yml looks like the following:

...
stages:
  - build
  - test

build:
  stage: build
  script:
    CALL %MSBuild% ...
  artifacts:
    paths:
      - /**/bin/
      - /**/obj/
    expire_in: 6 hrs

test:
  stage: test
  dependencies:
    - build
  ...

I tried to capture the artifacts using different ways, e.g.

**/bin/
**/obj/

(invalid syntax), or

.*/bin/
.*/obj/

but that one did not find any artifacts, just as /**/bin/ and /**/obj/, giving me following errors:

Uploading artifacts...
WARNING: /**/bin/: no matching files
WARNING: /**/obj/: no matching files

How can I specify a subdirectory pattern to be scanned for artifacts? Or is this even possible at all?

Simply using

artifacts:
  untracked: true

is not an option, because of a huge untracked packages/ subdirectory, which causes artifacts upload to fail because of a too large archive:

Uploading artifacts...
untracked: found 4513 files                        
ERROR: Uploading artifacts to coordinator... too large archive  id=36 responseStatus=413 Request Entity Too Large token=...
FATAL: Too large

Answer

swalex picture swalex · Jun 27, 2016

The gitlab-ci-multi-runner build runner is built using Go and currently uses filepath.Glob() to scan for any specified artifacts in file_archiver.go.

Go doesn't seem to support the double star glob expression as discussed in another question here at SO. So there seem to be no way to use a full-featured **/bin expression at the moment.

Because however all my projects are located at the same level below the solution root, it is still possible to use something like

artifacts:
  paths:
    - "*/bin"
    - "*/obj"

Note that the quotes (") seem to be required, as well as no trailing path separator at the end.

It should also be possible to explicitly add more levels by adding more globbing expressions (as described here):

paths:
  ...
  - "*/obj"
  - "*/*/bin"
  - "*/*/obj"
  ...

GitLab is tracking this issue here, and will possibly be fixed in a future version.