AWS CodeBuild buildspec.yml get all files and subfolders recursively

Question

I'm trying to use AWS CodeBuild to get all files and subfolders inside a nested public folder and deploy to an S3 bucket using CodePipeline. I was able to hook them all together but struggling to configure the buildspec.yml file to get the output I want.

My folder structure:

<path>/public/

├── 404.html
├── css
│   ├── ...
├── fonts
│   ├── bootstrap
│   │   ├── ...
│   ├── icomoon
│   │   ├── icomoon
│   │   │   ├── ...
│   └── simple-line-icons
│       ├── ...
├── images
│   ├── ...
├── index.html
├── index.xml
├── js
│   ├── ...
└── tags
    └── index.xml

I need to put everything (including the subfolders) inside the public folder into the root of an S3 bucket.

So far I've tried following the docs here, here and here. I've tried using:

• **/* to get everything recursively inside the folder public but the S3 bucket will have the path to the folder so the index.html is not in the root.
• discard-paths: yes to remove the path to the public folder but inside the S3 bucket, all files are there in the root, no sub-folder structure is kept.
• base-directory: as described here.
• artifacts: secondary-artifacts: artifact1: files: - directory/file artifact2: files: - directory/file2 to keep the folder structure but my build failed.
• Different combinations of all the syntaxes above but my build just failed.

Answer 1

TL;DR You probably don't need to use "discard-paths: yes". So, drop "discard-paths: yes" and instead use "base-directory" with "**/*" for glob pattern. Read on for more reasoning why.

So yeah, as it usually happens, the culprit was pretty trivial. In my case the directory structure looked like this:

dist/
|-- favicon.ico
|-- index.html
|-- js
|   `-- app.js
`-- revision.txt

So in order for the artifact to contain everything from the "dist" directory whilst preserving the nested directories structure, the buildspec had to look like this:

artifacts:
  files:
    - '**/*'
  base-directory: 'dist'

Essentially "base-directory" parameter behaves as unix's "cd" command, so at first CodeBuild jumps to that directory and only after that resolves the glob pattern, in this case - **/*. If you also happen to use "discard-paths" then what happens is that, presumably, CodeBuild will change current directory to "dist", resolve the files structure correctly but then remove all path prefixes from the resolved paths, thus the result will be a plain directory structure with no nesting.

Hope that helps!