Exclude multiple folders using AWS S3 sync

amazon-web-services amazon-s3 s3cmd
Ashish Karpe picture Ashish Karpe · Sep 4, 2015 · Viewed 53.3k times · Source

How to exclude multiple folders while using aws s3 syn ?

I tried :

# aws s3 sync  s3://inksedge-app-file-storage-bucket-prod-env   s3://inksedge-app-file-storage-bucket-test-env --exclude 'reportTemplate/* orders/* customers/*'

But still it's doing sync for folder "customer"

Output :

copy: s3://inksedge-app-file-storage-bucket-prod-env/customers/116/miniimages/IMG_4800.jpg to s3://inksedge-app-file-storage-bucket-test-env/customers/116/miniimages/IMG_4800.jpg
copy: s3://inksedge-app-file-storage-bucket-prod-env/customers/116/miniimages/DSC_0358.JPG to s3://inksedge-app-file-storage-bucket-test-env/customers/116/miniimages/DSC_0358.JPG

Answer

Ashish Karpe picture Ashish Karpe · Sep 4, 2015

At last this worked for me:

aws s3 sync s3://my-bucket s3://my-other-bucket \
            --exclude 'customers/*' \
            --exclude 'orders/*' \
            --exclude 'reportTemplate/*'

Hint: you have to enclose your wildcards and special characters in single or double quotes to work properly. Below are examples of matching characters. for more information regarding S3 commands, check it in amazon here.

*: Matches everything
?: Matches any single character
[sequence]: Matches any character in sequence
[!sequence]: Matches any character not in sequence

Related questions

how to add cache control in AWS S3?

I have moved 20000 files to AWS S3 by s3cmd command. Now i want to add cache-control for all images (.jpg) These files are located in ( s3://bucket-name/images/ ). How can i add cache-control for all images by s3cmd …

Read More
Zip an entire directory on S3

If I have a directory with ~5000 small files on S3, is there a way to easily zip up the entire directory and leave the resulting zip file on S3? I need to do this without having to manually access each …

Read More
How do I delete a versioned bucket in AWS S3 using the CLI?

I have tried both s3cmd: $ s3cmd -r -f -v del s3://my-versioned-bucket/ And the AWS CLI: $ aws s3 rm s3://my-versioned-bucket/ --recursive But both of these commands simply add DELETE markers to S3. The command for removing a …

Read More