Filter S3 list-objects results to find a key matching a pattern

amazon-s3 pattern-matching aws-cli jmespath
dnlbrky picture dnlbrky · Dec 3, 2014 · Viewed 37.4k times · Source

I would like to use the AWS CLI to query the contents of a bucket and see if a particular file exists, but the bucket contains thousands of files. How can I filter the results to only show key names that match a pattern? For example:

aws s3api list-objects --bucket myBucketName --query "Contents[?Key==*mySearchPattern*]"

Answer

dnlbrky picture dnlbrky · Dec 3, 2014

The --query argument uses JMESPath expressions. JMESPath has an internal function contains that allows you to search for a string pattern.

This should give the desired results:

aws s3api list-objects --bucket myBucketName --query "Contents[?contains(Key, `mySearchPattern`)]"

(With Linux I needed to use single quotes ' rather than back ticks ` around mySearchPattern.)

If you want to search for keys starting with certain characters, you can also use the --prefix argument:

aws s3api list-objects --bucket myBucketName --prefix "myPrefixToSearchFor"

Related questions

Downloading an entire S3 bucket?

I noticed that there doesn't seem to be an option to download an entire S3 bucket from the AWS Management Console. Is there an easy way to grab everything in one of my buckets? I was thinking about making the …

Read More
Listing contents of a bucket with boto3

How can I see what's inside a bucket in S3 with boto3? (i.e. do an "ls")? Doing the following: import boto3 s3 = boto3.resource('s3') my_bucket = s3.Bucket('some/path/') returns: s3.Bucket(name='some/path/…

Read More
Setting up FTP on Amazon Cloud Server

I am trying to set up FTP on Amazon Cloud Server, but without luck. I search over net and there is no concrete steps how to do it. I found those commands to run: $ yum install vsftpd $ ec2-authorize default …

Read More