s3- boto- list files within a bucket by upload time

python amazon-s3 boto bucket
Ron D. picture Ron D. · Nov 1, 2011 · Viewed 7.6k times · Source

I need to download every hour 100 newest files from s3 server.

bucketList = bucket.list(PREFIX)

The code above creates list of the files but it is not depend on the uploading time of the files, since it lists by file name?

I can do nothing with file name. It is given randomly.

Thanks.

Answer

kevinharvey picture kevinharvey · May 7, 2012

How big is the list? You could sort the list on the 'last_modified' attr of the Key

orderedList = sorted(bucketList, key=lambda k: k.last_modified)
keysYouWant = orderedList[0:100]

If your list is HUGE this may not be efficient. Check out the inline docs for the list() function in boto.s3.bucket.Bucket.

Related questions

How do I upload a CSV file in myBucket and Read File in S3 AWS using Python

How do I upload a CSV file from my local machine to my AWS S3 bucket and read that CSV file? bucket = aws_connection.get_bucket('mybucket') #with this i am able to create bucket folders = bucket.list("","/") for folder …

Read More
How can I check that a AWS S3 bucket exists?

Simple question here? ... How can I check with boto that a AWS bucket exists? ... preferably by providing the path? ... here is the approach I feel like taking: def bucket_exists(self, bucket_name): connection = boto.s3.connection.S3Connection('&…

Read More
Listing contents of a bucket with boto3

How can I see what's inside a bucket in S3 with boto3? (i.e. do an "ls")? Doing the following: import boto3 s3 = boto3.resource('s3') my_bucket = s3.Bucket('some/path/') returns: s3.Bucket(name='some/path/…

Read More