Python + Boto: botocore.exceptions.ClientError: An error occurred (404) when calling the HeadObject operation: Not Found

Dan Me picture Dan Me · Jul 4, 2017 · Viewed 22k times · Source

I’m downloading files from AWS S3 Bucket like so:

import boto3

s3client = boto3.client("s3")
s3 = boto3.resource('s3')

bucket_name = 'practice_bucket'
bucket = s3.Bucket(bucket_name)

for obj in bucket.objects.all():
    filename = obj.key.rsplit('/')[-1]
    s3client.download_file(bucket_name, obj.key, "/txt/" + filename)

When attempting to place some files under a subdirectory, e.g. /txt/ I get the error: botocore.exceptions.ClientError: An error occurred (404) when calling the HeadObject operation: Not Found.

Oddly, it works on other file types using the same method, but doesn’t work for ones with .json extension.

What could be the issue? I even tried without placing them in an absolute subdirectory path, and I get no error and downloads the file onto the same directory as the script downloading. But when I actually define the path to download the file to, I get the error.

Thank you in advance and will be sure to vote up/accept answer

Answer