What is the Difference between file_upload() and put_object() when uploading files to S3 using boto3

python amazon-web-services amazon-s3 boto3
Tushar Niras picture Tushar Niras · May 2, 2017 · Viewed 8.1k times · Source

I'm using boto3 and trying to upload files. It will be helpful if anyone will explain exact difference between file_upload() and put_object() s3 bucket methods in boto3 ?

  • Is there any performance difference?
  • Does anyone among these handles multipart upload feature in behind the scenes?
  • What are the best use cases for both?

Answer

garnaat picture garnaat · May 2, 2017

The upload_file method is handled by the S3 Transfer Manager, this means that it will automatically handle multipart uploads behind the scenes for you, if necessary.

The put_object method maps directly to the low-level S3 API request. It does not handle multipart uploads for you. It will attempt to send the entire body in one request.

Related questions

How to write a file or data to an S3 object using boto3

In boto 2, you can write to an S3 object using these methods: Key.set_contents_from_string() Key.set_contents_from_file() Key.set_contents_from_filename() Key.set_contents_from_stream() Is there a boto 3 equivalent? What is the …

Read More
Retrieving subfolders names in S3 bucket from boto3

Using boto3, I can access my AWS S3 bucket: s3 = boto3.resource('s3') bucket = s3.Bucket('my-bucket-name') Now, the bucket contains folder first-level, which itself contains several sub-folders named with a timestamp, for instance 1456753904534. I need to know the …

Read More
Boto3 to download all files from a S3 Bucket

I'm using boto3 to get files from s3 bucket. I need a similar functionality like aws s3 sync My current code is #!/usr/bin/python import boto3 s3=boto3.client('s3') list=s3.list_objects(Bucket='my_bucket_name')[…

Read More