Is boto3 client thread-safe

python amazon-s3 aws-sdk boto3
Andrey Novosad picture Andrey Novosad · Oct 15, 2018 · Viewed 12.1k times · Source

Is boto3 low level client for S3 thread-safe? Documentation is not explicit about it.

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#client

A similar issue is discussed in Github

https://github.com/boto/botocore/issues/1246

But still there is no answer from maintainers.

Answer

Skam picture Skam · Oct 15, 2018

If you take a look at the Multithreading/Processing documentation for boto3 you can see that they recommend one client per session as there is shared data between instance that can be mutated by individual threads.

It also looks like there's an open github issue for this exact question. https://github.com/boto/botocore/issues/1246

Related questions

Listing contents of a bucket with boto3

How can I see what's inside a bucket in S3 with boto3? (i.e. do an "ls")? Doing the following: import boto3 s3 = boto3.resource('s3') my_bucket = s3.Bucket('some/path/') returns: s3.Bucket(name='some/path/…

Read More
How to write a file or data to an S3 object using boto3

In boto 2, you can write to an S3 object using these methods: Key.set_contents_from_string() Key.set_contents_from_file() Key.set_contents_from_filename() Key.set_contents_from_stream() Is there a boto 3 equivalent? What is the …

Read More
How to upload a file to directory in S3 bucket using boto

I want to copy a file in s3 bucket using python. Ex : I have bucket name = test. And in the bucket, I have 2 folders name "dump" & "input". Now I want to copy a file from local directory to S3 "…

Read More