Append data to an S3 object

amazon-web-services amazon-s3 aws-sdk amazon-kinesis-firehose
Theodore picture Theodore · Jan 21, 2017 · Viewed 67.8k times · Source

Let's say that I have a machine that I want to be able to write to a certain log file stored on an S3 bucket.

So, the machine needs to have writing abilities to that bucket, but, I don't want it to have the ability to overwrite or delete any files in that bucket (including the one I want it to write to).

So basically, I want my machine to be able to only append data to that log file, without overriding it or downloading it.

Is there a way to configure my S3 to work like that? Maybe there's some IAM policy I can attach to it so it will work like I want?

Answer

duskwuff -inactive- picture duskwuff -inactive- · Jan 21, 2017

Unfortunately, you can't.

S3 doesn't have an "append" operation.* Once an object has been uploaded, there is no way to modify it in place; your only option is to upload a new object to replace it, which doesn't meet your requirements.

*: Yes, I know this post is a couple of years old. It's still accurate, though.

Related questions

Invoke multiple aws lambda functions

How can we invoke multiple AWS Lambda functions one after the other ?For example if an AWS Lambda chain consists of 8 separate lambda functions and each simulate a 1 sec processing event and then invoke the next function in the chain.

Read More
How to check if a specified key exists in a given S3 bucket using Java

I would like to check if a key exists in a given bucket using Java. I looked at the API but there aren't any methods that are useful. I tried to use getObject but it threw an exception.

Read More
The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256

I get an error AWS::S3::Errors::InvalidRequest The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256. when I try upload file to S3 bucket in new Frankfurt region. All works properly with US Standard region. …

Read More