Can I delete data records or shards from amazon Kinesis without deleting stream?

java amazon-web-services amazon-kinesis
Kishore Kumar Korada picture Kishore Kumar Korada · Aug 23, 2014 · Viewed 17k times · Source

I know data records in Kinesis Stream will be deleted automatically in 24 hrs. But in my application when ever I write some data into stream, for the 2nd time if I want to write some other data, Data inserted first should be deleted. Please anyone help me since I am new to using AWS Kinesis Stream...I didn't get any help from Kinesis Service API...

Answer

mk-baku picture mk-baku · Sep 9, 2014

You can not delete previously inserted data from stream, but you can read data using KCL. KCL will create checkpoint every after one data slot read, so whenever you go for next slot of new data, KCL will read it from last checkpoint created in dynamodb table, so previously read data will be not included in next slot.

Related questions

How to put data from server to Kinesis Stream

I am new to Kinesis. Reading out the documentation i found i can create the Kinesis Stream to get data from Producer. Then using KCL will read this data from Stream to further processing. I understand how to write the …

Read More
Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect)

I have a java project that runs on a webserver. I always hit this exception. I read some documentation, and found that pessimistic locking (or optimistic, but I read that pessimistic is better) is the best way to prevent this …

Read More
How to check if a specified key exists in a given S3 bucket using Java

I would like to check if a key exists in a given bucket using Java. I looked at the API but there aren't any methods that are useful. I tried to use getObject but it threw an exception.

Read More