Kafka consumer offset max value?

offset apache-kafka kafka-consumer-api
rodolk picture rodolk · Dec 22, 2015 · Viewed 9.4k times · Source

I was googling and reading Kafka documentation but I couldn't find out the max value of a consumer offset and whether there is offset wraparound after max value. I understand offset is an Int64 value so max value is 0xFFFFFFFFFFFFFFFF. If there is wraparound, how does Kafka handle this situation?

Answer

Matt Hulse picture Matt Hulse · Dec 22, 2015

According to this post, the offset is not reset:

We don't roll back offset at this moment. Since the offset is a long, it can last for a really long time. If you write 1TB a day, you can keep going for about 4 million days.

Plus, you can always use more partitions (each partition has its own offset).

So as Luciano said, probably not worth worrying about.

Related questions

Kafka-consumer. commitSync vs commitAsync

The quote from https://www.safaribooksonline.com/library/view/kafka-the-definitive/9781491936153/ch04.html#callout_kafka_consumers__reading_data_from_kafka_CO2-1 The drawback is that while commitSync() will retry the commit until it either succeeds or encounters a non-retriable failure, …

Read More
kafka-python consumer start reading from offset (automatically)

I'm trying to build an application with kafka-python where a consumer reads data from a range of topics. It is extremely important that the consumer never reads the same message twice, but also never misses a message. Everything seems to …

Read More
How does kafka consumer auto commit work?

I am reading this one: Automatic Commit The easiest way to commit offsets is to allow the consumer to do it for you. If you configure enable.auto.commit=true, then every five seconds the consumer will commit the largest …

Read More