difference between groupid and consumerid in Kafka consumer

apache-kafka kafka-consumer-api
Gnana picture Gnana · Dec 31, 2015 · Viewed 43.6k times · Source

I noticed that the Consumer configuration has two IDs. One is group.id (mandatory) and second one is consumer.id (not Mandatory).

What is the difference between these 2 IDs?

Answer

Aravind Yarram picture Aravind Yarram · Jan 1, 2016

Consumers groups is a Kafka abstraction that enables supporting both point-to-point and publish/subscribe messaging. A consumer can join a consumer group (let us say group_1) by setting its group.id to group_1. Consumer groups is also a way of supporting parallel consumption of the data i.e. different consumers of the same consumer group consume data in parallel from different partitions.

In addition to group.id, each consumer also identifies itself to the Kafka broker using consumer.id. This is used by Kafka to identify the currently ACTIVE consumers of a particular consumer group.

Read this documentation for more details.

Related questions

Kafka consumer list

I need to find out a way to ask Kafka for a list of topics. I know I can do that using the kafka-topics.sh script included in the bin\ directory. Once I have this list, I need all the …

Read More
What determines Kafka consumer offset?

I am relatively new to Kafka. I have done a bit of experimenting with it, but a few things are unclear to me regarding consumer offset. From what I have understood so far, when a consumer starts, the offset it …

Read More
How to read data using Kafka Consumer API from beginning?

Please can anyone tell me how to read messages using the Kafka Consumer API from the beginning every time when I run the consumer.

Read More