how to set group name when consuming messages in kafka using command line

apache-kafka kafka-consumer-api
Nilotpal picture Nilotpal · Jul 24, 2016 · Viewed 34.1k times · Source

Any idea how to set group name when consuming messages in kafka using command line.

I tried with the following command :

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic nil_RF2_P2 --from-beginning --config group.id=test1
'config' is not a recognized option

The goal is to find the offset of consumed messages with the following command:

bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --zookeeper localhost:2181 --group test1

Can somebody help in this regards!!

Thanks in advance !!

Answer

vtsamis picture vtsamis · Jul 27, 2017

The simplest solution is:

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic nil_RF2_P2 --from-beginning --consumer-property group.id=test1

In case you specify the flag --from-beginning just remember that the consumer group should not have consumed any records in the past or else your consumer will start consuming from the earliest not consumed record by the specified group (and not from the actual beginning, as you may incorrectly assume).

Related questions

Kafka consumer list

I need to find out a way to ask Kafka for a list of topics. I know I can do that using the kafka-topics.sh script included in the bin\ directory. Once I have this list, I need all the …

Read More
What determines Kafka consumer offset?

I am relatively new to Kafka. I have done a bit of experimenting with it, but a few things are unclear to me regarding consumer offset. From what I have understood so far, when a consumer starts, the offset it …

Read More
How to read data using Kafka Consumer API from beginning?

Please can anyone tell me how to read messages using the Kafka Consumer API from the beginning every time when I run the consumer.

Read More