Kafka bootstrap-servers vs zookeeper in kafka-console-consumer

apache-kafka kafka-consumer-api
nico picture nico · Jan 21, 2017 · Viewed 41.1k times · Source

I'm trying to test run a single Kafka node with 3 brokers & zookeeper. I wish to test using the console tools. I run the producer as such:

kafka-console-producer --broker-list localhost:9092,localhost:9093,localhost:9094 --topic testTopic

Then I run the consumer as such:

kafka-console-consumer --zookeeper localhost:2181 --topic testTopic --from-beginning

And I can enter messages in the producer and see them in the consumer, as expected. However, when I run the updated version of the consumer using bootstrap-server, I get nothing. E.g

kafka-console-consumer --bootstrap-server localhost:9092,localhost:9093,localhost:9094 --topic testTopic --from-beginning

This worked fine when I had one broker running on port 9092 so I'm thoroughly confused. Is there a way I can see what zookeeper is providing as the bootstrap server? Is the bootstrap server different from the broker list? Kafka compiled with Scala 2.11.

Answer

nico picture nico · Jan 23, 2017

I have no idea what was wrong. Likely I put Kafka or Zookeeper in a weird state. After deleting the topics in the log.dir of each broker AND the zookeeper topics in /brokers/topics then recreating the topic, Kafka consumer behaved as expected.

Related questions

Kafka consumer list

I need to find out a way to ask Kafka for a list of topics. I know I can do that using the kafka-topics.sh script included in the bin\ directory. Once I have this list, I need all the …

Read More
What determines Kafka consumer offset?

I am relatively new to Kafka. I have done a bit of experimenting with it, but a few things are unclear to me regarding consumer offset. From what I have understood so far, when a consumer starts, the offset it …

Read More
How to read data using Kafka Consumer API from beginning?

Please can anyone tell me how to read messages using the Kafka Consumer API from the beginning every time when I run the consumer.

Read More