Kafka Consumer's poll() method gets blocked

Question

I'm new to Kafka 0.9 and testing some features I realized a strange behaviour in the Java implemented Consumer (KafkaConsumer).

The Kafka broker is located in an Ambari external machine.

Even thou I could implement a Producer and start sending messages to the external broker, I have no clue why, when the consumer tries to read the events (poll), it gets stuck.

I know the producer is working well, since I do can consume messages through the console consumer (which is working locally on ambari). But when I execute the Java Consumer, nothing happens, just gets stuck. Debugging the code I could see that it gets blocked at the poll() line:

    ConsumerRecords<String, String> records = consumer.poll(100);

The timeout does nothing, by the way. Doesn't matter if you put 0, 100 or 1000 ms, the consumer gets blocked in this line and does not timeout nor throw exceptions.

I tried all kind of alternative properties, such as advertised.host.name, advertised.listener,... and so on, with zero luck.

Any help would be highly appreciated. Thanks in advance!

Answer 1

Reason might be the machine where your consumer code is running is unable to connect to zookeeper. Try running the same consumer code on the machine where your Kafka is installed (i tried this and worked for me). I also solved problem by mentioning the below properties in server.properties file: advertised.host.name="ip address which you want to expose" // in my case it is public ip of ec2 machine, i have kafka and zookeeper installed on same ec2. advertised.port=9092 ConsumerRecords<String, String> records = consumer.poll(100); The above statement doesn't mean consumer will time out after 100 ms, it is the polling period. Whatever data it captures in 100 ms is read into records collection.