Kafka : How to connect kafka-console-consumer to fetch remote broker topic content?

notrai picture notrai · Nov 7, 2015 · Viewed 19k times · Source

I have setup a kafka zookeeper and 3 brokers on one machine on ec2 with ports 9092..9094 and am trying to consume the topic content from another machine. The ports 2181 (zk), 9092, 9093 and 9094 (servers) are open to the consumer machine. I can even do a bin/kafka-topics.sh --describe --zookeeper 172.X.X.X:2181 --topic remotetopic which gives me

Topic:remotetopic PartitionCount:1 ReplicationFactor:3 Configs: Topic: remotetopic Partition: 0 Leader: 2 Replicas: 2,0,1 Isr: 2,0,1 Blockquote

However when i do bin/kafka-console-consumer.sh --zookeeper 172.X.X.X:2181 --from-beginning --topic remotetopic I get

WARN Fetching topic metadata with correlation id 0 for topics [Set(remotetopic)] from broker [id:0,host:localhost,port:9092] failed (kafka.client.ClientUtils$) java.nio.channels.ClosedChannelException

Why is the consumer trying to read from localhost? Is there any option or commandline or default file where this is read from; and I can change it?

Any help would be appreciated!

Answer

r2d3 picture r2d3 · Feb 11, 2016

Had the same problem. Fixed it by setting the property 'advertised.host.name' as instructed by the comments in the kafka configuration (config/server.properties), and restarting the kafka server.