Issue in connecting kafka from outside
I am using hortonwork Sandbox for kafka server trying to connect kafka from eclipse with java code . Use this configuration to connect to producer to send the message
metadata.broker.list=sandbox.hortonworks.com:45000
serializer.class=kafka.serializer.DefaultEncoder
zk.connect=sandbox.hortonworks.com:2181
request.required.acks=0
producer.type=sync
where sandbox.hortonworks.com is sandboxname to whom i connect
in kafka server.properties I changed this configuration
host.name=sandbox.hortonworks.com
advertised.host.name=System IP(on which my eclipse is running)
advertised.port=45000
did the port forwarding also ,
I am able to connect to kafka server from eclipse but while sending the message get the exception Exception"Failed to send messages after 3 tries."
Answer
First make sure you have configured host-only network for your Hortonworks Sandbox VM as described here:
http://hortonworks.com/community/forums/topic/use-host-only-networking-for-the-virtual-machine/
After doing this your sandbox VM should get a IP (e.g. 192.168.56.101) and it should be reachable from your host via SSH like
$ ssh [email protected]
Then open Ambari at http://192.168.56.101:8080/ and change the Kafka configuration to
listeners=PLAINTEXT://0.0.0.0:6667
advertised.listeners=PLAINTEXT://192.168.56.101:6667
The latter property must be added in the section "Custom kafka-broker" (See also http://hortonworks.com/community/forums/topic/ambari-alerts-how-to-change-kafka-port/).
Then start/restart Kafka via Ambari. You should now be able to access Kafka from outside the Hortonworks Sandbox VM. You can test this (from outside of the sandbox VM) using e.g. the Kafka console producer from the Kafka distribution like
$ bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
$ bin/kafka-console-producer.sh --topic test --broker-list 192.168.56.101:6667