Why I cannot connect to Kafka from outside?

docker amazon-ec2 apache-kafka virtual-machine
sms_1190 picture sms_1190 · Nov 28, 2014 · Viewed 52.2k times · Source

I am running kafka on ec2 instance. So amazon ec2 instance has two ips one is internal ip and second one is for external use.

I created producer from local machine, but it redirect to internal ip and give me connection unsuccessful error. Can anybody help me to configure kafka on ec2 instance, so that I can run producer from local machine. I am tried many combinations but didn't work.

Answer

Lundahl picture Lundahl · Nov 28, 2014

In the Kafka FAQ (updated for new properties) you can read:

When a broker starts up, it registers its ip/port in ZK. You need to make sure the registered ip is consistent with what's listed in bootstrap.servers in the producer config. By default, the registered ip is given by InetAddress.getLocalHost.getHostAddress(). Typically, this should return the real ip of the host. However, sometimes (e.g., in EC2), the returned ip is an internal one and can't be connected to from outside. The solution is to explicitly set the host ip and port to be registered in ZK by setting the advertised.listeners property in server.properties.

Related questions

How to install docker on Amazon Linux2

I wanna create docker image for Amazon ECR. but yum can't find it in my Amazon Linux2. [root@*** ~]# yum install -y docker Loaded plugins: amazon-id, rhui-lb, search-disabled-repos No package docker available. Error: Nothing to do Next, I tried to use …

Read More
Getting console output from a Docker container

I build an image with Python installed and a Python application too. My Python application is a Hello, World! application, just printing "Hello, World!" on the screen. Dockerfile: FROM python:2-onbuild CMD ["python", "./helloworld.py"] In the console I execute: …

Read More
Difference between Docker and AMI

In the context of AWS: AMI is used to package software and can be deployed on EC2. Docker can also be used to package software and can also be deployed to EC2. What's the difference between both and how do …

Read More