No Brokers Available error when trying to connect to Kafka

Question

I have a very strange problem when trying to connect locally to Kafka 0.10.0.0 using Python client on CentOS.

My connection options are pretty simple and default:

kafka_consumer = kafka.KafkaConsumer(
        bootstrap_servers=['localhost:9092'],
        client_id="python-test-consumer"
    )

When I manually set listeners option in Kafka's server.properties file like:

listeners=PLAINTEXT://localhost:9092

I get the kafka.errors.NoBrokersAvailable despite the fact that I can still easily connect to Kafka broker server with curl or other linux stuff.

No advertised.listeners or other deprecated advertised options help to solve the problem. Thus, the only state of configuration which is working is one without listeners. What is certainly unacceptable, because we need to setup local cluster somehow.

It seems that solution for this silly problem is simple and is wondering around, but we couldn't figure it ourselves.

Answer 1

This may sound silly, but the exact same problem happened to me because of this:

I upgraded to Kafka 0.10.0.0 via brew (Mac package manager). Brew then suggests to run like this one-liner:

$ zookeeper-server-start /usr/local/etc/kafka/zookeeper.properties; kafka-server-start /usr/local/etc/kafka/server.properties

Instead of how I executed before:

$ zkServer start
$ kafka-server-start /usr/local/etc/kafka/server.properties

The approach suggested kept throwing those "No Brokers Available" errors in the client. Then I just split the command in two lines:

$ zookeeper-server-start /usr/local/etc/kafka/zookeeper.properties
$ kafka-server-start /usr/local/etc/kafka/server.properties

And everything works like before!

Sorry if this doesn't work for you, but I figured it was worth mentioning.