I'm trying to get Kafka to work on docker-compose for the first time. The application runs fine without docker. But on docker, I get the error as described below. Any reason why Kafka would throw this error?
The error:
email-service_1 | 2018-12-01 14:32:02.448 WARN 1 --- [ntainer#0-0-C-1] o.a.k.c.NetworkClient : [Consumer clientId=consumer-2, groupId=kafka] 1 partitions have leader brokers without a matching listener, including [email-token-0]
My docker-compose config:
version: '3.3'
services:
zookeeper:
image: wurstmeister/zookeeper
ports:
- "2181:2181"
kafka:
image: wurstmeister/kafka
command: [start-kafka.sh]
environment:
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_HOST_NAME: 192.168.23.134
KAFKA_CREATE_TOPICS: "email-token:1:1"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
ports:
- "9092:9092"
depends_on:
- zookeeper
email-service:
build: ./email-service
environment:
SPRING_KAFKA_BOOTSTRAPSERVERS: kafka:9092
ports:
- "8081:8081"
depends_on:
- kafka
As stated in the comments to your question the problem seems to be with the advertised name for the Kafka broker. According to your docker-compose you should be using 192.168.23.134
but your email-service is using kafka:9092
. You can try with this docker-compose. I replaced the wurstmeister services with the latest Zookeeper and Kafka provided by confluentinc
and added your email-service.
---
version: '2'
services:
zookeeper:
image: confluentinc/cp-zookeeper:latest
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
kafka:
image: confluentinc/cp-kafka:latest
depends_on:
- zookeeper
ports:
- 9092:9092
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
email-service:
build: ./email-service
environment:
SPRING_KAFKA_BOOTSTRAPSERVERS: kafka:29092
ports:
- "8081:8081"
depends_on:
- kafka
advertised.listeners: Listeners to publish to ZooKeeper for clients to use, if different than the listeners config property. In IaaS environments, this may need to be different from the interface to which the broker binds. If this is not set, the value for listeners will be used. Unlike listeners it is not valid to advertise the 0.0.0.0 meta-address.
Please note that KAFKA_ADVERTISED_HOST_NAME
has been deprecated and it's recommended to use KAFKA_ADVERTISED_LISTENERS
instead. For more information about KAFKA_ADVERTISED_LISTENERS
check here.