Spring Kafka Consumer Retry

spring-boot apache-kafka kafka-consumer-api spring-kafka
Punter Vicky picture Punter Vicky · Oct 2, 2017 · Viewed 13.9k times · Source

I am using Spring Kafka consumer which fetches messages from a topic and persist them into a db. If a failure condition is met , say for instance the db is unavailable , does kafka consumer library provide mechanism to retry ? If it does , is there a way to set different retry intervals such as 1st retry should be done after 5 mins , 2nd after 30 mins , 3rd after 1 hr etc.

Answer

Artem Bilan picture Artem Bilan · Oct 2, 2017

Spring Kafka is shipped with the RetryingMessageListenerAdapter and RetryingAcknowledgingMessageListenerAdapter. If you use @KafkaListener, you can supply AbstractKafkaListenerContainerFactory with the RetryTemplate. And the last one can be injected with any custom RetryPolicy and BackOffPolicy from the Spring Retry project:

https://docs.spring.io/spring-kafka/reference/html/#retrying-deliveries

Also bear in mind that since version 2.0, there is transaction support in Spring Kafka, based on such one in the Apache Kafka 0.11.x.x:

https://docs.spring.io/spring-kafka/reference/html/#transactions

Related questions

org.apache.kafka.common.KafkaException: Failed to construct kafka consumer

I am manually starting Zookeeper, then Kafka server and finally the Kafka-Rest server with their respective properties file. Next, I am deploying my Spring Boot application on tomcat In the Tomcat log trace, I am getting the Error org.springframework.…

Read More
Simple embedded Kafka test example with spring boot

Edit FYI: working gitHub example I was searching the internet and couldn't find a working and simple example of an embedded Kafka test. My setup is: Spring boot Multiple @KafkaListener with different topics in one class Embedded Kafka for test …

Read More
Kafka Streams with Spring Boot

I want to work with Kafka Streams real time processing in my spring boot project. So I need Kafka Streams configuration or I want to use KStreams or KTable, but I could not find example on the internet. I did …

Read More