Apache Kafka: Replay messages in a topic

apache-kafka spring-kafka
rm12345 picture rm12345 · Jan 28, 2018 · Viewed 12.9k times · Source

I'm considering using Apache Kafka as an event store for storing events within a microservice.

One thing that I read through various blogs is that Kafka can be considered to be a single source of truth, where Kafka log will store all the events for a given topic.

I was wondering if Kafka has the ability to replay messages since the beginning of time (in case there is a hard drive/network crash that occurs for example)?

(note that i see that there are some logs stored in the /tmp folder under a topic directory). Does anyone know of any command (if any) that can be invoked to replay the messages in the topic?

Answer

Gary Russell picture Gary Russell · Jan 28, 2018

Yes, you can seek to a specific offset, but

beginning of time

depends on the topic or broker configuration. IIRC, the default retention is 7 days.

Refer to the the Kafka documentation.

Related questions

Simple embedded Kafka test example with spring boot

Edit FYI: working gitHub example I was searching the internet and couldn't find a working and simple example of an embedded Kafka test. My setup is: Spring boot Multiple @KafkaListener with different topics in one class Embedded Kafka for test …

Read More
org.apache.kafka.common.KafkaException: Failed to construct kafka consumer

I am manually starting Zookeeper, then Kafka server and finally the Kafka-Rest server with their respective properties file. Next, I am deploying my Spring Boot application on tomcat In the Tomcat log trace, I am getting the Error org.springframework.…

Read More
Kafka producer TimeoutException: Expiring 1 record(s)

I am using Kafka with Spring-boot: Kafka Producer class: @Service public class MyKafkaProducer { @Autowired private KafkaTemplate<String, String> kafkaTemplate; private static Logger LOGGER = LoggerFactory.getLogger(NotificationDispatcherSender.class); // Send Message public void sendMessage(String topicName, String message) throws Exception { …

Read More