Managing Kafka Topic with spring

Question

We are planning to use Kafka for queueing in our application. I have some bit of experience in RabbitMQ and Spring.

With RabbitMQ and Spring, we used to manage queue creation while starting up the spring service.

With Kafka, I'm not sure what could be the best way to create the topics? Is there a way to manage the topics with Spring.

Or, should we write a separate script which helps in creating topics? Maintaining a separate script for creating topics seems a bit weird for me.

Any suggestions will be appreciated.

Answer 1

In spring it is possible to create topics during the start of the application using beans:

@Bean
public KafkaAdmin admin() {
    Map<String, Object> configs = new HashMap<>();
    configs.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG,
            StringUtils.arrayToCommaDelimitedString(kafkaEmbedded().getBrokerAddresses()));
    return new KafkaAdmin(configs);
}

@Bean
public NewTopic topic1() {
    return new NewTopic("foo", 10, (short) 2);
}

Alternatively you can write your own create topics by autowiring the AdminClient, so for instance reading the list from an input file or specify advanced properties such as partition numbers:

@Autowired
private KafkaAdmin admin;
//...your implementation

Also note that since Kafka 1.1.0 auto.create.topics.enable is enabled by default (see Broker configs).

For more information refer to the spring-kafka docs