IBM MQ versus Apache Kafka
I am designing a new architecture big data where my client has as IBM MQ broker. We use to work with Kafka in big data architectures, because I suppose it is faster, reliable and stronger than the alternatives, and because storm and spark streaming work easier with Kafka.
My question is simple, that true? Or could we develop reliable solutions using IBM MQ as broker in our architecture with the same result as with Kafka?.
Are there true reasons for using Kafka instead of IBM MQ?
Answer
I hear this question every week... While IBM MQ or JMS in general is used for traditional messaging, Apache Kafka is used as streaming platform (messaging + distributed storage + processing of data). Both are built for different use cases.
You can use Kafka for "traditional messaging", but not use MQ for Kafka-specific scenarios.
The article Apache Kafka vs. Enterprise Service Bus (ESB)—Friends, Enemies, or Frenemies discusses why Kafka is not competitive but complementary to integration and messaging solutions (including IBM MQ) and how to integrate both.