DDS vs AMQP vs ZeroMQ

amqp zeromq data-distribution-service
Kisalay picture Kisalay · Jul 8, 2010 · Viewed 9.3k times · Source

I wanted a feedback on whether my evaluations and concerns are correct.

I have been reseaching the three, Data Distribution Service, AMQP and ZeroMQ for sometime now for building a data transport layer in a datacenter. All the three look promising, yet i encountered some blocking issues in few.

To give a context, my requirements are:

  1. Scale upto 500+ physical nodes, 1000+ publishers and subscribers.
  2. Support durable delivery of messages to take care of failing subscribers.
  3. The aggregated throughput should be north of 1 Million messages / sec

Problems with AMQP:

  1. The Broker architecture seems to be a bottleneck and central point of failure in the whole deployment setup. I can complicate my deployment by putting a federation and cluster to improve performance and availability of pending messages, but they still do not seem to be failproof.
  2. The performance for durable queues seems to be very less. My sample application could only clock 6-7K messages / core / queue/ application.

Problems with ZeroMQ:

  1. The Documentation seems to be a little wanting in depth.
  2. The behavior of the system for pending messages seems to invite trouble in PUB/SUB model of communication. Please refer to : How zeromq handles slow consumers with PUB/SUB mode

OpenSplice DDS: I didnt find anything lacking in the DDS protocol except for adoption in the industry. Will like to know a first hand review on this product in terms of stability, performance or limitations.

Answer

acor picture acor · Aug 21, 2010

I am surprised about your concerns on OpenSplice DDS adoption. OpenSplice DDS is today deployed on over several mission and business critical systems such as Naval Combat Management Systems, Military Vehicles, Air Traffic Control and Management, Metro, to High Frequency Auto-Trading. Just to give you another bit of info that should give you reassure you w.r.t. the adoption of the technology, the OMG DDS standard (the standard implemented by OpenSplice DDS) has been recommended by EUROCAE to exchange flight data plans across centers pan-european.

Let me know if you have any further questions either on the adoption or the technology.

-AC

Related questions

Recovering from zmq.error.ZMQError: Address already in use

I hit Ctrl-C while running a PAIR pattern (non-blocking client servers) connection with ZMQ. Later when I tried running the REQ-REP (blocking client single server connection) pattern, I keep getting the Address already in use error. I have tried running …

Read More
Why use AMQP/ZeroMQ/RabbitMQ

as opposed to writing your own library. We're working on a project here that will be a self-dividing server pool, if one section grows too heavy, the manager would divide it and put it on another machine as a separate …

Read More
Differences between AMQP and ZeroMQ

Recently started looking into these AMQP (RabbitMQ, ActiveMQ) and ZeroMQ technologies, being interested in distributed systems/computation. Been Googling and StackOverflow'ing around, couldn't find a definite comparison between the two. The farthest I got is that the two aren't really …

Read More