Scaling in Rabbitmq

rabbitmq rabbitmq-exchange rabbitmqctl
Lalit Singh Rana picture Lalit Singh Rana · May 25, 2015 · Viewed 8.9k times · Source

While using Clusters in Rabbit MQ, I was planning to use the Competing Subscriber pattern.

Producer : 1
Exchange : 1 direct
Queue : 1
Consumers : n (multiple) listening to the same queue.

Now if I have a cluster containing 3 nodes, there would be but one queue from where the consumer gets the message (irrespective of number of nodes in cluster). If I have to scale up with the above constraints, do you think I should be having multiple clusters.

Please advise.

Second related question, how clustering would help in scalablity?

Answer

old_sound picture old_sound · Jun 8, 2015

Clustering is not there to improve Scalability (although it might help in some situations) but to improve High Availability.

For Scalability see the Sharding Plugin which can be used together with clustering https://github.com/rabbitmq/rabbitmq-sharding

Note that you need to find out if said plugin meets your requirements, the README tells of the limitations of the plugin

Related questions

RabbitMQ Declare Exchange from Terminal - Access refused: /api/exchanges/

I'm using rabbitmq to handle messages between my databases for an enterprise application I work on. As part of the process, I'm trying to help automate the setup of the servers (e.g. script it). In this process, I've tried …

Read More
why do we need routing key in rabbitmq

I am wondering why do we need routing key to route message from exchange to queue. Can't we use the simply queue name to route the message. Also, in case of publishing to multiple queues we can use multiple queue …

Read More
Rabbitmq File Descriptor Limit

Rabbitmq documentation says that we need to do some configuration before we use it on production. One of the configuration is about maximum open file number (which is an OS parameter). Rabbitmq server we use is running on Ubuntu 16.04 and …

Read More