Choosing between .NET Service Bus Queues vs Azure Queue Service
Just a quick question regarding an Azure application. If I have a number of Web and Worker roles that need to communicate, documentation says to use the Azure Queue Service.
However, I've just read that the new .NET Service Bus now also offers queues. These look to be more powerful as they appear to offer a much more detailed API. Whilst the .NSB looks more interesting it has a couple of issues that make me wary of using it in distributed application. (for example, Queue Expiration... if I cannot guarantee that a queue will be renewed on time I may lose it all!).
Has anyone had any experience using either of these two technologies and could give any advice on when to choose one over the other.
I suspect that whilst the service bus looks more powerful, as my use case is really just enabling Web/Worker roles to communicate between each other, that the Azure Queue Service is what I'm after. But I'm just really looking for confirmation of that before progamming myself in to a corner :-)
Thanks in advance.
UPDATE
Have read up about the two systems over the break. It defo looks like .NET service bus is more specifically designed for integrating systems rather than providing a general purpose reliable messaging system. Azure Queues are distributed and so reliable and scalable in a way that .NSB queues are not and so more suitable for code hosted within Azure itself.
Thanks for the responses.
Answer
Storage Queues vs Service Bus
Here is a breakdown of some of the different considerations I had in thinking through this question.
Availability
Since the storage outage last November Azure promised they would never deploy code to all the regions at once again - built it into the system to make it impossible. https://azure.microsoft.com/en-us/blog/final-root-cause-analysis-and-improvement-areas-nov-18-azure-storage-service-interruption/
Here's what msdn says about availability:
If you are already using Azure Storage Blobs or Tables and you start using queues, you are guaranteed 99.9% availability. If you use Blobs or Tables with Service Bus queues, you will have lower availability.
Azure Queues are designed to support the decoupling of application components to increase scalability and tolerance for failures.
Development
Personally, I am comfortable with the storage APIs and already have need for its blob storage in other areas of most apps. Storage queues use the very same sdk as storage blobs. Azure Queues provide a uniform and consistent programming model across queues, tables, and BLOBs
Cost
The Receive and Delete mode supported by Service Bus provides the ability to reduce the messaging operation count (and associated cost) in exchange for lowered delivery assurance.
It seems like there is some tooling around cost control for service bus that you could leverage if you had to start keeping a budget to run your app - I did an attempt at breaking down the potential storage queue costs below :). It comes out to less than a hundred bucks a month at more than 40,000 queues an hour for GRS. Grouped in with the rest of our storage costs I don't see benefit to focusing on cutting costs here. (Bandwidth is the same for both and cancels itself out when comparing)
Storage pricing
You get unlimited free queues and operations - you pay for space
- assume 30K message size as an average
- assume 1000K in an MB not 1024
- assume you will not hit the graduated pricing above 1TB
30K / 1 message * 1 TB / 1000000000K * $.095 / 1 GB * 1000GB / 1 TB = $0.00000285 / message for the first TB of use
1 message / ~30K * 1000000000K / 1 TB = 33333333 messages in a TB
33333333 messages * $0.00000285 / message = ~$95 dollars for the first TB
spread out over a month we can do like 40,000 messages an hour with that 1st TB
Service Bus pricing
- 10 bucks a month base price
- pay per operation (any api call is an op) - add a queue / receive a queue / monitor the queue / etc.
- you get 12.5 million free ops / month
- pay per million ops after that
Hard to estimate usage here but 100 million operations costs 80 bucks a month
Batch Receive
Storage can batch up to a maximum of 32 messages by specifying message count when retrieving messages while Service Bus allows a queue client to batch multiple messages into a single send operation.
So storage is batch receive while service bus is batch send.
Monitoring
Azure Queues enable you to obtain a detailed log of all of the transactions executed against the queue, as well as aggregated metrics. This kind of support does not come out of box with Service Bus - but could probably find a prebuilt solution somewhere.
Forwarding
Service Bus has an auto-forwarding feature that storage queues is missing.
auto-forwarding enables thousands of queues to auto-forward their messages to a single queue, from which the receiving application consumes the message. You can use this mechanism to achieve security, control flow, and isolate storage between each message publisher.
Duplicates
The duplication detection functionality supported by Service Bus queues automatically removes duplicate messages sent to a queue or topic, based on the value of the MessageID property.
Storage queue messages can duplicate without warning.
Metadata
Service Bus gives you 2 parts of a message header+body. This is a very useful feature for a globally deployed infrastructure. Letting you decorate your messages with things like region name and instance id. Queue messages are simple strings. On the other hand Azure storage queues provide support for arbitrary attributes that can be applied to the queue description, in the form of name/value pairs. So you can decorate the message in Service Bus and decorate the queue with storage queues
Delivery Guarantee
Service Bus offers At-Most-Once and At-Least-Once while Queues only offer At-Least-Once delivery. This could limit our ability to use queues if concurrent subscribers are ever a problem.
Performance
Azure storage queues offer a 10ms latency (within a datacenter) while Service Bus 20-25ms latency. Service bus does offer long-polling which would be even better than 10ms if you have a need for it.
Security
Storage queues use the primary/secondary shared key thing while service bus provides RBAC via Active Directory with Sender/Receiver/Admin roles.