Is there a GCP equivalent to AWS SQS?

user1384205 picture user1384205 · Sep 14, 2020 · Viewed 9.7k times · Source

Im curious to understand the implementation of GCP's PubSub. Although Pubsub seems to point to follow a Publish-Subscribe design pattern, it seems more close to AWS's SQS (queue) than AWS SNS (that use publish-subscribe model). Why is think this is, GCP's pubSub

  1. Allows upto 10,000 subscriptions per project.
  2. Allows filtering on subscriptions
  3. It even allows ordering (beta) - which should involve a FIFA queue somewhere.
  4. It exposes synchronous api for request/response pattern.

It makes me wonder if subscriptions in pub/sub are merely queues of SQS. I would like your opinions on this comparison. The confusion is due to lack of implementation details on PubSub and the obvious name indicating a certain design pattern.

Regards,

Answer

Kamal Aboul-Hosn picture Kamal Aboul-Hosn · Sep 14, 2020

The division for messaging in GCP is along slightly different lines than what you may see in AWS. GCP breaks down messaging into three categories:

  • Torrents: Messaging pipelines that are designed to handle large amounts of throughput on pipes that are persistent. In other words, one creates a new pipeline rarely and sends messages over it for long periods of time. The scaling pattern for torrents is a relatively small number of pipelines transmitting a lot of data. For this category, Cloud Pub/Sub is the right product.
  • Trickles: Messaging pipelines that are largely ephemeral or require broadcast to a very large number of end-user devices. These pipelines have a low throughput but the number of pipelines can be extremely large. Firebase Cloud Messaging is the product that fits into this category.
  • Queues: Messaging pipelines where one has more control over the end-to-end message delivery. These pipelines are not really high throughput nor is the number of pipelines large, but more advanced properties are supported, e.g., the ability to delay or cancel the delivery of a message. Cloud Tasks fits in this category, though Cloud Pub/Sub is also adopting features that make it more and more viable for this use case.

So Cloud Pub/Sub is the publish/subscribe aspects of SQS+SNS, where SNS is used as a means to distribute messages to different SQS queues. It also serves as the big-data ingestion mechanism a la Kinesis. Firebase Cloud Messaging covers the portions of SNS designed to reach end user devices. Cloud Tasks (and Cloud Pub/Sub, more and more) provide functionality of a single queue in SQS.