Finding certain messages in SQS

amazon-web-services amazon-sqs
iLemming picture iLemming · Sep 28, 2012 · Viewed 20k times · Source

I know SQS ain't build for that, but I'm curious is it possible to find messages in a queue that meet some criteria?

I can pull messages in a loop, search the message bodies for some pattern (without even deserializing them), and filter the messages I needed. But then it is possible to end up with an infinite loop - the first messages I read will be back to the queue by the time when I reach the end of the queue...

Extending visibility of the messages possible, but how do I know exactly how long it will take to scan the entire queue, and for how long should I extend the visibility? What if I have literally ten thousand messages in there?

Is there any workaround here? I need to scan the queue for some messages, and delete those...

Answer

Ryan Parman picture Ryan Parman · Oct 5, 2012

Short answer: no.

Queues are designed for things like tasks. A machine grabs a new task (i.e., message) from the queue, executes the task, then deletes the task.

If you're trying to search the messages to filter them, I can't help but wonder if you're using the wrong tool for the job…

Related questions

What is the difference between Amazon SNS and Amazon SQS?

I don't understand when I would use SNS versus SQS, and why are they always coupled together?

Read More
Why should I use Amazon Kinesis and not SNS-SQS?

I have a use case where there will be stream of data coming and I cannot consume it at the same pace and need a buffer. This can be solved using an SNS-SQS queue. I came to know the Kinesis …

Read More
Why do SqS messages sometimes remain in-flight on queue

I'm using Amazon SQS queues in a very simple way. Usually, messages are written and immediately visible and read. Occasionally, a message is written, and remains In-Flight(Not Visible) on the queue for several minutes. I can see it from …

Read More