Best practices for draining or clearing a Google Cloud pubsub topic

Question

For pubsub topics with number of messages in the range of ~100k, what is the best practice for draining/dropping/clearing/deleting all messages using gcloud-java SDK?

Possible solutions:

• Deleting and recreating the subscribers and then the publishers

• High concurrency pull+ack (easy to hit the quota this way)

• Something else

My hope is that this process can be fast (not more than ~60 seconds, say), robust, and uses supported SDK methods with minimal other code.

Answer 1

Update with description of snapshot and seek feature: One can use seek on a Pub/Sub subscription to ack older messages by seeking to a timestamp corresponding to now. The best way is via the gcloud command line tool. The command to acknowledge messages published up to a particular timestamp would be:

gcloud pubsub subscriptions seek <subscription path> --time=yyyy-mm-ddThh:mm:ss

To delete all messages up to now:

gcloud pubsub subscriptions seek <subscription path> --time=$(date +%Y-%m-%dT%H:%M:%S) 

Previous answer prior to the addition of snapshot and seek: Currently, Google Cloud Pub/Sub has no way clear older messages, though it is something we are looking to add. Deleting and recreating the subscription would be the most efficient way to clear it, both in terms of time and cost. You wouldn't have to do anything with your publishers; any messages published from the point after recreation on will be sent to subscribers on the recreated subscription.