Setting Time Limit on specific task with celery

celery celeryd
jimstandard picture jimstandard · Jul 26, 2012 · Viewed 39.6k times · Source

I have a task in Celery that could potentially run for 10,000 seconds while operating normally. However all the rest of my tasks should be done in less than one second. How can I set a time limit for the intentionally long running task without changing the time limit on the short running tasks?

Answer

mher picture mher · Jul 26, 2012

You can set task time limits (hard and/or soft) either while defining a task or while calling.

from celery.exceptions import SoftTimeLimitExceeded

@celery.task(time_limit=20)
def mytask():
    try:
        return do_work()
    except SoftTimeLimitExceeded:
        cleanup_in_a_hurry()

or 

mytask.apply_async(args=[], kwargs={}, time_limit=30, soft_time_limit=10)

Related questions

How can I recover unacknowledged AMQP messages from other channels than my connection's own?

It seems the longer I keep my rabbitmq server running, the more trouble I have with unacknowledged messages. I would love to requeue them. In fact there seems to be an amqp command to do this, but it only applies …

Read More
How do I restart celery workers gracefully?

While issuing a new build to update code in workers how do I restart celery workers gracefully? Edit: What I intend to do is to something like this. Worker is running, probably uploading a 100 MB file to S3 A new …

Read More
How to start a Celery worker from a script/module __main__?

I've define a Celery app in a module, and now I want to start the worker from the same module in its __main__, i.e. by running the module with python -m instead of celery from the command line. I …

Read More