How do I restart celery workers gracefully?

Quintin Par picture Quintin Par · Mar 19, 2012 · Viewed 48.5k times · Source

While issuing a new build to update code in workers how do I restart celery workers gracefully?

Edit: What I intend to do is to something like this.

  • Worker is running, probably uploading a 100 MB file to S3
  • A new build comes
  • Worker code has changes
  • Build script fires signal to the Worker(s)
  • Starts new workers with the new code
  • Worker(s) who got the signal after finishing the existing job exit.

Answer

armonge picture armonge · Mar 19, 2012

The new recommended method of restarting a worker is documented in here http://docs.celeryproject.org/en/latest/userguide/workers.html#restarting-the-worker

$ celery multi start 1 -A proj -l info -c4 --pidfile=/var/run/celery/%n.pid
$ celery multi restart 1 --pidfile=/var/run/celery/%n.pid

According to http://ask.github.com/celery/userguide/workers.html#restarting-the-worker you can restart a worker sending a HUP signal

 ps auxww | grep celeryd | grep -v "grep" | awk '{print $2}' | xargs kill -HUP