How to write an Ubuntu Upstart job for Celery (django-celery) in a virtualenv

Question

I really enjoy using upstart. I currently have upstart jobs to run different gunicorn instances in a number of virtualenvs. However, the 2-3 examples I found for Celery upstart scripts on the interwebs don't work for me.

So, with the following variables, how would I write an Upstart job to run django-celery in a virtualenv.

Path to Django Project:

/srv/projects/django_project

Path to this project's virtualenv:

/srv/environments/django_project

Path to celery settings is the Django project settings file (django-celery):

/srv/projects/django_project/settings.py

Path to the log file for this Celery instance:

/srv/logs/celery.log

For this virtual env, the user:

iamtheuser

and the group:

www-data

I want to run the Celery Daemon with celerybeat, so, the command I want to pass to the django-admin.py (or manage.py) is:

python manage.py celeryd -B

It'll be even better if the script starts after the gunicorn job starts, and stops when the gunicorn job stops. Let's say the file for that is:

/etc/init/gunicorn.conf

Answer 1

You may need to add some more configuration, but this is an upstart script I wrote for starting django-celery as a particular user in a virtualenv:

start on started mysql
stop on stopping mysql

exec su -s /bin/sh -c 'exec "$0" "$@"' user -- /home/user/project/venv/bin/python /home/user/project/django_project/manage.py celeryd

respawn

It works great for me.

I know that it looks ugly, but it appears to be the current 'proper' technique for running upstart jobs as unprivileged users, based on this superuser answer.

I thought that I would have had to do more to get it to work inside of the virtualenv, but calling the python binary inside the virtualenv is all it takes.