Consumer Connection error with django and celery+rabbitmq?

python django rabbitmq celery django-celery
Paul Wicks picture Paul Wicks · May 16, 2012 · Viewed 13.3k times · Source

I'm trying to set up celeryd with django and rabbit-mq. So far, I've done the following:

  • Installed celery from pip
  • Installed rabbitmq via the debs available from their repository
  • Added a user and vhost to rabbitmq via rabbitmqctl, as well as permissions for that user
  • Started the rabbitmq-server
  • Installed django-celery via pip
  • Set up django-celery, including its tables
  • Configured the various things in settings.py (BROKER_HOST, BROKER_PORT, BROKER_USER, BROKER_PASSWORD, BROKER_VHOST, as well as importing djecelery, calling the setup function and adding it to the INSTALLED APPS). I've double checked and all of these values are correct (at least, user, password and vhost are correct).

So now, when I run python manage.py celeryd -l info I get connection errors (see below). Anyone have any idea why?

$ python manage.py celeryd -l info
/usr/local/lib/python2.7/dist-packages/djcelery/loaders.py:108: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!
  warnings.warn("Using settings.DEBUG leads to a memory leak, never "
[2012-05-15 18:38:04,486: WARNING/MainProcess]  

 -------------- celery@ubuntu v2.5.3
---- **** -----
--- * ***  * -- [Configuration]
-- * - **** ---   . broker:      amqp://celeryuser@localhost:5672/celeryhost
- ** ----------   . loader:      djcelery.loaders.DjangoLoader
- ** ----------   . logfile:     [stderr]@INFO
- ** ----------   . concurrency: 1
- ** ----------   . events:      OFF
- *** --- * ---   . beat:        OFF
-- ******* ----
--- ***** ----- [Queues]
 --------------   . celery:      exchange:celery (direct) binding:celery


[Tasks]


[2012-05-15 18:38:04,562: INFO/PoolWorker-1] child process calling self.run()
[2012-05-15 18:38:04,565: WARNING/MainProcess] celery@ubuntu has started.
[2012-05-15 18:38:07,572: ERROR/MainProcess] Consumer: Connection Error: [Errno 104] Connection reset by peer. Trying again in 2 seconds...
^C[2012-05-15 18:38:08,434: WARNING/MainProcess] celeryd: Hitting Ctrl+C again will terminate all running tasks!
[2012-05-15 18:38:08,435: WARNING/MainProcess] celeryd: Warm shutdown (MainProcess)
[2012-05-15 18:38:09,372: INFO/PoolWorker-1] process shutting down
[2012-05-15 18:38:09,373: INFO/PoolWorker-1] process exiting with exitcode 0
[2012-05-15 18:38:09,376: INFO/MainProcess] process shutting down

Answer

Amit picture Amit · Sep 27, 2012

Your problem is in the BROKER_URL.

With an additional VHOST, the right config would be:

BROKER_URL='amqp://celeryuser@localhost:5672//'
BROKER_VHOST='/celeryhost'

Related questions

Celery: When should you choose Redis as a message broker over RabbitMQ?

My rough understanding is that Redis is better if you need the in-memory key-value store feature, however I am not sure how that has anything to do with distributing tasks? Does that mean we should use Redis as a message …

Read More
Why do we need to use rabbitmq

Why do we need RabbitMQ when we have a more powerful network framework in Python called Twisted. I am trying to understand the reason why someone would want to use RabbitMQ. Could you please provide a scenario or an example …

Read More
'pip' is not recognized as an internal or external command

I'm running into a weird error when trying to install Django on my computer. This is the sequence that I typed into my command line: C:\Python34> python get-pip.py Requirement already up-to-date: pip in c:\python34\lib\site-packages …

Read More