Top "Celery" questions

Celery is a distributed task queue framework for Python, used for asynchronous and parallel execution.

Celery difference between concurrency, workers and autoscaling

In my /etc/defaults/celeryd config file, I've set: CELERYD_NODES="agent1 agent2 agent3 agent4 agent5 agent6 agent7 agent8" CELERYD_…

python concurrency celery
How can I run a celery periodic task from the shell manually?

I'm using celery and django-celery. I have defined a periodic task that I'd like to test. Is it possible to …

python django celery django-celery celery-task
Why do we need message brokers like RabbitMQ over a database like PostgreSQL?

I am new to message brokers like RabbitMQ which we can use to create tasks / message queues for a scheduling …

postgresql redis rabbitmq message-queue celery
Setting Time Limit on specific task with celery

I have a task in Celery that could potentially run for 10,000 seconds while operating normally. However all the rest of …

celery celeryd
Celery parallel distributed task with multiprocessing

I have a CPU intensive Celery task. I would like to use all the processing power (cores) across lots of …

python django multithreading multiprocessing celery
Rabbitmq or Gearman - choosing a jobs queue

At work, we need to build a jobs server for things like sending emails, building PDFs, crunching some data, etc. …

message-queue rabbitmq celery gearman
How to debug "could not receive data from client: Connection reset by peer"

I'm running a django-celery application on Ubuntu-12.04. When I run a celery task from my web interface, I get the …

django database-connection celery django-celery django-postgresql
Dealing with duplicate primary keys on insert in SQLAlchemy (declarative style)

My application is using a scoped session and the declarative style of SQLALchemy. It's a web-app and a lot of …

python mysql sqlalchemy celery
How to use the @shared_task decorator for class based tasks

As seen on the documentation the @shared_task decorator lets you create tasks without having any concrete app instance. The …

python celery
Celery task that runs more tasks

I am using celerybeat to kick off a primary task that kicks of a number of secondary tasks. I have …

python django task celery