How to structure celery tasks

Question

I have 2 types of task: async tasks and schedule tasks. So, here is my dir structure:

proj
  |
  -- tasks
      |
      -- __init__.py
      |
      -- celeryapp.py     => celery instance defined in this file.
      |
      -- celeryconfig.py
      |
      -- async
      |    |
      |    -- __init__.py
      |    |
      |    -- task1.py    => from proj.tasks.celeryapp import celery
      |    |
      |    -- task2.py    => from proj.tasks.celeryapp import celery
      |
      -- schedule
           |
           -- __init__.py
           |
           -- task1.py    => from proj.tasks.celeryapp import celery
           |
           -- task2.py    => from proj.tasks.celeryapp import celery

But when I run celery worker like below, it does not work. It can not accept the task from celery beat scheduler.

 $ celery worker --app=tasks -Q my_queue,default_queue

So, is there any best practice on multiple task files organization?

Answer 1

Based on celery documentation you can import a structure of celery tasks like this:

For example if you have an (imagined) directory tree like this:

|
|-- foo
|    |-- __init__.py
|    |-- tasks.py
|
|-- bar
     |-- __init__.py
     |-- tasks.py

Then calling app.autodiscover_tasks(['foo', bar']) will result in the modules foo.tasks and bar.tasks being imported.