I have setup Celery in a Django project on which I am working. I would like to separate the logging for celery tasks vs the remainder of celery logs (celerycam, celerybeat, etc).
Based on the Celery documentation (http://docs.celeryproject.org/en/latest/userguide/tasks.html#logging) it seems like I should just be able to define a Django logger for 'celery.task' which should do this. However, when I do this, nothing shows up in the logs. Everything does show up if I create a generic 'celery' logger, implying that this may be something to do with the name of the logger.
What am I missing here? Is there a way to make this work, or must all celery logs go together?
For what it's worth, I have set CELERYD_HIJACK_ROOT_LOGGER = False.
My logging setup in settings.py:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse'
}
},
'formatters': {
'standard': {
'format': '%(asctime)s %(levelname)s [%(name)s: %(lineno)s] -- %(message)s',
'datefmt': '%m-%d-%Y %H:%M:%S'
},
},
'handlers': {
'logfile': {
'level': 'INFO',
'filters': None,
'class': 'logging.handlers.RotatingFileHandler',
'filename': '/vagrant/logs/logfile.log',
'maxBytes': 1024*1024*5,
'backupCount': 3,
'formatter': 'standard'
},
'debug_logfile': {
'level': 'DEBUG',
'filters': None,
'class': 'logging.handlers.RotatingFileHandler',
'filename': '/vagrant/logs/debug_logfile.log',
'maxBytes': 1024*1024*5,
'backupCount': 5,
'formatter': 'standard'
},
'default_logger': {
'level': 'WARNING',
'filters': None,
'class': 'logging.handlers.RotatingFileHandler',
'filename': '/vagrant/logs/default.log',
'maxBytes': 1024*1024*5,
'backupCount': 2,
'formatter': 'standard'
},
'celery_logger': {
'level': 'DEBUG',
'filters': None,
'class': 'logging.handlers.RotatingFileHandler',
'filename': '/vagrant/logs/celery.log',
'maxBytes': 1024*1024*5,
'backupCount': 2,
'formatter': 'standard'
},
'celery_task_logger': {
'level': 'DEBUG',
'filters': None,
'class': 'logging.handlers.RotatingFileHandler',
'filename': '/vagrant/logs/celery_tasks.log',
'maxBytes': 1024*1024*5,
'backupCount': 2,
'formatter': 'standard'
},
},
'loggers': {
'': {
'handlers': ['default_logger'],
'level': 'WARNING',
'propagate': True,
},
'django': {
'handlers': ['logfile'],
'level': 'INFO',
'propagate': True,
},
'feedmanager': {
'handlers': ['logfile', 'debug_logfile'],
'level': 'DEBUG',
'propagate': True,
},
'recipemanager': {
'handlers': ['logfile', 'debug_logfile'],
'level': 'DEBUG',
'propagate': True,
},
'menumanager': {
'handlers': ['logfile', 'debug_logfile'],
'level': 'DEBUG',
'propagate': True,
},
'celery.task': {
'handlers': ['celery_task_logger'],
'level': 'DEBUG',
'propagate': True,
},
'celery': {
'handlers': ['celery_logger'],
'level': 'DEBUG',
'propagate': True,
},
}
}
i'm guessing in your tasks your doing this
from celery.utils.log import get_task_logger
logger = get_task_logger(__name__)
if your doing that then your logger name will be the module name, so if your task is in a tasks.py file in an app called MyApp your logger will be named 'MyApp.tasks' and you'd have to create the logger 'MyApp.tasks' in your settings.
You can just put a different string in place of __name__ for all of the tasks to log to same logger if you have them all over place. ie: 'celery.task'
oh and make sure your worker loglevel is to what you want it to be