supervisord always returns exit status 127 at WebFaction

Brandon picture Brandon · Feb 20, 2015 · Viewed 11.1k times · Source

I keep getting the following errors from supervisord at webFaction when tailing the log:

INFO exited: my_app (exit status 127; not expected)
INFO gave up: my_app entered FATAL state, too many start retries too quickly

Here's my supervisord.conf:

[unix_http_server]
file=/home/btaylordesign/tmp/supervisord.sock

[rpcinterface:supervisor]
supervisor.rpcinterface_factory=supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///home/btaylordesign/tmp/supervisord.sock

[supervisord]
logfile=/home/btaylordesign/tmp/supervisord.log
logfile_maxbytes=50MB
logfile_backups=5
loglevel=info
nodaemon=false
pidfile=/home/btaylordesign/tmp/supervisord.pid supervisord.pid

[program:my_app]
directory=/home/btaylordesign/webapps/my_app/my_app
command=celery worker -A my_app --concurrency=3 --loglevel=debug

I'm starting supervisord from the same directory as supervisord.conf:

$ supervisord -c ./supervisord.conf

but I can't seem to find the right combination of settings. I need to be able to do three things:

  1. Start my celery workers in the background and keep them running.
  2. Stop the celery workers when I deploy code.
  3. Restart the celery workers when the deploy is complete.

But, I can't do any of that until I resolve the error. What am I doing wrong?

Answer

Steven Kryskalla picture Steven Kryskalla · Feb 20, 2015

Exit code 127 means "command not found":

http://www.tldp.org/LDP/abs/html/exitcodes.html

Try passing the full path to the celery command:

command=/home/something/bin/celery worker -A my_app --concurrency=3 --loglevel=debug

Also, try setting the redirect_stderr and stdout_logfile options in your [program:x] section to capture the error message and make debugging easier.