Ubuntu upstart will hang on start/stop/etc

Question

I've got a several services on Ubuntu which will start using 'upstart'. They are working as requested, but when I use 'stop/start/restart {myservice}' it will hang (but WILL do as requested).

I understand it has something to do with forking.

My services are python scripts, which will create new threads on startup. One script will create 1 new thread (and will continue running on the main as well), the second one will create 2 new threads and will continue running on the main as well and the third one will create no new threads.

All of them hang on the command.

All use the same code in /etc/init as follows:

description "my service"
version "1.0"
author "my name, 2013"

expect fork

start on runlevel [2345]
stop on runlevel [!2345]
respawn


chdir <to script dir>

exec /usr/bin/python ./scriptname/

what do you think might be the problem? Does 'fork' has anything to do with creating new threads?

Answer 1

I think you are confusing creating threads in python with linux forking. Python will manage its own threads while it is running (assuming you are using regular constructs), from upstarts view it is only monitoring the running python process that you have started - python should be set up to manage its own children.

Possible solution 1

Try removing the expect clause altogether, then rebooting (you must reboot because upstart may be tracking a nonexistent PID).

I saw this start/stop hanging when upstart was tracking a nonexistent PID as a result of an incorrect expect, which was solved by removing the expect and rebooting. You can diagnose this by executing status myservice and upstart will report being stopped as well as a PID. (For other users whom rebooting is not an option, you could alternatively use this script)

Possible solution 2

Setting the user using exec su root -c "/usr/bin/python ./scriptname/" (or some other user) would solve the problem if python/your program is trying to access environment variables that only exist for a certian user (upstart has a very minimal set of environment variables by default)

Note

You should try see the output to help you debug. In real life you might pipe to logger however modifying your script to something like /usr/bin/python ./scriptname/ >> /home/myuser/output.log 2>&1 then viewing the contents would help you for now.