How do I set up a daemon with python-daemon?

Question

I'm new to daemons so apologies if this is a newbie question.

In several other answers (for example, this question) people suggested the python-daemon package was the way to go because it fully implements the PEP 3143 standard.

Unfortunately, python-daemon is a bit light on documentation (or more likely I am a bit light on knowledge / experience... ;) ), and I think I am probably missing something really basic. Here's what I'm doing:

I have the following:

import daemon

logfile = open('daemon.log', 'w')

context = daemon.DaemonContext(stdout = logfile, stderr = logfile)

context.open()

with context:
    do_something_1()
    do_something_2()

Question: How do I set up a daemon with python-daemon, how can I start it and stop it?

---

Side notes:

I'm basically taking a wild guess about how / whether the .open() method should be used here -- docs were not real clear on this point. Same thing seems to happen whether I include it or not.

So, now what do I do? When I try running this file, eg:

python startConsumerDaemons.py

it appears to run do_something_1(), but not the second. And, it appears to leave the program attached to the terminal window. IE, stdout isn't redirected, and when I close the terminal window the process is killed. So, I'm pretty sure I'm doing something wrong here... what should I be doing differently?

And, lastly, once I get the daemon running, how do I stop / restart it (for example if I make changes to the underlying code)?

Answer 1

Here is what I have, that works for me. It also has a sysv init script. Repo is at GitHub, and I also have a brief blog post with links to other possible solutions I found.

There can only be one daemon process running: that is managed by the PID lock file, like most other Linux daemons. To stop it, do

kill `cat /var/run/eg_daemon.pid`

To see if it is running:

ps -elf | grep `cat /var/run/eg_daemon.pid`

Using the pidfile submodule, the PID file is managed automatically. When the daemon is stopped, the pidfile is cleared up. Please see the linked GitHub repo for the init script.

Here's the Python daemon code:

#!/usr/bin/env python3.5
import sys
import os
import time
import argparse
import logging
import daemon
from daemon import pidfile

debug_p = False

def do_something(logf):
    ### This does the "work" of the daemon

    logger = logging.getLogger('eg_daemon')
    logger.setLevel(logging.INFO)

    fh = logging.FileHandler(logf)
    fh.setLevel(logging.INFO)

    formatstr = '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
    formatter = logging.Formatter(formatstr)

    fh.setFormatter(formatter)

    logger.addHandler(fh)

    while True:
        logger.debug("this is a DEBUG message")
        logger.info("this is an INFO message")
        logger.error("this is an ERROR message")
        time.sleep(5)


def start_daemon(pidf, logf):
    ### This launches the daemon in its context

    ### XXX pidfile is a context
    with daemon.DaemonContext(
        working_directory='/var/lib/eg_daemon',
        umask=0o002,
        pidfile=pidfile.TimeoutPIDLockFile(pidf),
        ) as context:
        do_something(logf)


if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Example daemon in Python")
    parser.add_argument('-p', '--pid-file', default='/var/run/eg_daemon.pid')
    parser.add_argument('-l', '--log-file', default='/var/log/eg_daemon.log')

    args = parser.parse_args()

    start_daemon(pidf=args.pid_file, logf=args.log_file)

For completeness' sake, here is the init script. Note that "kill" is really just a method for sending a POSIX signal -- see man page for signal(7) for an overview. The python-daemon context will catch the signal, terminate the process cleanly closing file descriptors, and delete the PID file automatically. So, it really is a clean termination.

You can write your code to catch SIGUSR1 or something similar, in order to do a reload of the daemon config. There is no advantage to writing Python stop the daemon.

#!/bin/bash
#
# eg_daemon      Startup script for eg_daemon
#
# chkconfig: - 87 12
# description: eg_daemon is a dummy Python-based daemon
# config: /etc/eg_daemon/eg_daemon.conf
# config: /etc/sysconfig/eg_daemon
# pidfile: /var/run/eg_daemon.pid
#
### BEGIN INIT INFO
# Provides: eg_daemon
# Required-Start: $local_fs
# Required-Stop: $local_fs
# Short-Description: start and stop eg_daemon server
# Description: eg_daemon is a dummy Python-based daemon
### END INIT INFO

# Source function library.
. /etc/rc.d/init.d/functions

if [ -f /etc/sysconfig/eg_daemon ]; then
        . /etc/sysconfig/eg_daemon
fi

eg_daemon=/var/lib/eg_daemon/eg_daemon.py
prog=eg_daemon
pidfile=${PIDFILE-/var/run/eg_daemon.pid}
logfile=${LOGFILE-/var/log/eg_daemon.log}
RETVAL=0

OPTIONS=""

start() {
        echo -n $"Starting $prog: "

        if [[ -f ${pidfile} ]] ; then
            pid=$( cat $pidfile  )
            isrunning=$( ps -elf | grep  $pid | grep $prog | grep -v grep )

            if [[ -n ${isrunning} ]] ; then
                echo $"$prog already running"
                return 0
            fi
        fi
        $eg_daemon -p $pidfile -l $logfile $OPTIONS
        RETVAL=$?
        [ $RETVAL = 0 ] && success || failure
        echo
        return $RETVAL
}

stop() {
    if [[ -f ${pidfile} ]] ; then
        pid=$( cat $pidfile )
        isrunning=$( ps -elf | grep $pid | grep $prog | grep -v grep | awk '{print $4}' )

        if [[ ${isrunning} -eq ${pid} ]] ; then
            echo -n $"Stopping $prog: "
            kill $pid
        else
            echo -n $"Stopping $prog: "
            success
        fi
        RETVAL=$?
    fi
    echo
    return $RETVAL
}

reload() {
    echo -n $"Reloading $prog: "
    echo
}

# See how we were called.
case "$1" in
  start)
    start
    ;;
  stop)
    stop
    ;;
  status)
    status -p $pidfile $eg_daemon
    RETVAL=$?
    ;;
  restart)
    stop
    start
    ;;
  force-reload|reload)
    reload
    ;;
  *)
    echo $"Usage: $prog {start|stop|restart|force-reload|reload|status}"
    RETVAL=2
esac

exit $RETVAL