Interprocess communication in Python

Question

What is a good way to communicate between two separate Python runtimes? Thing's I've tried:

• reading/writing on named pipes e.g. os.mkfifo (feels hacky)
• dbus services (worked on desktop, but too heavyweight for headless)
• sockets (seems too low-level, surely there's a higher level module to use?)

My basic requirement is to be able to run python listen.py and have that process just doing it's thing there, like a daemon, able to receive messages from python client.py --bar. The client call should just send a message to the existing process and terminate, with return code 0 for success or nonzero for failure (i.e. some two-way communication will be required)

Answer 1

The multiprocessing library provides listeners and clients that wrap sockets and allow you to pass arbitrary python objects.

Your server could listen to receive python objects:

from multiprocessing.connection import Listener

address = ('localhost', 6000)     # family is deduced to be 'AF_INET'
listener = Listener(address, authkey='secret password')
conn = listener.accept()
print 'connection accepted from', listener.last_accepted
while True:
    msg = conn.recv()
    # do something with msg
    if msg == 'close':
        conn.close()
        break
listener.close()

Your client could send commands as objects:

from multiprocessing.connection import Client

address = ('localhost', 6000)
conn = Client(address, authkey='secret password')
conn.send('close')
# can also send arbitrary objects:
# conn.send(['a', 2.5, None, int, sum])
conn.close()