Python: separate processes logging to same file?

python multithreading logging nfs
sjbx picture sjbx · Feb 26, 2013 · Viewed 14.8k times · Source

Does Python's logging library provide serialised logging for two (or more) separate python processes logging to the same file? It doesn't seem clear from the docs (which I have read).

If so, what about on completely different machines (where the shared log file would exist on an NFS export accessible by both).

Answer

Hayden Crocker picture Hayden Crocker · Feb 26, 2013

No it is not supported. From python logging cookbook:

Although logging is thread-safe, and logging to a single file from multiple threads in a single process is supported, logging to a single file from multiple processes is not supported, because there is no standard way to serialize access to a single file across multiple processes in Python.

Afterwards the cookbook suggests to use a single socket-server process that handles the logs and the other processes sending log messages to it. There is a working example of this apporach in the section Sending and Receiving logging events across a network.

Related questions

'thread._local' object has no attribute

I was trying to change the logging format by adding a context filter. My Format is like this FORMAT = "%(asctime)s %(VAL)s %(message)s" This is the class I use to set the VAL in the format. class TEST: …

Read More
How can I use threading in Python?

I am trying to understand threading in Python. I've looked at the documentation and examples, but quite frankly, many examples are overly sophisticated and I'm having trouble understanding them. How do you clearly show tasks being divided for multi-threading?

Read More
Is there any way to kill a Thread?

Is it possible to terminate a running thread without setting/checking any flags/semaphores/etc.?

Read More