Read from a log file as it's being written using python

python file logging file-io monitoring
Anon picture Anon · Jul 20, 2010 · Viewed 63.7k times · Source

I'm trying to find a nice way to read a log file in real time using python. I'd like to process lines from a log file one at a time as it is written. Somehow I need to keep trying to read the file until it is created and then continue to process lines until I terminate the process. Is there an appropriate way to do this? Thanks.

Answer

Wayne Werner picture Wayne Werner · Jul 20, 2010

Take a look at this PDF starting at page 38, ~slide I-77 and you'll find all the info you need. Of course the rest of the slides are amazing, too, but those specifically deal with your issue:

import time
def follow(thefile):
    thefile.seek(0,2) # Go to the end of the file
    while True:
        line = thefile.readline()
        if not line:
            time.sleep(0.1) # Sleep briefly
            continue
        yield line

Related questions

logger configuration to log to file and print to stdout

I'm using Python's logging module to log some debug strings to a file which works pretty well. Now in addition, I'd like to use this module to also print the strings out to stdout. How do I do this? In …

Read More
Python using basicConfig method to log to console and file

I don't know why this code prints to the screen, but not to the file? File "example1.log" is created, but nothing is written there. #!/usr/bin/env python3 import logging logging.basicConfig(level=logging.DEBUG, format='%(asctime)s %(…

Read More
How do I check whether a file exists without exceptions?

How do I check if a file exists or not, without using the try statement?

Read More