What is the best practice for formatting logs?

Andy Smith picture Andy Smith · Nov 19, 2009 · Viewed 44.7k times · Source

I'm writing a piece of honeypot software that will have extensive logging of interactions with it, I plan to log in plaintext .log files.

I have two questions, from someone who isn't too familiar with how servers log.

  1. Firstly how shall I break up my log files, I'm assuming after running this for a month I don't want one big .log file, do I do this by day, month, year? Is there some standard for it?

  2. The format of each line, do I have one standard delimiter that is whatever, *, -, +, anything? Is there a standard anywhere (my googling hasn't brought up much)?

Answer

Pierre-Antoine LaFayette picture Pierre-Antoine LaFayette · Nov 19, 2009

I like this format for log files:

$ python simple_logging_module.py
2005-03-19 15:10:26,618 - simple_example - DEBUG - debug message
2005-03-19 15:10:26,620 - simple_example - INFO - info message
2005-03-19 15:10:26,695 - simple_example - WARNING - warn message
2005-03-19 15:10:26,697 - simple_example - ERROR - error message
2005-03-19 15:10:26,773 - simple_example - CRITICAL - critical message

This is from python's logging module. I usually have a file per day, one folder for each month, one folder for each year. You'll get huge log files that you can't edit properly otherwise.

logs/
  2009/
    January/
     01012009.log
     02012009.log
     ...
    February/
     ...
  2008/
   ...