Can structured logging be done with Pythons standard library?

Martin Thoma picture Martin Thoma · Jan 9, 2018 · Viewed 9.7k times · Source

I recently read about structured logging (here). The idea seems to be to log not by appending simple strings as a line to a logfile, but instead JSON objects. This makes it possible to analyze the logfile by automatic tools.

Can Pythons logging library do structured logging? If not, is there a "mainstream" solution for it (e.g. like numpy/scipy is the mainstream solution for scientific calculations)? I found structlog, but I'm not sure how widespread it is.

Answer

Anil_M picture Anil_M · Jan 9, 2018

Have you looked at python docs site section describing Implementing structured logging that explain how python built-in logger can be utilized for structured logging?

Below is a simple example as listed on above site .

import json
import logging

class StructuredMessage(object):
    def __init__(self, message, **kwargs):
        self.message = message
        self.kwargs = kwargs

    def __str__(self):
        return '%s >>> %s' % (self.message, json.dumps(self.kwargs))

m = StructuredMessage   # optional, to improve readability

logging.basicConfig(level=logging.INFO, format='%(message)s')
logging.info(m('message 1', foo='bar', bar='baz', num=123, fnum=123.456))

Which results in following log.

message 1 >>> {"fnum": 123.456, "num": 123, "bar": "baz", "foo": "bar"}

Hope this helps.