Masking sensitive data before getting logged - log4j

Question

For logging purposes, instead of using toString() method, Jackson's writeValueAsString(object) method has been used in the project I have been working for.

LOGGER.info(mapper.writeValueAsString(object));

Now, I got the requirement to mask the sensitive information like passwords and credit card numbers in logs. If toString() is being used, I could have removed those sensitive data from the toString() method. But in my case, I could not find such simple yet correct way of getting my problem solved. I am not in a situation where I can change the entire thing to use toString() too.

I read that by using %replace method, I can replace the data which I don't need to be logged in using a predefined pattern. But all the sensitive data that need to be masked wont follow a single pattern.

I tried by intercepting the log event, look for the particular information and mask them(Using a class which implements LogEventFactory). Even though it is a working solution, I don't think that it is a good solution since looking for the data in big strings every time is gonna cost.

Is there any way that I haven't come across yet to get my problem resolved? Is the approach with %replace is the way to go? If so, how?

Answer 1

I think there are three ways to solve your problem which you all covered. Most efficient way is to prevent log sensitive data to logger, which you need a bunch of work to do.

And the second way is to modify Appender. By extend log4j Appenders, you modify the LogEvent.

And last way is to modify the PatternLayout. Here is an example.

public class CardNumberFilteringLayout extends PatternLayout {
    private static final String MASK = "$1++++++++++++";
    private static final Pattern PATTERN = Pattern.compile("([0-9]{4})([0-9]{9,15})");

    @Override
    public String format(LoggingEvent event) {
        if (event.getMessage() instanceof String) {
            String message = event.getRenderedMessage();
            Matcher matcher = PATTERN.matcher(message);

            if (matcher.find()) {
                String maskedMessage = matcher.replaceAll(MASK);
                @SuppressWarnings({ "ThrowableResultOfMethodCallIgnored" })
                Throwable throwable = event.getThrowableInformation() != null ? 
                        event.getThrowableInformation().getThrowable() : null;
                LoggingEvent maskedEvent = new LoggingEvent(event.fqnOfCategoryClass,
                        Logger.getLogger(event.getLoggerName()), event.timeStamp, 
                        event.getLevel(), maskedMessage, throwable);

                return super.format(maskedEvent);
            }
        }
        return super.format(event);
    }
}