Logstash grok square brackets

logstash logstash-grok
DMack picture DMack · Jul 16, 2015 · Viewed 12.5k times · Source

I'm trying to get some sort of grok pattern to work with the following logging format :

*Sun 07:05:18.372 INFO  [main] [userID] perf - 0ms - select x from y

The problem I'm having is the field in square brackets that I've annotated here as userID. Sometimes this field is populated and at other times it is not. If I use the grok pattern below :

*%{DAY:Day} %{TIME:Time} %{LOGLEVEL:Loglevel}\s+(\[%{WORD:module}\]\s+)(\[%{HOSTNAME:id}\]\s+)%{GREEDYDATA:logline}*

It parses correctly as long as there is some data in the UserID field. If that field is empty ( example below ) it doesn't match. Any ideas gratefully received!

*Sun 07:05:18.372 INFO  [main] [] perf - 0ms - select x from y

Answer

Erez Rabih picture Erez Rabih · Jul 16, 2015

Did you try to escape the brackets with backward slash? As in \[%{WORD:module}\]

Related questions

How to handle non-matching Logstash grok filters

I am wondering what the best approach to take with my Logstash Grok filters. I have some filters that are for specific log entries, and won't apply to all entries. The ones that don't apply always generate _grokparsefailure tags. For …

Read More
Is there any existing grok{} pattern for date format YYYY/MM/DD HH:mm:ss?

I was checking the nginx error logs at our server and found that they start with date formatted as: 2015/08/30 05:55:20 i.e. YYYY/MM/DD HH:mm:ss. I was trying to find an existing grok date pattern which might help …

Read More
How to process multiline log entry with logstash filter?

Background: I have a custom generated log file that has the following pattern : [2014-03-02 17:34:20] - 127.0.0.1|ERROR| E:\xampp\htdocs\test.php|123|subject|The error message goes here ; array ( 'create' => array ( 'key1' => 'value1', 'key2' => …

Read More