How to combine characters to create custom pattern in GROK

Jerald Sabu M picture Jerald Sabu M · Jul 27, 2016 · Viewed 12.2k times · Source

I'm new to logstash and grok and have a question regarding a pattern.

Jul 26 09:46:37

The above content contains %{MONTH} %{MONTHDAY} %{TIME} and white spaces.

I need to know how to combine all these and create a pattern %{sample_timestamp}

Thanks!

Answer

Will Barnwell picture Will Barnwell · Jul 27, 2016

Quotes from the Grok Custom Patterns Docs (RTFM):

First, you can use the Oniguruma syntax for named capture which will let you match a piece of text and save it as a field:

(?<field_name>the pattern here)

...

Alternately, you can create a custom patterns file.

  • Create a directory called patterns with a file in it called extra (the file name doesn’t matter, but name it meaningfully for yourself)
  • In that file, write the pattern you need as the pattern name, a space, then the regexp for that pattern.

So you could create a pattern file that contained the line:

CUST_DATE %{MONTH} %{MONTHDAY} %{TIME}

Then use the patterns_dir setting in this plugin to tell logstash where your custom patterns directory is.

 filter {
   grok {
     patterns_dir => ["./patterns"]
     match => { "message" => "%{CUST_DATE:datestamp}" }
   }
 }

Would result in the field:

 datestamp => "Jul 26 09:46:37"