GROK Parsing with regex

user2359303 picture user2359303 · May 14, 2013 · Viewed 36.7k times · Source

I am using the following regexes:

INT (?:[+-]?(?:[0-9]+))
VALUE ([0-9]+)
SPACE \s*
DATA .*?
USERNAME [a-zA-Z0-9._-]+
YEAR (?>\d\d){1,2}
MONTHNUM (?:0?[1-9]|1[0-2])
MONTHDAY (?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])
HOUR (?:2[0123]|[01]?[0-9])
MINUTE (?:[0-5][0-9])
SECOND (?:(?:[0-5][0-9]|60))
ISO8601_TIMEZONE (?:Z|[+-]%{HOUR}(?::?%{MINUTE}))
TIMESTAMP %{YEAR:year}/%{MONTHNUM:monthnum}/%{MONTHDAY:monthday}-%{HOUR:hour}:%{MINUTE:minute}:%{SECOND:second}.%{VALUE:_second}
MESSAGE %{DATA}ERR_SYSTEM%{DATA}
PARSE_ERROR %{TIMESTAMP:ts}%{SPACE}%{USERNAME:type1}%{SPACE}%{USERNAME:slave}%{SPACE}%{USERNAME:type2}%{SPACE}[%{USERNAME:fibre1}/USERNAME:fibre2]%{SPACE}%{MESSAGE:message}

Now I have to parse this line:

2013/05/13-05:19:16.776 INFO abcd1 gamereporting
[0000000000000000/00000000000000000000] [GameReportingSlaveImpl:0x30bf7699a010].processReport() : Error processing report for id=18014398509852207, type=frostbite_m, error=ERR_SYSTEM

I get the following after the parsing:

type1: INFO
slave : abcd1
type2: gamereportin

Now type 2 always misses the 'g'. Why is it happening?

Could somebody provide the correct regex for the above line?

Answer

Adam picture Adam · May 20, 2013

Are you using the Grok Debugger?

And where is your Grok pattern? I don't see it in your post.

I would use something similar to the following to start off. As you gave no insight as to how you'd like to save the data, you'll have to add that as you figure it out.

%{DATESTAMP}%{SPACE}%{LOGLEVEL}%{SPACE}%{WORD}%{SPACE}%{WORD}%{SPACE}(?<some_id>\[\d+\/\d+\])

Use the Grok Debugger - it will save you a lot of time.