Need to extract and re-format with RegEx

Question

I'm using Splunk to parse some logs that have our "hub" and "comp" IDs embedded in them, down in the body of the message. I need to use a field extraction RegEx to pull them out in the form: HHHH-CCCC where the data appears like this:

Hub:[HHHH] Comp: [HHHH]

Here's an example record:

RecordID:[00UJ9ANUHO5551212] TrackingID:[1234ANUHO5551212] Hub:[0472] Comp:[N259] Some event occurred, the log is in here::[\server\share\0472\N258\blah\blah\blah\somefile.txt], No exceptions raised.

From that, I'd like to return:

0472-N259

I'm trying to learn (re-learn! I learned this stuff 30 years ago!) capturing groups, and came up with this:

(?<=Hub:\[)([A-Z0-9]{4})

From that I can get the 4 characters for the hub, but it won't let me do:

(?<=Hub:\[)([A-Z0-9]{4})  (?<=Comp:\[)([A-Z0-9]{4})

I'm kind of close, but am getting frustrated and it's time to go home, so I thought maybe SO could help me out overnight. 100 bounty for the best answer (please explain the solution). I promise to come back and award when this question is eligible. Answer doesn't have to be in splunk form (with <fieldname>) but that's helpful too.

It's helpful if the RegEx can be pasted into http://gskinner.com/RegExr/ so I can experiment further.

Answer 1

Theres two ways you can achieve what you're looking to do...

Using search

Extract the fields with rex and use eval to concatenate the values.

| rex field=_raw "Hub:\[(?<Hub>[^\]]*)\]\sComp:\[(?<Comp>[^\]]*)\]" | eval someNewField=Hub."-".Comp

The rex command allows you to run a regular expression against a field, _raw is a special field name that contains the entire event data. The regex itself captures any characters between [ and ] and extracts it to the field named within the <>.

This is the easiest way as you don't need to modify any configuration to do it, but the drawback is that you'll need to add this to your search string to get the values extracted and formatting the way you want.

Using search time extraction with prop.conf and transforms.conf

In transforms.conf, add a transform to extract the fields...

[hubCompExtract]
REGEX = Hub:\[(?<Hub>[^\]]*)\]\sComp:\[(?<Comp>[^\]]*)\]

In props.conf, execute the extract and concatenate the values using an eval...

[yourSourceTypeName]
REPORT-fieldExtract = hubCompExtract
EVAL-yourNewFieldName = Hub."-".Comp

No need to add anything to your search string, but it does require config file changes.

Regex example

gSkinner example (without the capture group names).