Generating filebeat custom fields

Shachar Ashkenazi picture Shachar Ashkenazi · Apr 8, 2016 · Viewed 21.8k times · Source

I have an elasticsearch cluster (ELK) and some nodes sending logs to the logstash using filebeat. All the servers in my environment are CentOS 6.5.

The filebeat.yml file in each server is enforced by a Puppet module (both my production and test servers got the same configuration).

I want to have a field in each document which tells if it came from a production/test server.

I wanted to generate a dynamic custom field in every document which indicates the environment (production/test) using filebeat.yml file.

In order to work this out i thought of running a command which returns the environment (it is possible to know the environment throught facter) and add it under an "environment" custom field in the filebeat.yml file but I couldn't find any way of doing so.

Is it possible to run a command through filebeat.yml? Is there any other way to achieve my goal?

Answer

Michael Cho picture Michael Cho · May 8, 2016

In your filebeat.yml:

filebeat:
  prospectors:
    -
      paths:
        - /path/to/my/folder
      input_type: log

      # Optional additional fields. These field can be freely picked
      # to add additional information to the crawled log files
      fields:
        mycustomvar: production