How do I force rebuild log's data in filebeat 5

elasticsearch logstash filebeat
周宏成 picture 周宏成 · Jan 17, 2017 · Viewed 13.9k times · Source

I have filebeats 5.x ship logs to logstash.

How do I reset the “file pointer” in filebeat

This is a similar problem to

I cleaned all elasticsearch's data, delete the /var/lib/filebeat/registry. but filebeat is only shipping the new line.

change the registry_file is invalid, the file's offset saved to new file (delete file is the same problem) filebeat.registry_file: registry

Answer

Yasir picture Yasir · Sep 12, 2017
  1. Stop filbeat service.
  2. Rename the register file - usually found in /var/lib/filebeat/registry
  3. Start filbeat service.

sudo service filbeat stop

mv /var/lib/filebeat/registry /var/lib/filebeat/registry.old

sudo service filbeat start

Related questions

Elasticsearch: No handler for type [keyword] declared on field [hostname]

I get above Mapper Parsing Error on Elasticsearch when indexing log from filebeat. I tried both Filebeat -> Elasticserach and Filebeat -> Logstash -> Elasticsearch approach. I have followed their own documentations, I installed filebeat template as …

Read More
Generating filebeat custom fields

I have an elasticsearch cluster (ELK) and some nodes sending logs to the logstash using filebeat. All the servers in my environment are CentOS 6.5. The filebeat.yml file in each server is enforced by a Puppet module (both my production …

Read More
How to define seperated indexes for different logs in Filebeat/ELK?

I am wondering how to create separated indexes for different logs fetched into logstash (which were later passed onto elasticsearch), so that in kibana, I can define two indexes for them and discover them. In my case, I have a …

Read More