Filebeat vs Rsyslog for forwarding logs

elasticsearch logstash rsyslog filebeat
Luv33preet picture Luv33preet · Jun 6, 2017 · Viewed 7.2k times · Source

I am currently using filebeat to forward logs to logstash and then to elasticsearch.

Now, I am thinking about forwarding logs by rsyslog to logstash. The benefit of this would be that, I would not need to install and configure filebeat on every server, and also I can forward logs in JSON format which is easy to parse and filter.

I can use TCP/UDP to forward logs to logstash by rsyslog.

I want to know the more benefits and drawbacks of rsyslog over filebeat, in terms of performance, reliability and ease of use.

Answer

ugosan picture ugosan · Jun 6, 2017

When you couple Beats with Logstash you have something called "back pressure management" - Beats will stop flooding the Logstash server with messages in case something goes wrong on the network, for instance.

Another advantage of using Beats is that in Logstash you can have persisted queues, which prevents you from losing log messages in case your elasticsearch cluster goes down. So Logstash will persist messages on disk. Be careful because Logstash can't ensure you wont lose messages if you are using UDP, this link will be helpful.

Related questions

How to retrieve unique count of a field using Kibana + Elastic Search

Is it possible to query for a distinct/unique count of a field using Kibana? I am using elastic search as my backend to Kibana. If so, what is the syntax of the query? Heres a link to the Kibana …

Read More
Export to csv/excel from kibana

I am building a proof of concept using Elasticsearch Logstash and Kibana for one of my projects. I have the dashboard with the graphs working without any issue. One of the requirements for my project is the ability to download …

Read More
Redis Vs RabbitMQ as a data broker/messaging system in between Logstash and elasticsearch

We are defining an architecture to collect log information by Logstash shippers which are installed in various machines and index the data in one elasticsearch server centrally and use Kibana as the graphical layer. We need a reliable messaging system …

Read More