Strange "pattern not match" error on fluentd

apache logging fluentd

nixmind · Jul 31, 2015 · Viewed 7.3k times · Source

Can someone tell if its normal that fluentd raise this error in td-agent.log file?

2015-07-31 13:15:19 +0000 [warn]: pattern not match: "- - - [31/Jul/2015:13:15:19 +0000] GET http://172.31.108.218/ HTTP/1.1 200 0 \"-\" \"ELB-HealthChecker/1.0\""

While this is a well formated apache2 log:

- - - [31/Jul/2015:13:15:19 +0000] GET http://172.31.108.218/ HTTP/1.1 200 0 \"-\" \"ELB-HealthChecker/1.0\"

And here is the source configuration:

<source>
  type tail
  format apache2
  path /var/log/varnish/varnishncsa.log
  pos_file /var/log/td-agent/tmp/access.log.pos
  tag "apache2.varnish-access"
</source>

I can't figure out what's wrong there above.

Answer

The problem is that these ELB-HealthChecker line log has an empty referer ip field. And then the log doesn't match apache2 log format for fluentd.

So the way to fix that is to filter logs with ELB-HealthChecker user-agent.

Strange "pattern not match" error on fluentd

Answer

Related questions