How to parse several log nginx files using goaccess

logging nginx goaccess
somenxavier picture somenxavier · Aug 30, 2016 · Viewed 13.6k times · Source

I want to parse all my logs of nginx (you can see here):

ls /var/log/nginx/
access.log    access.log.21.gz  error.log.1      error.log.22.gz
access.log.1      access.log.22.gz  error.log.10.gz  error.log.23.gz
access.log.10.gz  access.log.23.gz  error.log.11.gz  error.log.24.gz
access.log.11.gz  access.log.24.gz  error.log.12.gz  error.log.2.gz
access.log.12.gz  access.log.2.gz   error.log.13.gz  error.log.3.gz
access.log.13.gz  access.log.3.gz   error.log.14.gz  error.log.4.gz
access.log.14.gz  access.log.4.gz   error.log.15.gz  error.log.5.gz
access.log.15.gz  access.log.5.gz   error.log.16.gz  error.log.6.gz
access.log.16.gz  access.log.6.gz   error.log.17.gz  error.log.7.gz
access.log.17.gz  access.log.7.gz   error.log.18.gz  error.log.8.gz
access.log.18.gz  access.log.8.gz   error.log.19.gz  error.log.9.gz
access.log.19.gz  access.log.9.gz   error.log.20.gz
access.log.20.gz  error.log     error.log.21.gz

but I don't know how to do that. First of all, it seems like goaccess can't parse .gz files.

What's the best way of parse all the information contained in these logs?

Answer

Sijia Din picture Sijia Din · Aug 31, 2016

Quoting the man page and assuming you have a Combined Log Format:

If we would like to process all access.log.*.gz we can do one of the following:

# zcat -f access.log* | goaccess --log-format=COMBINED

OR

# zcat access.log.*.gz | goaccess --log-format=COMBINED

On Mac OS X, use gunzip -c instead of zcat.

Related questions

nginx log analysis with goaccess

I want to parse and analyze the nginx logs with goaccess and take a report from the analyzed logs. But, when I run the zcat -f access.log.*.gz | goaccess -a -c command, it gives me the following error : GoAccess …

Read More
Logging POST data from $request_body

I have my config setup to handle a bunch of GET requests which render pixels that work fine to handle analytics and parse query strings for logging. With an additional third party data stream, I need to handle a POST …

Read More
logging proxy activity in nginx

I am using rewrite and proxy_pass and I am having some intermittent issues with the site I am proxying said rewrites too. Is there a means to log the proxy activity so I that I can debug this problem?

Read More