But how I can I generated a single HTML file for all the log files?
manual linked in 1st post and 2nd post has examples of using zcat example Code: cd /home/nginx/domains/domainname.com/log zcat -f /home/nginx/domains/domainname.com/log/access.log-*.gz | cat /home/nginx/domains/domainname.com/log/access.log* | cat /home/nginx/domains/domainname.com/log/access.log-* | goaccess --exclude-ip=192.73.252.209 --exclude-ip=199.38.182.28 --exclude-ip=209.177.145.67 --exclude-ip=209.177.157.208 --exclude-ip=103.25.58.42 --exclude-ip=103.6.85.32 --exclude-ip=103.6.87.219 --exclude-ip=23.239.17.158 --exclude-ip=106.186.122.163 --exclude-ip=37.235.56.13 --exclude-ip=212.71.238.144 --exclude-ip=208.111.40.251 --exclude-ip=176.58.89.83 --exclude-ip=198.58.122.143 -p /etc/goaccess.conf -a > report.html
you can use Code: zcat -f /home/nginx/domains/*/log/access.log | goaccess -p /etc/goaccess.conf -a and goaccess.conf Code: time-format %T date-format %d/%b/%Y log-format %h %^[%d:%t %^] "%r" %s %b "%R" "%u"
Nice goaccess 1.x branch is out and has alot of nice revamped html reports GoAccess - Features live demo Server Statistics - 2016-06-30 21:23:37
I want to recommend to have port 7890 open by default on Centminmod installation for it as the latest version has many cool info
just whitelist the 7890 port in TCP_IN comma separated list in CSF firewall config file /etc/csf/csf.conf and restart csf firewall CSF Firewall - CentminMod.com LEMP Nginx web stack for CentOS or more restricted by ip address of your ISP add to /etc/csf/csf.allow and restart csf firewall Code (Text): tcp|in|d=7890|s=11.22.33.44 where 11.22.33.44 is your ISP ip address so only you can visit on port 7890
This log format recommend it by @Eduardo doesn't work anymore with the latest version Code: log-format %h %^[%d:%t %^] "%r" %s %b "%R" "%u" result: Code: Parsed 10 lines producing the following errors: Token '9.103.253.1 - - [22/Feb/2017:03:42:26 +0200]' doesn't match specifier '%x' Token '9.129.118.30 - - [22/Feb/2017:03:43:18 +0200]' doesn't match specifier '%x' Token '9.103.253.1 - - [22/Feb/2017:03:43:27 +0200]' doesn't match specifier '%x' Token '6.249.75.160 - - [22/Feb/2017:03:43:35 +0200]' doesn't match specifier '%x' Token '0.97.190.157 - - [22/Feb/2017:03:43:54 +0200]' doesn't match specifier '%x' Token '88.4.13.172 - - [22/Feb/2017:03:43:59 +0200]' doesn't match specifier '%x' Token '88.4.13.172 - - [22/Feb/2017:03:44:01 +0200]' doesn't match specifier '%x' Token '9.129.118.30 - - [22/Feb/2017:03:44:19 +0200]' doesn't match specifier '%x' Token '57.55.39.184 - - [22/Feb/2017:03:44:21 +0200]' doesn't match specifier '%x' Token '57.55.39.184 - - [22/Feb/2017:03:44:23 +0200]' doesn't match specifier '%x' Format Errors - Verify your log/date/time format From the config file i can read this: # NOTE: If the time/date is a timestamp in seconds or microseconds # %x must be used instead of %d & %t to represent the date & time.
yeah Centmin Mod default log format changed nginx custom log format parsing ? · Issue #669 · allinurl/goaccess · GitHub you sure you're using Code (Text): # NCSA Combined Log Format log-format %h %^[%d:%t %^] "%r" %s %b "%R" "%u" on my test local centos 7 it works Code (Text): goaccess -p /etc/goaccess.conf -f /home/nginx/domains/newdomain.com/log/access.log -a > report.html
ooh in goaccess FAQ that gives me Code (Text): ./nginx2goaccess.sh '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for" ' 'rt=$request_time ua="$upstream_addr" ' 'us="$upstream_status" ut="$upstream_response_time" ' 'ul="$upstream_response_length" ' 'cs=$upstream_cache_status' - Generated goaccess config: time-format %T date-format %d/%b/%Y log_format %h - %^ [%d:%t %^] "%r" so maybe Code (Text): log_format %h - %^ [%d:%t %^] "%r" %s %b "%R" "%u"
zcat gives error Code: Parsed 1 linesproducing the following errors: Token for '%h' specifier is NULL.
did you do the 1st post steps for configuraing goaccess ? looks like goaccess.conf symlink in that linked instruction has changed since to now Code (Text): rm -f /etc/goaccess.conf ln -s /usr/local/etc/goaccess/goaccess.conf /etc/goaccess.conf did you modifying Centmin Mod's default nginx log format by any chance ? the error is related to goaccess not finding a format it recognises AFAIK https://github.com/allinurl/goaccess/issues/1141 which is fixed by instructions at https://community.centminmod.com/th...rce-real-time-web-log-analyzer.286/#post-1305
when i checked that file, it was blank one (is this OK or should i reinstall goaccess). i did as stated and pasted a content to etc/goaccess.conf from this post https://community.centminmod.com/th...rce-real-time-web-log-analyzer.286/#post-4100 edited all required parameters. When i tried to add access.log file which is yet to be written in access.log i get error. For old gzip and Today's file of access.log.txt it wont throw error. Also the report html file wont update it after an hour or on next day too. What i should do to make it update (live with may be delay of 1 hour or 256k memory buffer). I have not changed any format of logs and using it as it is.
i updated instructions at https://community.centminmod.com/th...rce-real-time-web-log-analyzer.286/#post-1305 to updated goaccess.conf symlink so need to edit /etc/goaccess.conf with the changes so goaccess recognises centmin mod nginx's log format first need to setup your own script to run the html log and then put that script in cronjob to schedule running
I had setup cron in crontab with following line Code: 0 * * * * /usr/bin/zcat -f /home/nginx/domains/mydomain.com/log/access.log* | goaccess -p /etc/goaccess.conf -a > /home/nginx/domains/mydomain.com/public/mydomain_report.html If i run this command manually, it updates the report html but when it run as a cron, it makes the report html to go blank without any details. adding >/dev/null 2>&1 to end of cron command also makes it blank only. can you please do help in setting up cron for this.
I made cron script at /root/tools/cron-goaccess.sh when i run this script via cli, it works and status gets updated. When i put that in crontab, it makes the html page go blank. My crontab command 0 * * * * /root/tools/cron-goaccess.sh tried with and without >/dev/null 2>&1 result were always same, it make html go blank.