Ok it is better to have a fail2ban compatible log file than for Amplify :-)
FYI, Nginx Amplify can do metric alerts PHP-FPM - How to configure PHP-FPM Status Monitoring with Nixstats & Nginx Amplify
@eva2000 So as I understand centminmod works out of the box with nginx Amplify by detecting the main_ext that comes installed by default as explained in Beta Branch - switch nginx access_log to custom main_ext format by default 123.09beta01. So I enabled PHP-FPM status PHP-FPM - CentminMod.com LEMP Nginx web stack for CentOS. I set up everything, also including the MySQL monitoring, looks like it works fine except for two nginx metrics: NGINX HTTP Version NGINX Request time I get "Metrics unavailable. Please check the docs" for both. This is the relevant configuration of nginx.conf, completely vanilla and untouched: Code: log_format main_ext '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for" ' '"$host" sn="$server_name" ' 'rt=$request_time ' 'ua="$upstream_addr" us="$upstream_status" ' 'ut="$upstream_response_time" ul="$upstream_response_leng$ 'cs=$upstream_cache_status' ; access_log off; error_log logs/error.log warn; I don't think that access_log being off and the error_log without main_ext is okay? Or is it? How does centminmod auto-detects the main_ext?
log_format main_ext is an optional format so you change and edit your nginx vhosts to use main_ext similar to how example at Module ngx_http_log_module changes access_log to use custom log_format = compression
Oh! So I have to change the access_log from combined to main_ext for every vhost conf and also virtual.conf to make it work with nginx Amplify? error_log should be fine at error level as I see. I thought this was "out of the box" too, haha.
Yup. I did have it as default temporarily but it may break fail2ban and it's scanning of logged regular expressions Security - fail2ban for Centmin Mod + CSF Firewall / Cloudflare API
Technically you can support both by having duplicate access_log defined one for default combine and one for main_ext Code (Text): access_log /home/nginx/domains/domain.com/log/access.log combined buffer=256k flush=5m; access_log /home/nginx/domains/domain.com/log/access-ext.log main_ext buffer=256k flush=5m; which will mean double the disk space consumed by nginx access logs and more resources used for logging on high traffic sites.
Which metrics will help me to know if i am close to a limit for: 1)Nginx worker_connections 50000; 2)Phpfpm pm.max_children = 80 ?
for PHP-FPM see PHP-FPM - How to configure PHP-FPM Status Monitoring with Nixstats & Nginx Amplify for nginx = current connections I believe
My amplify service periodically reports that NGINX is down, but its obviously running because sites are running. When I use the restart command everything seems to work fine again. Anyone else experiencing that?.... I've only been running CentMinMod for a few days....very happy so far.....but still a total noob. its not a big deal i guess...but i do enjoy having the amplify service for monitoring. unless there is a better one I am unaware of. Cheers
Yeah only one of my servers at OVH runs into that for unexplained reasons sometimes but every other server seems fine. Haven't been able to figure out why but the OVH server has no live traffic it's just for testing so maybe related.
let me know if u ever figure it out ..... and i will do the same..... is there a way to automate the restart somehow? this is where some of my linux noobness is showing.
You mean the amplify-agent service Amplify | Library ? or nginx ? You can write your own custom script via cronjob to check if service is running and restart it or you can use something like monit which you need to install and configure yourself Easy, proactive monitoring of processes, programs, files, directories, filesystems and hosts | Monit Though for me amplify-agent service is always running even when amplify doesn't report stats so checking to see if amplify-agent service is running might not work.
I meant the amplify agent ..... i will look into creating cron job to check and restart. thanks for the hint.
This is with monit monitoring configured on centos 7.4 for some centmin mod services Code (Text): monit summary Monit 5.25.1 uptime: 1m ┌─────────────────────────────────┬────────────────────────────┬───────────────┐ │ Service Name │ Status │ Type │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ hostname │ OK │ System │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ sshd │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ postfix │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ php-fpm │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ nginx │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ memcached │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ mysqld │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ lfd │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ crond │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ amplify-agent │ OK │ Process │ └─────────────────────────────────┴────────────────────────────┴───────────────┘ Code (Text): monit status Monit 5.25.1 uptime: 1m Process 'sshd' status OK monitoring status Monitored monitoring mode active on reboot start pid 5589 parent pid 1 uid 0 effective uid 0 gid 0 uptime 4d 11h 12m threads 1 children 2 cpu - cpu total - memory 0.2% [3.2 MB] memory total 0.6% [11.9 MB] security attribute (null) disk read 0 B/s [293.4 MB total] disk write 0 B/s [416.5 MB total] port response time 9.524 ms to 127.0.0.1:22 type TCP/IP protocol SSH data collected Tue, 17 Apr 2018 20:51:52 Process 'postfix' status OK monitoring status Monitored monitoring mode active on reboot start pid 3033 parent pid 1 uid 0 effective uid 0 gid 0 uptime 4d 10h 53m threads 1 children 7 cpu - cpu total - memory 0.1% [1.1 MB] memory total 1.5% [30.1 MB] security attribute (null) disk read 0 B/s [56.6 MB total] disk write 0 B/s [1.5 MB total] data collected Tue, 17 Apr 2018 20:51:52 Process 'php-fpm' status OK monitoring status Monitored monitoring mode active on reboot start pid 20712 parent pid 1 uid 0 effective uid 0 gid 0 uptime 4h 48m threads 1 children 1 cpu - cpu total - memory 0.5% [9.4 MB] memory total 0.9% [19.3 MB] security attribute (null) disk read 0 B/s [6.1 MB total] disk write 0 B/s [12 kB total] port response time 0.313 ms to localhost:9000 type TCP/IP protocol DEFAULT data collected Tue, 17 Apr 2018 20:51:52 Process 'nginx' status OK monitoring status Monitored monitoring mode active on reboot start pid 20700 parent pid 1 uid 0 effective uid 0 gid 0 uptime 4h 48m threads 1 children 2 cpu - cpu total - memory 2.2% [45.2 MB] memory total 8.7% [178.6 MB] security attribute (null) disk write 0 B/s [4 kB total] data collected Tue, 17 Apr 2018 20:51:52 Process 'memcached' status OK monitoring status Monitored monitoring mode active on reboot start pid 17999 parent pid 1 uid 1001 effective uid 1001 gid 1002 uptime 4d 10h 51m threads 10 children 0 cpu - cpu total - memory 0.1% [1.2 MB] memory total 0.1% [1.2 MB] security attribute (null) disk read 0 B/s [352 kB total] port response time 0.217 ms to localhost:11211 type TCP/IP protocol DEFAULT data collected Tue, 17 Apr 2018 20:51:52 Process 'mysqld' status OK monitoring status Monitored monitoring mode active on reboot start pid 5136 parent pid 1 uid 996 effective uid 996 gid 994 uptime 4d 10h 49m threads 24 children 0 cpu - cpu total - memory 3.2% [65.6 MB] memory total 3.2% [65.6 MB] security attribute (null) disk read 0 B/s [23.2 MB total] disk write 0 B/s [64.8 MB total] data collected Tue, 17 Apr 2018 20:51:52 Process 'lfd' status OK monitoring status Monitored monitoring mode active on reboot start pid 14513 parent pid 1 uid 0 effective uid 0 gid 0 uptime 20h 51m threads 1 children 0 cpu - cpu total - memory 1.2% [24.3 MB] memory total 1.2% [24.3 MB] security attribute (null) disk read 0 B/s [88 kB total] disk write 0 B/s [12 kB total] data collected Tue, 17 Apr 2018 20:51:52 Process 'crond' status OK monitoring status Monitored monitoring mode active on reboot start pid 5652 parent pid 1 uid 0 effective uid 0 gid 0 uptime 4d 11h 12m threads 1 children 0 cpu - cpu total - memory 0.1% [1.3 MB] memory total 0.1% [1.3 MB] security attribute (null) disk read 0 B/s [504.0 MB total] disk write 0 B/s [120.3 MB total] data collected Tue, 17 Apr 2018 20:51:52 Process 'amplify-agent' status OK monitoring status Monitored monitoring mode active on reboot start pid 22522 parent pid 1 uid 1000 effective uid 1000 gid 1001 uptime 1h 10m threads 3 children 0 cpu - cpu total - memory 1.8% [37.1 MB] memory total 1.8% [37.1 MB] security attribute (null) disk write 0 B/s [1.1 MB total] data collected Tue, 17 Apr 2018 20:51:52 System 'hostname' status OK monitoring status Monitored monitoring mode active on reboot start load average [0.04] [0.03] [0.00] cpu 0.0%us 0.0%sy 0.0%wa memory usage 522.1 MB [25.5%] swap usage 4.6 MB [0.2%] uptime 4d 11h 30m boot time Fri, 13 Apr 2018 09:21:43 data collected Tue, 17 Apr 2018 20:51:52 Test monit auto restarts of monitored services. Manually kill amplify-agent service Code (Text): kill -9 $(pidof amplify-agent) check monit summary there's a 60s delay and 120 second interval check for downed services before restarts configured so Code (Text): monit summary Monit 5.25.1 uptime: 9m ┌─────────────────────────────────┬────────────────────────────┬───────────────┐ │ Service Name │ Status │ Type │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ hostname.domainname.com │ OK │ System │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ sshd │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ postfix │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ php-fpm │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ nginx │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ memcached │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ mysqld │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ lfd │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ crond │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ amplify-agent │ Does not exist │ Process │ └─────────────────────────────────┴────────────────────────────┴───────────────┘ monit detects amplify-agent is down so restarts it automatically after 60 seconds delay from default 120 sec interval checks so ~3 mins until restart happens Code (Text): tail -100 /var/log/messages | grep monit | sed -e "s|$(hostname -s)|hostname|g" | grep -v Skipping Apr 17 21:20:22 hostname monit[2096]: 'amplify-agent' process is not running Apr 17 21:20:22 hostname monit: 'amplify-agent' process is not running Apr 17 21:20:22 hostname monit[2096]: 'amplify-agent' trying to restart Apr 17 21:20:22 hostname monit[2096]: 'amplify-agent' start: '/etc/init.d/amplify-agent start' Apr 17 21:20:22 hostname monit: 'amplify-agent' trying to restart Apr 17 21:20:22 hostname monit: 'amplify-agent' start: '/etc/init.d/amplify-agent start' Apr 17 21:22:25 hostname monit[2096]: 'amplify-agent' process is running with pid 3275 Apr 17 21:22:25 hostname monit: 'amplify-agent' process is running with pid 3275 recheck monit summary Code (Text): monit summary Monit 5.25.1 uptime: 21m ┌─────────────────────────────────┬────────────────────────────┬───────────────┐ │ Service Name │ Status │ Type │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ hostname.domainname.com │ OK │ System │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ sshd │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ postfix │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ php-fpm │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ nginx │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ memcached │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ mysqld │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ lfd │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ crond │ OK │ Process │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ amplify-agent │ OK │ Process │ └─────────────────────────────────┴────────────────────────────┴───────────────┘
@eva2000 I've installed the "monit" service and after when i type the monit summary command; Code (Text): [15:05][root@server ~]# monit summary Monit 5.25.1 uptime: 1m ┌─────────────────────────────────┬────────────────────────────┬───────────────┐ │ Service Name │ Status │ Type │ ├─────────────────────────────────┼────────────────────────────┼───────────────┤ │ server │ OK │ System │ └─────────────────────────────────┴────────────────────────────┴───────────────┘ Why other process don't show ?
not enough to install monit, it needs to be configured specifically for each service you want to monitor