Learn about Centmin Mod LEMP Stack today
Register Now

Wordpress 502 Bad Gateway Errors

Discussion in 'Blogs & CMS usage' started by savitzer, Nov 28, 2018.

  1. savitzer

    savitzer New Member

    10
    0
    1
    Nov 23, 2018
    Ratings:
    +0
    Local Time:
    12:26 PM
    I'm facing this errors constantly since past couple of hours on a live site. Everything was working fine before couple of hours, same traffic everything.

    I looked into php-fpm logs, nginx logs and mysql logs but didn't find anything there. What else should I be looking into to get more info?

    Also, any idea if this could be due to heavy traffic? I have a wordpress site that gets around 400-500 realtime. And I am on SSD Nodes 16gb ram & 4vcpu plan.

    Any help & ideas appreciated. Thanks
     
  2. savitzer

    savitzer New Member

    10
    0
    1
    Nov 23, 2018
    Ratings:
    +0
    Local Time:
    12:26 PM
    Update - It was something to do with mysql. I restarted mysql using
    Code:
    mysqlrestart
    and the site started functioning normally.

    I looked into mysql log using
    Code:
    journalctl -u mariadb --no-pager
    but didn't find any info there.

    Any ideas why this could have happened?
     
  3. eva2000

    eva2000 Administrator Staff Member

    37,227
    8,133
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,521
    Local Time:
    4:56 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    Your php issues most likely are due to PHP-FPM needing tuning. Read PHP-FPM - How to troubleshoot & optimize PHP-FPM server?

    Centmin Mod is provide as is, so short of scripted related bugs or issues, any further optimisation to the web stack components - nginx, php-fpm, mariadb mysql, csf firewall etc or web app specific configurations are left to the Centmin Mod user to deal with. So I do not provide any free support for such.

    Nginx 502 or 504 Bad Gateway Errors



    Bad gateway 502 /504 timeouts are usually related to Nginx timing out waiting on PHP-FPM to respond as PHP-FPM is overloaded or overwhelmed with requests, so may need to tune PHP-FPM values. It also maybe due to PHP-FPM in turn being queued and backed up waiting on MariaDB MySQL server to respond - so also need to look at MySQL.

    You'll need to tune your PHP-FPM settings with php-fpm main pool config file at /usr/local/etc/php-fpm.conf (overview of config files) and this is left up to end user to do but here's a thread for starters to enable PHP-FPM status page output outlined at
    Enabling PHP-FPM status also allows setting up 3rd party PHP-FPM status metric monitoring from services like:

    Checking PHP-FPM etc logs



    You'll also need to check into your PHP-FPM, Nginx and MariaDB logs which you can find as outlined at How to troubleshoot Centmin Mod initial install issues

    Server logs include Nginx, PHP-FPM, MariaDB MySQL error logs as well as others. You can find your Centmin Mod install/menu logs at FAQ 7 and server logs at FAQ 19 at Centmin Mod FAQ (most up to date info in FAQ so always read that first). Spoiler tag below has info too but may not be up to date.

    Some of Centmin Mod's installed software will have their own access and error logs which maybe useful for diagnosing errors or give info, notes, or warning notices.

    Note: There's no support provided by me for diagnosing such errors which may occur for various reasons including misconfiguration of installed php/mysql scripts or applications.

    In SSH2 telnet you can use tail command to view the last X number of lines in the file.

    For example for viewing last 10 lines in the file for:

    For Nginx access and error logs:
    Code:
      tail -10 /usr/local/nginx/logs/access.log
      tail -10 /usr/local/nginx/logs/error.log
    
    For specific domainname.com access and error log:
    Code:
      tail -10 /home/nginx/domains/domainname.com/log/access.log
      tail -10 /home/nginx/domains/domainname.com/log/error.log
    
    For other system error logs located at /var/log:

    list /var/log files in ascending time order so the most recently modified files are at the bottom
    Code:
      ls -lhrt /var/log
    
    Code:
    total 2.7M
    -rw------- 1 root  root    0 Aug 29 15:33 tallylog
    -rw------- 1 root  root    0 Aug 29 15:33 spooler
    drwx------ 3 root  root 4.0K Aug 29 15:35 samba
    drwxr-xr-x 2 root  root 4.0K Aug 29 15:35 mail
    -rw-r--r-- 1 root  500     0 Oct  8 18:13 dmesg.old
    -rw------- 1 root  500     0 Oct  8 18:13 boot.log
    -rw-r--r-- 1 root  500     0 Oct  8 18:14 dmesg
    drwx------ 2 root  root 4.0K Oct  8 18:14 httpd
    drwxr-xr-x 2 root  root 4.0K Oct  8 19:08 php-fpm
    -rw-rw---- 1 mysql root 2.3K Oct  9 12:38 mysqld.log
    -rw------- 1 root  root 9.2K Oct 26 10:48 yum.log
    -rw------- 1 root  utmp  94K Nov  7 22:59 btmp
    drwxr-xr-x 2 root  root 4.0K Nov  8 00:00 sa
    -rw------- 1 root  root 269K Nov  8 21:39 messages
    -rw------- 1 root  root 110K Nov  8 23:08 secure
    -rw-rw-r-- 1 root  utmp  43K Nov  8 23:08 wtmp
    -rw-r--r-- 1 root  root 144K Nov  8 23:08 lastlog
    -rw------- 1 root  root  69K Nov  8 23:08 lfd.log
    -rw------- 1 root  root 332K Nov  8 23:08 maillog
    -rw------- 1 root  500  1.6M Nov  8 23:10 cron
    
    For PHP-FPM error log:
    Code:
      tail -10 /var/log/php-fpm/www-error.log
    
    and/or
    Code:
      /var/log/php-fpm/www-php.error.log
    
    For MySQL / MariaDB error log:
    Code:
      tail -10 /var/log/mysqld.log
    
    For CSF firewall LFD log:
    Code:
      tail -10 /var/log/lfd.log
    
    For Mail log:
    Code:
      tail -10 /var/log/maillog
    
    For Cron job logs:
    Code:
      tail -10 /var/log/cron
    

    How to edit php.ini and php-fpm configuration files ?



    Centmin Mod install created command short cuts outlined here to allow you to quickly edit your /usr/local/lib/php.ini file and your /usr/local/etc/php-fpm.conf file. Full list of command shortcuts below:
    • Edit php.ini = phpedit ( /usr/local/lib/php.ini )
    • Edit my.cnf = mycnf ( /etc/my.cnf )
    • Edit php-fpm.conf = fpmconf ( /usr/local/etc/php-fpm.conf )
    • Edit nginx.conf = nginxconf ( /usr/local/nginx/conf/nginx.conf )
    • Edit (nginx) virtual.conf = vhostconf - only edits /usr/local/nginx/conf/conf.d/virtual.conf not the additional vhost domain.com.conf files added later
    • Edit (nginx) php.conf = phpinc ( /usr/local/nginx/conf/php.conf )
    • Edit (nginx) drop.conf = dropinc ( /usr/local/nginx/conf/drop.conf )
    • Edit (nginx) staticfiles.conf = statfilesinc ( /usr/local/nginx/conf/staticfiles.conf )
    • nginx stop/start/restart = ngxstop/ngxstart/ngxrestart
    • php-fpm stop/start/restart = fpmstop/fpmstart/fpmrestart
    • mysql stop/start/restart = mysqlstop/mysqlstart/mysqlrestart
    • nginx + php-fpm stop/start/restart = npstop/npstart/nprestart
    • memcached stop/start/restart =memcachedstop/memcachedstart/memcachedrestart
    • csf stop/start/restart = csfstop/csfstart/csfrestart

    Troubleshooting Tools



    However, there's many linux tools and scripts that can help you figure out what was causing the load issues and when.

    Tools and commands you will want to read up on and learn for basic system admin tasks and troubleshooting.
    Notes:
    However, Centmin Mod users are free to help each other out and ask questions or give answers on this community forum. My hopes are that this community forum evolves so that more veteran long time Centmin Mod users help new Centmin Mod users out :)
     
  4. savitzer

    savitzer New Member

    10
    0
    1
    Nov 23, 2018
    Ratings:
    +0
    Local Time:
    12:26 PM
    Thanks Eva, I looked into all the log files that you have mentioned in the documentation but didn't find anything. Anything else I should be looking at to figure out why mysqld is using heavy cpu?
     
  5. savitzer

    savitzer New Member

    10
    0
    1
    Nov 23, 2018
    Ratings:
    +0
    Local Time:
    12:26 PM
    Ok, I figured out what was causing the issue in the first place. It seems no cache options are working properly so that's creating overload on php-fpm when even 20+ users visit the site at a time.

    Now, I'm not sure why the cache isn't working. I installed wordpress using menu option 22. I tried with cache enabler, wp super cache & redis. Also, I have tried the same setup on different machines (Digitalocean's 4gb 2cpus & Scaleway's 16gb ram 8cpus bare metal)

    For now I'm using Cloudflare's cache everything so the site is running smooth.

    Any suggestions?
     
  6. eva2000

    eva2000 Administrator Staff Member

    37,227
    8,133
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,521
    Local Time:
    4:56 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    Tried updating 123.09beta01 first via cmupdate command before you do a centmin.sh menu option 22 run ? strange that no of the cache options work.. how are you verifying they are not working ?
     
  7. savitzer

    savitzer New Member

    10
    0
    1
    Nov 23, 2018
    Ratings:
    +0
    Local Time:
    12:26 PM
    This is from the nginx-helper logs

    Code:
    2018-11-29 02:40:21  | INFO | Purging post type archive ( post )
    2018-11-29 02:40:21  | ERROR | - Cache Not Found | http://domain.com
    2018-11-29 02:40:21  | INFO | * * * * *
    2018-11-29 02:40:21  | INFO | Purging date
    2018-11-29 02:40:21  | ERROR | - Cache Not Found | http://domain.com/2018/
    2018-11-29 02:40:21  | INFO | * * * * *
    2018-11-29 02:40:21  | ERROR | - Cache Not Found | http://domain.com/2018/11/
    2018-11-29 02:40:21  | INFO | * * * * *
    2018-11-29 02:40:21  | ERROR | - Cache Not Found | http://domain.com/2018/11/29/
    2018-11-29 02:40:21  | INFO | * * * * *
    
    The current setup is using redis cache.

    I've installed the beta version using this command everytime

    Code:
    yum -y update; curl -O https://centminmod.com/betainstaller7.sh && chmod 0700 betainstaller7.sh && bash betainstaller7.sh
    Do i still need to update 123.09beta01 first via cmupdate command?
     
  8. eva2000

    eva2000 Administrator Staff Member

    37,227
    8,133
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,521
    Local Time:
    4:56 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    123.09beta01 is continuously updated everyday - cmupdate updates the code for that so ideally yes

    that just says cache not found, which is correct if cache has been populated yet so not necessarily means cache not working just not populated yet
     
  9. savitzer

    savitzer New Member

    10
    0
    1
    Nov 23, 2018
    Ratings:
    +0
    Local Time:
    12:26 PM
    Ok, will do that tonight.

    How do I check if the cache is populated or not? Right now cloudflare is caching everything so things are running smooth. As soon as I take that layer off, the server gets flooded with too many php-fpm requests which I am assuming is because the cache is not working properly.
     
  10. eva2000

    eva2000 Administrator Staff Member

    37,227
    8,133
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,521
    Local Time:
    4:56 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    nginx-helper set it to write html comment at bottom of your pages i.e. on my Wordpress7 demo site which uses centmin.sh menu option 22 and redis nginx level caching and source code of page at bottom
    Code (Text):
    <!--Cached using Nginx-Helper on 2018-11-29 15:23:13. It took 136 queries executed in 0.471 seconds.-->
    <!--Visit http://wordpress.org/extend/plugins/nginx-helper/faq/ for more details-->
    

    any reason you disabling Cloudflare ? you should use it in conjunction with wordpress caching.
     
..