Join the community today
Become a Member

Nginx 502 bad gateaway

Discussion in 'Nginx, PHP-FPM & MariaDB MySQL' started by Oxide, Jun 17, 2015.

  1. Oxide

    Oxide Active Member

    534
    29
    28
    Mar 19, 2015
    Ratings:
    +59
    Local Time:
    7:48 PM
    When trying to send mail through "Contact Us" footer it gives this error. I am using XenForo.


    I am also using Mandrill, the mails sends successfully but for some reason this error appears. I heard it's because of php-fpm issue..

    While I know you only support regarding centmin it self, i thik this issue is not to do with my forum software.. and if it was, you also run xenforo so you might know why?
     
  2. eva2000

    eva2000 Administrator Staff Member

    54,860
    12,239
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,810
    Local Time:
    7:48 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Either connection to Mandril and/or PHP-FPM most likely

    Centmin Mod is provide as is, so short of scripted related bugs or issues, any further optimisation to the web stack components - nginx, php-fpm, mariadb mysql, csf firewall etc or web app specific configurations are left to the Centmin Mod user to deal with. So I do not provide any free support for such.

    Nginx 502 Bad Gateway Errors



    Bad gateway 502 is usually related to Nginx timing out waiting on PHP-FPM to respond as PHP-FPM is overloaded or overwhelmed with requests, so may need to tune PHP-FPM values. It also maybe due to PHP-FPM in turn being queued and backed up waiting on MariaDB MySQL server to respond - so also need to look at MySQL. Or could be delays in PHP-FPM connecting to Mandrill.

    You'll need to tune your PHP-FPM settings and this is left up to end user to do but here's a thread for starters to enable php status page output outlined at PHP PHP-FPM - Centmin Mod - Menu based Nginx installer for CentOS servers and PHP-FPM - pm.max_children | Centmin Mod Community which outlines the official PHP-FPM config documentation as well.

    Checking PHP-FPM etc logs



    You'll also need to check into your PHP-FPM, Nginx and MariaDB logs which you can find as outlined at How to troubleshoot Centmin Mod initial install issues

    Server logs include Nginx, PHP-FPM, MariaDB MySQL error logs as well as others. You can find your Centmin Mod install/menu logs at FAQ 7 and server logs at FAQ 19 at Centmin Mod FAQ (most up to date info in FAQ so always read that first). Spoiler tag below has info too but may not be up to date.

    Some of Centmin Mod's installed software will have their own access and error logs which maybe useful for diagnosing errors or give info, notes, or warning notices.

    Note: There's no support provided by me for diagnosing such errors which may occur for various reasons including misconfiguration of installed php/mysql scripts or applications.

    In SSH2 telnet you can use tail command to view the last X number of lines in the file.

    For example for viewing last 10 lines in the file for:

    For Nginx access and error logs:
    Code (Text):

    tail -10 /usr/local/nginx/logs/access.log
    tail -10 /usr/local/nginx/logs/error.log

    For specific domainname.com access and error log:
    Code (Text):

    tail -10 /home/nginx/domains/domainname.com/log/access.log
    tail -10 /home/nginx/domains/domainname.com/log/error.log

    For other system error logs located at /var/log:

    list /var/log files in ascending time order so the most recently modified files are at the bottom
    Code (Text):

    ls -lhrt /var/log

    Code (Text):

    total 2.7M
    -rw------- 1 root root 0 Aug 29 15:33 tallylog
    -rw------- 1 root root 0 Aug 29 15:33 spooler
    drwx------ 3 root root 4.0K Aug 29 15:35 samba
    drwxr-xr-x 2 root root 4.0K Aug 29 15:35 mail
    -rw-r--r-- 1 root 500 0 Oct 8 18:13 dmesg.old
    -rw------- 1 root 500 0 Oct 8 18:13 boot.log
    -rw-r--r-- 1 root 500 0 Oct 8 18:14 dmesg
    drwx------ 2 root root 4.0K Oct 8 18:14 httpd
    drwxr-xr-x 2 root root 4.0K Oct 8 19:08 php-fpm
    -rw-rw---- 1 mysql root 2.3K Oct 9 12:38 mysqld.log
    -rw------- 1 root root 9.2K Oct 26 10:48 yum.log
    -rw------- 1 root utmp 94K Nov 7 22:59 btmp
    drwxr-xr-x 2 root root 4.0K Nov 8 00:00 sa
    -rw------- 1 root root 269K Nov 8 21:39 messages
    -rw------- 1 root root 110K Nov 8 23:08 secure
    -rw-rw-r-- 1 root utmp 43K Nov 8 23:08 wtmp
    -rw-r--r-- 1 root root 144K Nov 8 23:08 lastlog
    -rw------- 1 root root 69K Nov 8 23:08 lfd.log
    -rw------- 1 root root 332K Nov 8 23:08 maillog
    -rw------- 1 root 500 1.6M Nov 8 23:10 cron

    For PHP-FPM error log:
    Code (Text):

    tail -10 /var/log/php-fpm/www-error.log

    and/or
    Code (Text):

    /var/log/php-fpm/www-php.error.log

    For MySQL / MariaDB error log:
    Code (Text):

    tail -10 /var/log/mysqld.log

    For CSF firewall LFD log:
    Code (Text):

    tail -10 /var/log/lfd.log

    For Mail log:
    Code (Text):

    tail -10 /var/log/maillog

    For Cron job logs:
    Code (Text):

    tail -10 /var/log/cron

    How to edit php.ini and php-fpm configuration files ?



    Centmin Mod install created command short cuts outlined here to allow you to quickly edit your /usr/local/lib/php.ini file and your /usr/local/etc/php-fpm.conf file. Full list of command shortcuts below:
    • Edit php.ini = phpedit ( /usr/local/lib/php.ini )
    • Edit my.cnf = mycnf ( /etc/my.cnf )
    • Edit php-fpm.conf = fpmconf ( /usr/local/etc/php-fpm.conf )
    • Edit nginx.conf = nginxconf ( /usr/local/nginx/conf/nginx.conf )
    • Edit (nginx) virtual.conf = vhostconf - only edits /usr/local/nginx/conf/conf.d/virtual.conf not the additional vhost domain.com.conf files added later
    • Edit (nginx) php.conf = phpinc ( /usr/local/nginx/conf/php.conf )
    • Edit (nginx) drop.conf = dropinc ( /usr/local/nginx/conf/drop.conf )
    • Edit (nginx) staticfiles.conf = statfilesinc ( /usr/local/nginx/conf/staticfiles.conf )
    • nginx stop/start/restart = ngxstop/ngxstart/ngxrestart
    • php-fpm stop/start/restart = fpmstop/fpmstart/fpmrestart
    • mysql stop/start/restart = mysqlstop/mysqlstart/mysqlrestart
    • nginx + php-fpm stop/start/restart = npstop/npstart/nprestart
    • memcached stop/start/restart =memcachedstop/memcachedstart/memcachedrestart
    • csf stop/start/restart = csfstop/csfstart/csfrestart

    Troubleshooting Tools



    However, there's many linux tools and scripts that can help you figure out what was causing the load issues and when.

    Tools and commands you will want to read up on and learn for basic system admin tasks and troubleshooting.
    Notes:
    However, Centmin Mod users are free to help each other out and ask questions or give answers on this community forum. My hopes are that this community forum evolves so that more veteran long time Centmin Mod users help new Centmin Mod users out :)
     
  3. Oxide

    Oxide Active Member

    534
    29
    28
    Mar 19, 2015
    Ratings:
    +59
    Local Time:
    7:48 PM
    Code:
    [root@root ~]# tail -10 /var/log/php-fpm/www-error.log
    [17-Jun-2015 09:25:38] WARNING: [pool www] child 21635 exited on signal 11 (SIGSEGV) after 1324.137894 seconds from start
    [17-Jun-2015 09:26:49] WARNING: [pool www] child 21638 exited on signal 11 (SIGSEGV) after 1319.465134 seconds from start
    [17-Jun-2015 09:36:01] WARNING: [pool www] child 25795 exited on signal 11 (SIGSEGV) after 552.292175 seconds from start
    [17-Jun-2015 09:36:28] WARNING: [pool www] child 21637 exited on signal 11 (SIGSEGV) after 1913.916550 seconds from start
    [17-Jun-2015 09:38:08] WARNING: [pool www] child 22824 exited on signal 11 (SIGSEGV) after 1704.555175 seconds from start
    [17-Jun-2015 09:42:27] WARNING: [pool www] child 28154 exited on signal 11 (SIGSEGV) after 358.939700 seconds from start
    [17-Jun-2015 09:43:00] WARNING: [pool www] child 29333 exited on signal 11 (SIGSEGV) after 32.652322 seconds from start
    [17-Jun-2015 09:43:23] WARNING: [pool www] child 28155 exited on signal 11 (SIGSEGV) after 315.629329 seconds from start
    [17-Jun-2015 09:44:02] WARNING: [pool www] child 29334 exited on signal 11 (SIGSEGV) after 61.926925 seconds from start
    [17-Jun-2015 09:45:12] WARNING: [pool www] child 25787 exited on signal 11 (SIGSEGV) after 1173.698622 seconds from start
    
    This happens after sending mail via mandrill smtp, idk if this is anything interesting.
     
  4. eva2000

    eva2000 Administrator Staff Member

    54,860
    12,239
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,810
    Local Time:
    7:48 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    what version of PHP ?
    Code:
    php -v
    extensions loaded ?
    Code:
    php --ini
    php -m
     
  5. Oxide

    Oxide Active Member

    534
    29
    28
    Mar 19, 2015
    Ratings:
    +59
    Local Time:
    7:48 PM
    all defualt installation:

    Code:
    [root@root ~]# php -v
    PHP 5.4.42 (cli) (built: Jun 16 2015 03:50:11)
    Copyright (c) 1997-2014 The PHP Group
    Zend Engine v2.4.0, Copyright (c) 1998-2014 Zend Technologies
    
    [root@root ~]# php --ini
    Configuration File (php.ini) Path: /usr/local/lib
    Loaded Configuration File:         /usr/local/lib/php.ini
    Scan for additional .ini files in: /etc/centminmod/php.d
    Additional .ini files parsed:      /etc/centminmod/php.d/apc.ini,
    /etc/centminmod/php.d/igbinary.ini,
    /etc/centminmod/php.d/imagick.ini,
    /etc/centminmod/php.d/memcache.ini,
    /etc/centminmod/php.d/memcached.ini
    
    [root@root ~]# php -m
    [PHP Modules]
    apc
    bcmath
    bz2
    calendar
    Core
    ctype
    curl
    date
    dom
    enchant
    ereg
    exif
    filter
    ftp
    gd
    gettext
    gmp
    hash
    iconv
    igbinary
    imagick
    imap
    intl
    json
    libxml
    mbstring
    mcrypt
    memcache
    memcached
    mhash
    mysql
    mysqli
    mysqlnd
    openssl
    pcntl
    pcre
    PDO
    pdo_mysql
    pdo_sqlite
    Phar
    posix
    pspell
    readline
    Reflection
    session
    shmop
    SimpleXML
    snmp
    soap
    sockets
    SPL
    sqlite3
    standard
    sysvmsg
    sysvsem
    sysvshm
    tidy
    tokenizer
    xml
    xmlreader
    xmlrpc
    xmlwriter
    zip
    zlib
    
    [Zend Modules]
    
    
     
  6. eva2000

    eva2000 Administrator Staff Member

    54,860
    12,239
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,810
    Local Time:
    7:48 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    php-fpm is segmentation faulting (segfaults) so need to found out why but that's up to you to troubleshoot i.e. doing straces on php processes etc

    Code:
    exited on signal 11 (SIGSEGV)
    I don't provide support for that as Centmin Mod is provided as is, Google might help see php+strace and php-fpm+segfaults

    could be APC Cache, so if you don't use APC Cache for data caching, can try centmin.sh menu option 12 submenu option 1 to install Zend Opcache instead of APC Cache
    Code:
    --------------------------------------------------------
    Centmin Mod 1.2.3-eva2000.08 - http://centminmod.com
    --------------------------------------------------------
                       Centmin Mod Menu                 
    --------------------------------------------------------
    1).  Centmin Install
    2).  Add Nginx vhost domain
    3).  NSD setup domain name DNS
    4).  Nginx Upgrade / Downgrade
    5).  PHP Upgrade / Downgrade
    6).  XCache Re-install
    7).  APC Cache Re-install
    8).  XCache Install
    9).  APC Cache Install
    10). Memcached Server Re-install
    11). MariaDB 5.2, 5.5, 10, 10.1 Upgrade Sub-Menu
    12). Zend OpCache Install/Re-install
    13). Install ioping.sh vbtechsupport.com/1239/
    14). SELinux disable
    15). Install/Re-install ImageMagick PHP Extension
    16). Change SSHD Port Number
    17). Multi-thread compression: pigz,pbzip2,lbzip2,p7zip etc
    18). Suhosin PHP Extension install
    19). Install FFMPEG and FFMPEG PHP Extension
    20). NSD Re-install
    21). Update - Nginx + PHP-FPM + Siege
    22). Add Wordpress Nginx vhost + WP Super Cache
    23). Update Centmin Mod Code Base
    24). Exit
    --------------------------------------------------------
    Enter option [ 1 - 24 ] 12
    --------------------------------------------------------
    
    --------------------------------------------------------
            Zend OpCache Sub-Menu            
    --------------------------------------------------------
    1). Install Zend OpCache for PHP <5.5 or <5.6
    2). Reinstall Zend OpCache for PHP <5.5 or <5.6
    3). Back to Main menu
    --------------------------------------------------------
    Enter option [ 1 - 3 ] 
    Code:
     php -v
    PHP 5.4.42 (cli) (built: Jun 17 2015 08:11:45)
    Copyright (c) 1997-2014 The PHP Group
    Zend Engine v2.4.0, Copyright (c) 1998-2014 Zend Technologies
        with Zend OPcache v7.0.5, Copyright (c) 1999-2015, by Zend Technologies
     
    Last edited: Jun 17, 2015
  7. Oxide

    Oxide Active Member

    534
    29
    28
    Mar 19, 2015
    Ratings:
    +59
    Local Time:
    7:48 PM
  8. eva2000

    eva2000 Administrator Staff Member

    54,860
    12,239
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,810
    Local Time:
    7:48 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    centos 6.6 already fixed CVE-2011-1720 in backported postfix 2.6.6-4
    Don't think it's related as I have including this forum, 40+ servers running Centmin Mod LEMP stack with postfix 2.6.6-6 latest available and with problems with smtp sending from local, mandril, amazon ses smtps.

    Code:
    yum list postfix -q                  
    Installed Packages
    postfix.x86_64                                              2:2.6.6-6.el6_5                                               @bas
    But yeah Nginx and PHP-FPM upgrades you should keep on top of
     
  9. Oxide

    Oxide Active Member

    534
    29
    28
    Mar 19, 2015
    Ratings:
    +59
    Local Time:
    7:48 PM
    [root@root src]# yum list postfix -q
    Installed Packages
    postfix.x86_64 2:2.6.6-6.el6_5 @base


    -- other than that, still issue after updating..