Want to subscribe to topics you're interested in?
Become a Member

Nginx PHP-FPM All website suddently not accessible on one server

Discussion in 'Nginx, PHP-FPM & MariaDB MySQL' started by yojimbo45, Oct 3, 2022.

  1. yojimbo45

    yojimbo45 New Member

    14
    3
    3
    Jan 10, 2020
    Ratings:
    +7
    Local Time:
    2:04 AM
    Hi, yesterday after adding a domain i have a server with 5+ website who suddenly get unreachable.
    I can log in SSH and i restarted several times servers but it still not working.
    Please which log should I check to understand what is causing an issue?

    setup: centminmod with php 8.1.11


    Cloudflare return : '
    Origin is unreachable Error code 523'

     
  2. eva2000

    eva2000 Administrator Staff Member

    54,365
    12,198
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,763
    Local Time:
    5:04 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Please fill in any relevant information that applies to you:
    • CentOS Version: i.e. CentOS 6 32bit or 64bit / CentOS 7 64bit ?
    • Centmin Mod Version Installed: i.e. 123.08stable or 123.09beta01
    • Nginx Version Installed: i.e. 1.15.3
    • PHP Version Installed: i.e. 5.6.37, 7.0.31, 7.1.21, 7.2.9
    • MariaDB MySQL Version Installed: i.e. 10.0.x or 10.1.xx or 10.2.xx
    • When was last time updated Centmin Mod code base ? : i.e. run centmin.sh menu option 23 submenu option 2 or cmupdate command
    • Persistent Config: Do you have any persistent config file options set in /etc/centminmod/custom_config.inc ? You can check via this command:
      Code (Text):
      cat /etc/centminmod/custom_config.inc
      

      Post output in CODE tags.
    https://support.cloudflare.com/hc/en-us/articles/115003011431-Troubleshooting-Cloudflare-5XX-errors

    Check that Nginx, PHP-FPM and MariaDB services are active and online/started

    For Nginx
    Code (Text):
    service nginx status | sed -e "s|$HOSTNAME|hostname|g"

    and check Nginx config
    Code (Text):
    nginx -t

    For PHP-FPM
    Code (Text):
    service php-fpm status | sed -e "s|$HOSTNAME|hostname|g"

    For CSF Firewall
    Code (Text):
    service csf status | sed -e "s|$HOSTNAME|hostname|g"

    For CSF's lfd login failure daemon
    Code (Text):
    service lfd status | sed -e "s|$HOSTNAME|hostname|g"
     
  3. yojimbo45

    yojimbo45 New Member

    14
    3
    3
    Jan 10, 2020
    Ratings:
    +7
    Local Time:
    2:04 AM
    Nginx is the problem, i have the following issues:

    [14:05][root@CentOS-79-64-minimal autoprotect]# nginx -t
    nginx: [emerg] unknown "q_ignorearg" variable
    nginx: configuration file /usr/local/nginx/conf/nginx.conf test failed



    [14:08][root@CentOS-79-64-minimal autoprotect]# service nginx status | sed -e "s|$HOSTNAME|hostname|g"
    Redirecting to /bin/systemctl status nginx.service
    ● nginx.service - Centmin Mod NGINX Server
    Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; vendor preset: disabled)
    Drop-In: /etc/systemd/system/nginx.service.d
    └─failure-restart.conf, openfileslimit.conf
    Active: failed (Result: start-limit) since Mon 2022-10-03 14:05:29 UTC; 3min 25s ago
    Process: 8552 ExecStartPre=/usr/local/sbin/nginx -t (code=exited, status=1/FAILURE)
    Main PID: 1784 (code=exited, status=0/SUCCESS)

    Oct 03 14:05:24 hostname systemd[1]: Unit nginx.service entered failed state.
    Oct 03 14:05:24 hostname systemd[1]: nginx.service failed.
    Oct 03 14:05:29 hostname systemd[1]: nginx.service holdoff time over, scheduling restart.
    Oct 03 14:05:29 hostname systemd[1]: Stopped Centmin Mod NGINX Server.
    Oct 03 14:05:29 hostname systemd[1]: start request repeated too quickly for nginx.service
    Oct 03 14:05:29 hostname systemd[1]: Failed to start Centmin Mod NGINX Server.
    Oct 03 14:05:29 hostname systemd[1]: Unit nginx.service entered failed state.
    Oct 03 14:05:29 hostname systemd[1]: nginx.service failed.
    [14:08][root@CentOS-79-64-minimal autoprotect]#
     
  4. eva2000

    eva2000 Administrator Staff Member

    54,365
    12,198
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,763
    Local Time:
    5:04 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Are you using centmin.sh menu option 22 Wordpress installer with Cache Enabler caching option selected or have used it in the past but disabled it later? That variable is part of cache enabler's mapped args variables at /usr/local/nginx/conf/wpcacheenabler_map.conf include file which is referenced in /usr/local/nginx/conf/nginx.conf
    Code (Text):
    map $http_user_agent $cmwpcache_device {
        default                                     'desktop';
        ~*(iPad|iPhone|Android|IEMobile|Blackberry) 'mobile';
        "~*Firefox.*Mobile"                         'mobile';
        "~*ipod.*mobile"                            'mobile';
        "~*Opera\ Mini"                             'mobile';
        "~*Opera\ Mobile"                           'mobile';
        "~*Mobile"                                  'mobile';
        "~*Tablet"                                  'mobile';
        "~*Kindle"                                  'mobile';
        "~*Windows\ Phone"                          'mobile';
    }
    
    map $args $q_ignorearg {
      default               0;
      "~*fbclid"            1;
      "~*gclid"             1;
      "~*utm"               1;
      "~*fb_action_ids"     1;
      "~*fb_action_types"   1;
      "~*fb_source"         1;
      "~*age-verified"      1;
      "~*ao_noptimize"      1;
      "~*usqp"              1;
      "~*cn-reloaded"       1;
      "~*_ga"               1;
      "~*_ke"               1;
      "~*mc_cid"            1;
      "~*mc_eid"            1;
      "~*ref"               1;
    }
    

    and then is usually referenced in your nginx vhost's include file for /usr/local/nginx/conf/wpincludes/yourdomain.com/wpcacheenabler_yourdomain.com.conf
    Code (Text):
        # include query strings fbclid, gclid, utm in cache via stripping them with
        # 302 redirect via mapping in /usr/local/nginx/conf/wpcacheenabler_map.conf
        if ($q_ignorearg) {
          set $check_qurl $request_uri;
          set $check_surl $request_uri;
          set $cache_uri $uri;
          #rewrite ^ $uri? redirect;
        }
    

    The error you have is most likely due to the include file /usr/local/nginx/conf/wpcacheenabler_map.conf not being enabled in /usr/local/nginx/conf/nginx.conf and your nginx vhost or include file at /usr/local/nginx/conf/wpincludes/yourdomain.com/wpcacheenabler_yourdomain.com.conf is referencing $q_ignorearg variable but it isn't finding it it declared in include file /usr/local/nginx/conf/wpcacheenabler_map.conf as it maybe commented out with a hash # in front or something is disabled
    Code (Text):
    grep -C1 wpcacheenabler_map.conf /usr/local/nginx/conf/nginx.conf
    include /usr/local/nginx/conf/redisupstream.conf;
    include /usr/local/nginx/conf/wpcacheenabler_map.conf;
    include /usr/local/nginx/conf/lua-package-path-include.conf;
    
     
  5. yojimbo45

    yojimbo45 New Member

    14
    3
    3
    Jan 10, 2020
    Ratings:
    +7
    Local Time:
    2:04 AM
    also noticed i don't have this file: /etc/init.d/nginx

    I use for all website the option 1) Keycdn cache enabler. So it should not be disabled
     
    Last edited: Oct 4, 2022
  6. eva2000

    eva2000 Administrator Staff Member

    54,365
    12,198
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,763
    Local Time:
    5:04 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    If you're using Centmin Mod 130.00beta01, it has now switched form init.d/nginx service to systemd native service file for fresh installs with SWITCH_NGINX_SYSTEMD='y' variable enabled by default
     
  7. yojimbo45

    yojimbo45 New Member

    14
    3
    3
    Jan 10, 2020
    Ratings:
    +7
    Local Time:
    2:04 AM
    Thank you for your replies.

    I added this 3 lines at /usr/local/nginx/conf/nginx.conf

    include /usr/local/nginx/conf/redisupstream.conf;
    include /usr/local/nginx/conf/wpcacheenabler_map.conf;
    include /usr/local/nginx/conf/lua-package-path-include.conf;

    i get now the error:

    Code:
    [16:16][root@CentOS-79-64-minimal autoprotect]# nginx -t
    nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
    nginx: [emerg] zero size shared memory zone "xwprpc"
    nginx: configuration file /usr/local/nginx/conf/nginx.conf test failed

    by comparing with a healthy nginx config file , i saw that one line was missing,
    this one:
    limit_req_zone $binary_remote_addr zone=xwprpc:10m rate=30r/s;

    now nginx is running
     
  8. eva2000

    eva2000 Administrator Staff Member

    54,365
    12,198
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,763
    Local Time:
    5:04 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    You may not need to add all 3 lines
    Code (Text):
    include /usr/local/nginx/conf/redisupstream.conf;
    include /usr/local/nginx/conf/wpcacheenabler_map.conf;
    include /usr/local/nginx/conf/lua-package-path-include.conf;
    

    the top and bottom ones are only added conditionally/dynamically by Centmin Mod if you use Redis based Nginx caching for Wordpress via centmin.sh menu option 22 as well and bottom if you use Nginx Lua.
     
  9. yojimbo45

    yojimbo45 New Member

    14
    3
    3
    Jan 10, 2020
    Ratings:
    +7
    Local Time:
    2:04 AM
    Now nginx is running but websites are still not reachable. PHP-fpm is also working and firewall is disabled. Not sure what to check now.



    Code:
    [16:44][root@CentOS-79-64-minimal tools]# nginx -t
    nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
    nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful
    
    Code:
    [16:38][root@CentOS-79-64-minimal tools]# service nginx status | sed -e "s|$HOSTNAME|hostname|g"
    Redirecting to /bin/systemctl status nginx.service
    ● nginx.service - Centmin Mod NGINX Server
       Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; vendor preset: disabled)
      Drop-In: /etc/systemd/system/nginx.service.d
               └─failure-restart.conf, openfileslimit.conf
       Active: active (running) since Mon 2022-10-03 16:37:31 UTC; 1min 30s ago
      Process: 9727 ExecStop=/bin/sh -c /bin/kill -s TERM $(/bin/cat /usr/local/nginx/logs/nginx.pid) (code=exited, status=0/SUCCE                      SS)
      Process: 9735 ExecStart=/usr/local/sbin/nginx -c /usr/local/nginx/conf/nginx.conf (code=exited, status=0/SUCCESS)
      Process: 9732 ExecStartPre=/usr/local/sbin/nginx -t (code=exited, status=0/SUCCESS)
     Main PID: 9736 (nginx)
       CGroup: /system.slice/nginx.service
               ├─9736 nginx: master process /usr/local/sbin/nginx -c /usr/local/nginx/conf/nginx.con
               ├─9737 nginx: worker process
               ├─9738 nginx: worker process
               ├─9739 nginx: worker process
               ├─9740 nginx: worker process
               ├─9741 nginx: worker process
               └─9742 nginx: worker process
    
    Oct 03 16:37:31 hostname systemd[1]: Starting Centmin Mod NGINX Server...
    Oct 03 16:37:31 hostname nginx[9732]: nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
    Oct 03 16:37:31 hostname nginx[9732]: nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful
    Oct 03 16:37:31 hostname systemd[1]: Started Centmin Mod NGINX Server.
    


    Code:
    ● php-fpm.service - PHP FastCGI Process Manager
       Loaded: loaded (/usr/lib/systemd/system/php-fpm.service; enabled; vendor preset: disabled)
      Drop-In: /etc/systemd/system/php-fpm.service.d
               └─failure-restart.conf, limit.conf
       Active: active (running) since Mon 2022-10-03 16:25:51 UTC; 18min ago
      Process: 1188 ExecStart=/usr/local/sbin/php-fpm --daemonize --fpm-config /usr/local/etc/php-fpm.conf --pid /var/run/php-fpm/php-fpm.pid (code=exited, status=0/SUCCESS)
     Main PID: 1622 (php-fpm)
       CGroup: /system.slice/php-fpm.service
               └─1622 php-fpm: master process (/usr/local/etc/php-fpm.conf)
    
    Oct 03 16:25:51 hostname systemd[1]: Starting PHP FastCGI Process Manager...
    Oct 03 16:25:51 hostname systemd[1]: Started PHP FastCGI Process Manager.
    
     
  10. eva2000

    eva2000 Administrator Staff Member

    54,365
    12,198
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,763
    Local Time:
    5:04 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    If you disable CSF Firewall, then ports aren't usually opened by default. Re-enable CSF Firewall so the proper ports are opened.
     
  11. yojimbo45

    yojimbo45 New Member

    14
    3
    3
    Jan 10, 2020
    Ratings:
    +7
    Local Time:
    2:04 AM
    oh yesss it's working, thank you so much Eva !
     
  12. Tracy Perry

    Tracy Perry Active Member

    280
    118
    43
    Aug 24, 2014
    Texas
    Ratings:
    +210
    Local Time:
    1:04 PM
    1.21.6
    MariaDB 10.3.36
    And THIS boys and girls is why I do, and will continue to, recommend CentMin either on CentOS 7 or the newer "stuff" to anyone that is willing to listen.
    If I had a few spare 10K in cash, I'd gladly send it to @eva2000, but I remain a simple civil servant that retired on a very limited retirement... but I DO encourage anyone that can - send money!
     
  13. cloud9

    cloud9 Premium Member Premium Member

    431
    117
    43
    Oct 6, 2015
    England
    Ratings:
    +217
    Local Time:
    7:04 PM
    1.25.3
    10.6.x
    I stay a premium member to support @eva2000 - his work is worth 10's of thousands for sure.....
     
  14. eva2000

    eva2000 Administrator Staff Member

    54,365
    12,198
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,763
    Local Time:
    5:04 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+