502 error with one user online

pamamolf · Nov 3, 2016

Hello

I have some steps that you can follow and always get a 502 error.....

So if you install on Centos 7.x 64bit Wordpress or Invision and set Maintenance mode ON after refreshing main page a few times you will get once result OK and once 502 error...

Don't know why but i can reproduce it anytime

I use one time installer latest Centminmod beta version....

What is the special thing i do?

After installation i am enabling io threads and enable multiple pools....

fpm settings are reasonable per pool like :
Code:
pm = dynamic
pm.max_children = 70
pm.start_servers = 10
pm.min_spare_servers = 5
pm.max_spare_servers = 68
pm.max_requests = 4000
So it will be great if anyone can test it so we may found what is causing this and fix it

If i disable maintenance mode i never get that error even if i got 500 users online....

Also if i am logged in i do not have that issue ! Only as guest i have the problem....

Don't know if it is related to any cookies or any cached systems like Memcached or Zend Opcache.....

Thanks

eva2000 · Nov 3, 2016

pamamolf said: ↑

enable multiple pools....
Click to expand...

how did you set multiple php-fpm pools ? as per Beta Branch - Centmin Mod .08 beta03+ Multiple PHP-FPM pools support added | Centmin Mod Community ?

outline your specific steps

pamamolf · Nov 4, 2016

Yes that's the exact steps that i follow

eva2000 · Nov 4, 2016

pamamolf said: ↑

Yes that's the exact steps that i follow
Click to expand...

there's 2 sets of steps there, list your exact steps done

pamamolf · Nov 5, 2016

Post #1

eva2000 · Nov 5, 2016

i meant exact steps for enabling multiple php-fpm pools

pamamolf · Nov 5, 2016

Code:

nano /usr/local/etc/php-fpm.conf

uncomment:

Code:

include=/usr/local/nginx/conf/phpfpmd/*.conf

Code:

nano /usr/local/nginx/conf/nginx.conf

under http { i add:

Code:

upstream phpbackend {
ip_hash;
keepalive 5;
  server 127.0.0.1:9000 weight=50;
  server 127.0.0.1:9002 weight=50;
  server 127.0.0.1:9003 weight=50;
  server 127.0.0.1:9004 weight=50;
  server 127.0.0.1:9005 weight=50;
}

Code:

nano /usr/local/nginx/conf/php.conf

comment :

Code:

#fastcgi_pass   127.0.0.1:9000;

under it i add:

Code:

fastcgi_next_upstream error timeout http_500 http_503;
fastcgi_pass phpbackend;
fastcgi_keep_conn on;

Code:

nano /usr/local/nginx/conf/phpstatus.conf

and i add:

Code:

            location ~ ^/(phpstatus-pool2|phpping-pool2)$ {
                access_log          off;
                fastcgi_pass        127.0.0.1:9002;
                fastcgi_param     SCRIPT_FILENAME $fastcgi_script_name;
                include          fastcgi_params;
                allow              127.0.0.1;
                #allow YOURIPADDRESS;
                deny                all;
            }

            location ~ ^/(phpstatus-pool3|phpping-pool3)$ {
                access_log          off;
                fastcgi_pass        127.0.0.1:9003;
                fastcgi_param     SCRIPT_FILENAME $fastcgi_script_name;
                include          fastcgi_params;
                allow              127.0.0.1;
                #allow YOURIPADDRESS;
                deny                all;
            }

            location ~ ^/(phpstatus-pool4|phpping-pool4)$ {
                access_log          off;
                fastcgi_pass        127.0.0.1:9004;
                fastcgi_param     SCRIPT_FILENAME $fastcgi_script_name;
                include          fastcgi_params;
                allow              127.0.0.1;
                #allow YOURIPADDRESS;
                deny                all;
            }

            location ~ ^/(phpstatus-pool5|phpping-pool5)$ {
                access_log          off;
                fastcgi_pass        127.0.0.1:9005;
                fastcgi_param     SCRIPT_FILENAME $fastcgi_script_name;
                include          fastcgi_params;
                allow              127.0.0.1;
                #allow YOURIPADDRESS;
                deny                all;
            }

Code:

nano /usr/local/nginx/conf/conf.d/virtual.conf

and i uncomment:

Code:

include /usr/local/nginx/conf/phpstatus.conf;

Then i restart phpfpm and Nginx also !

eva2000 · Nov 10, 2016

for this issue with 502 errors for php-fpm when using multiple php-fpm pools and sitestatus 503 maintenance feature, if you edit /usr/local/nginx/conf/php.conf and change

from
Code (Text):
fastcgi_intercept_errors on;
to
Code (Text):
fastcgi_intercept_errors off;
restart nginx and php-fpm
Code (Text):
nprestart
does it help or fix the issue ?

Module ngx_http_fastcgi_module

Syntax: fastcgi_intercept_errors on | off;
Default:
fastcgi_intercept_errors off;
Context: http, server, location
Determines whether FastCGI server responses with codes greater than or equal to 300 should be passed to a client or be intercepted and redirected to nginx for processing with the error_page directive.
Click to expand...

pamamolf · Nov 10, 2016

Nope it doesn't work

When i click after the edit and nprestart to open the page is ok and then when i refresh 502 error and then on refresh ok and the 502 again and so on

pamamolf · Nov 10, 2016

I disable also the multiple pools:
Code:
;include=/usr/local/nginx/conf/phpfpmd/*.conf
then nprestart and then again the same issue

eva2000 · Nov 10, 2016

what if you edit /usr/local/nginx/conf/503include-main.conf to add

Code (Text):

open_file_cache off;

so it becomes

Code (Text):

    # only uncomment if you do not need to exclude images or js
    # from 503 redirect rewrites
     #include /usr/local/nginx/conf/503include-only.conf;
     error_page 503 @maintenance;
     location @maintenance {
          #if ($maintenance = 1) {
          rewrite ^ /maintenance.html break;
          open_file_cache off;
          #ry_files /maintenance.html =503;
          #}
     }

restart nginx

eva2000 · Nov 10, 2016

Check your nginx and php error logs for any clues

Nginx in your access.log and error.log in vhost listed paths in your nginx vhost where domain.com is your domain name at
Code (Text):
tail -25 /home/nginx/domains/domain.com/log/access.log
tail -25 /home/nginx/domains/domain.com/log/error.log
PHP-FPM in
Code (Text):
tail -25 /var/log/php-fpm/www-error.log
and/or
Code (Text):
tail -25 /var/log/php-fpm/www-php.error.log
tail -25 lists last 25 lines of the log.

pamamolf · Nov 10, 2016

I don't use that file as i have commented it:
Code:
#include /usr/local/nginx/conf/503include-main.conf;

eva2000 · Nov 10, 2016

for now but when you had enabled maintenance mode that file was uncommented right ? you need to uncomment 2 files for 503 sitestatus maintenance mode to work a /usr/local/nginx/conf/503include-main.conf and /usr/local/nginx/conf/503include-only.conf include files. From example at Beta Branch - sitestatus maintenance mode | Centmin Mod Community
Code (Text):
  root /home/nginx/domains/testdomain.com/public;
  include /usr/local/nginx/conf/503include-main.conf;

  location / {
  include /usr/local/nginx/conf/503include-only.conf;
...
  }

pamamolf · Nov 10, 2016

Code:

/home/nginx/domains/domain.com/log/access.log

Code:

123.456.789.000 - - [10/Nov/2016:04:01:58 +0000] "GET / HTTP/2.0" 503 7077 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.59 Safari/537.36 OPR/41.0.2353.46" "-" rt=0.100 ua="127.0.0.1:9003, 127.0.0.1:9004, 127.0.0.1:9005, 127.0.0.1:9002, 127.0.0.1:9000" us="502, 502, 502, 502, 503" ut="0.000, 0.000, 0.000, 0.000, 0.100" ul="0, 0, 0, 0, 6832" cs=-
123.456.789.000 - - [10/Nov/2016:04:02:02 +0000] "GET / HTTP/2.0" 502 668 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.59 Safari/537.36 OPR/41.0.2353.46" "-" rt=0.000 ua="127.0.0.1:9000, phpbackend" us="502, 502" ut="0.000, 0.000" ul="0, 0" cs=-
123.456.789.000 - - [10/Nov/2016:04:02:03 +0000] "GET / HTTP/2.0" 503 6933 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.59 Safari/537.36 OPR/41.0.2353.46" "-" rt=0.000 ua="127.0.0.1:9003, 127.0.0.1:9004, 127.0.0.1:9005, 127.0.0.1:9002, 127.0.0.1:9000" us="502, 502, 502, 502, 503" ut="0.000, 0.000, 0.000, 0.000, 0.000" ul="0, 0, 0, 0, 6837" cs=-
123.456.789.000 - - [10/Nov/2016:04:02:04 +0000] "GET / HTTP/2.0" 502 668 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.59 Safari/537.36 OPR/41.0.2353.46" "-" rt=0.000 ua="127.0.0.1:9000, phpbackend" us="502, 502" ut="0.000, 0.000" ul="0, 0" cs=-
123.456.789.000 - - [10/Nov/2016:04:02:05 +0000] "GET / HTTP/2.0" 503 6933 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.59 Safari/537.36 OPR/41.0.2353.46" "-" rt=0.000 ua="127.0.0.1:9003, 127.0.0.1:9004, 127.0.0.1:9005, 127.0.0.1:9002, 127.0.0.1:9000" us="502, 502, 502, 502, 503" ut="0.000, 0.000, 0.000, 0.000, 0.000" ul="0, 0, 0, 0, 6837" cs=

Code:

/home/nginx/domains/domain.com/log/error.log

Code:

2016/11/10 04:02:04 [error] 12003#12003: *1 no live upstreams while connecting to upstream, client: 123.456.789.000, server: domain.com, request: "GET / HTTP/2.0", upstream: "fastcgi://phpbackend", host: "domain.com"
2016/11/10 04:02:05 [error] 12003#12003: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 123.456.789.000, server: domain.com, request: "GET / HTTP/2.0", upstream: "fastcgi://127.0.0.1:9003", host: "domain.com"
2016/11/10 04:02:05 [error] 12003#12003: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 123.456.789.000, server: domain.com, request: "GET / HTTP/2.0", upstream: "fastcgi://127.0.0.1:9004", host: "domain.com"
2016/11/10 04:02:05 [error] 12003#12003: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 123.456.789.000, server: domain.com, request: "GET / HTTP/2.0", upstream: "fastcgi://127.0.0.1:9005", host: "domain.com"
2016/11/10 04:02:05 [error] 12003#12003: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 123.456.789.000, server: domain.com, request: "GET / HTTP/2.0", upstream: "fastcgi://127.0.0.1:9002", host: "domain.com"
2016/11/10 04:02:06 [error] 12004#12004: *21 connect() failed (111: Connection refused) while connecting to upstream, client: 123.456.789.000, server: domain.com, request: "GET /?app=core&module=system&controller=ajax&do=instantNotifications&csrfKey=aa7d5381d26b1fcbe0bbda3d8da4b96c&notifications=0&messages=0 HTTP/2.0", upstream: "fastcgi://127.0.0.1:9003", host: "domain.com", referrer: "https://domain.com/"
2016/11/10 04:02:06 [error] 12004#12004: *21 connect() failed (111: Connection refused) while connecting to upstream, client: 123.456.789.000, server: domain.com, request: "GET /?app=core&module=system&controller=ajax&do=instantNotifications&csrfKey=aa7d5381d26b1fcbe0bbda3d8da4b96c&notifications=0&messages=0 HTTP/2.0", upstream: "fastcgi://127.0.0.1:9004", host: "domain.com", referrer: "https://domain.com/"
2016/11/10 04:02:06 [error] 12004#12004: *21 connect() failed (111: Connection refused) while connecting to upstream, client: 123.456.789.000, server: domain.com, request: "GET /?app=core&module=system&controller=ajax&do=instantNotifications&csrfKey=aa7d5381d26b1fcbe0bbda3d8da4b96c&notifications=0&messages=0 HTTP/2.0", upstream: "fastcgi://127.0.0.1:9005", host: "domain.com", referrer: "https://domain.com/"
2016/11/10 04:02:06 [error] 12004#12004: *21 connect() failed (111: Connection refused) while connecting to upstream, client: 123.456.789.000, server: domain.com, request: "GET /?app=core&module=system&controller=ajax&do=instantNotifications&csrfKey=aa7d5381d26b1fcbe0bbda3d8da4b96c&notifications=0&messages=0 HTTP/2.0", upstream: "fastcgi://127.0.0.1:9002", host: "domain.com", referrer: "https://domain.com/"

Code:

/var/log/php-fpm/www-error.log

empty

Code:

/var/log/php-fpm/www-php.error.log

empty

pamamolf · Nov 10, 2016

I am not talking about Centminmod maintenance mode and i have commented all lines related to that file......

I am talking about Invision set forum as offline and maintenance mode for Wordpress....

Or do you mean something else?

eva2000 · Nov 10, 2016

looks like pools 2-5 the additional pools aren't responding or refusing connections/down so give 502

pamamolf said: ↑

I am not talking about Centminmod maintenance mode and i have commented all lines related to that file......

I am talking about Invision set forum as offline and maintenance mode for Wordpress....

Or do you mean something else?
Click to expand...

oh i was talking about sitestatus 503 maintenance mode heh

eva2000 · Nov 10, 2016

what does your Invision board nginx vhost look like for domain.com.conf and/or domain.com.ssl.conf ?

pamamolf · Nov 10, 2016

looks like pools 2-5 the additional pools aren't responding or refusing connections/down so give 502
Click to expand...

I can get the same issue even with multiple pools disabled by:
Code:
#include /usr/local/nginx/conf/503include-main.conf;

pamamolf · Nov 10, 2016

Code:

#x# HTTPS-DEFAULT
 server {
 
   server_name domain.com www.domain.com;
   return 302 https://$server_name$request_uri;
   include /usr/local/nginx/conf/staticfiles-hsts.conf;
 }

server {
  listen 443 ssl http2;
  server_name domain.com www.domain.com;

  include /usr/local/nginx/conf/ssl/domain.com/domain.com.crt.key.conf;
  include /usr/local/nginx/conf/ssl_include.conf;

  http2_max_field_size 16k;
  http2_max_header_size 32k;
  # mozilla recommended
  ssl_ciphers EECDH+CHACHA20:EECDH+CHACHA20-draft:EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA256:EECDH+ECDSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+SHA384:EECDH+AES128:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS:!RC4:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA:!CAMELLIA;
  ssl_prefer_server_ciphers   on;
  #add_header Alternate-Protocol  443:npn-spdy/3;

  # before enabling HSTS line below read centminmod.com/nginx_domain_dns_setup.html#hsts
  add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";
  #add_header X-Frame-Options SAMEORIGIN;
  #add_header X-Xss-Protection "1; mode=block" always;
  #add_header X-Content-Type-Options "nosniff" always;
  #spdy_headers_comp 5;
  ssl_buffer_size 1369;
  ssl_session_tickets on;
 
  # enable ocsp stapling
  resolver 8.8.8.8 8.8.4.4 valid=10m;
  resolver_timeout 10s;
  ssl_stapling on;
  ssl_stapling_verify on;

# ngx_pagespeed & ngx_pagespeed handler
#include /usr/local/nginx/conf/pagespeed.conf;
#include /usr/local/nginx/conf/pagespeedhandler.conf;
#include /usr/local/nginx/conf/pagespeedstatslog.conf;

  # limit_conn limit_per_ip 16;
  # ssi  on;

  access_log /home/nginx/domains/domain.com/log/access.log main_ext buffer=256k flush=60m;
  error_log /home/nginx/domains/domain.com/log/error.log;

  include /usr/local/nginx/conf/autoprotect/domain.com/autoprotect-domain.com.conf;
  root /home/nginx/domains/domain.com/public;
  # uncomment cloudflare.conf include if using cloudflare for
  # server and/or vhost site
  #include /usr/local/nginx/conf/cloudflare.conf;
  #include /usr/local/nginx/conf/503include-main.conf;

  location / {
  #include /usr/local/nginx/conf/503include-only.conf;

# block common exploits, sql injections etc
include /usr/local/nginx/conf/block.conf;

  # Enables directory listings when index file not found
  #autoindex  on;

  # Shows file listing times as local time
  #autoindex_localtime on;

  # Enable for vBulletin usage WITHOUT vbSEO installed
  try_files    $uri $uri/ /index.php;

  }

  include /usr/local/nginx/conf/staticfiles-hsts.conf;
  #include /usr/local/nginx/conf/staticfiles.conf;
  include /usr/local/nginx/conf/php.conf;
  include /usr/local/nginx/conf/drop.conf;
  #include /usr/local/nginx/conf/errorpage.conf;
  include /usr/local/nginx/conf/vts_server.conf;
}

But it is very easy to produce it by installing Centminmod 09 beta and then wordpress and then set it to maintenance mode and check.......

As you may be able to find the cause more easy if you can produce it yourself.... ?

The problem exist on http also ! Tested on another server with Wordpress and http....

Log in / Register

Forums

Join the community today

502 error with one user online

pamamolf Premium Member Premium Member

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

pamamolf Premium Member Premium Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

pamamolf Premium Member Premium Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

pamamolf Premium Member Premium Member

.

Log in / Register

Useful Searches

Join the community today

502 error with one user online

pamamolf Premium Member Premium Member

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

pamamolf Premium Member Premium Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

pamamolf Premium Member Premium Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

pamamolf Premium Member Premium Member

.