Nginx 504 Gateway Timeout error

danielsl · May 19, 2016

Error 502 after adding new domain and editing redirects in conf

hi
encountered the same behaviour
logs
error access emty
but from time to to time error.log

Code:

2016/05/18 09:15:41 [error] 21060#21060: *103947 open() "/home/nginx/domains/e36v.ru/public/assets/components/bootstrap/dist/js/bootstrap.min.js" failed (2: No such file or directory), client: 128.71.178.200, server: e36v.ru, request: "GET /assets/components/bootstrap/dist/js/bootstrap.min.js HTTP/1.1", host: "e36v.ru", referrer: "http://e36v.ru/"
2016/05/18 09:15:48 [error] 21060#21060: *103943 open() "/home/nginx/domains/e36v.ru/public/assets/components/bootstrap/dist/js/bootstrap.min.js" failed (2: No such file or directory), client: 128.71.178.200, server: e36v.ru, request: "GET /assets/components/bootstrap/dist/js/bootstrap.min.js HTTP/1.1", host: "e36v.ru", referrer: "http://e36v.ru/index.php?id=10"
2016/05/18 09:45:24 [error] 22224#22224: *44 open() "/home/nginx/domains/e36v.ru/public/assets/components/bootstrap/dist/js/bootstrap.min.js" failed (2: No such file or directory), client: 37.9.122.201, server: e36v.ru, request: "GET /assets/components/bootstrap/dist/js/bootstrap.min.js HTTP/1.1", host: "e36v.ru"
2016/05/18 10:34:59 [error] 22224#22224: *74 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 50.121.214.104, server: e36v.ru, request: "GET /index.php?id=10 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "e36v.ru"
2016/05/18 10:35:40 [error] 22224#22224: *78 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 50.121.214.104, server: e36v.ru, request: "GET /index.php?id=12 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "e36v.ru"
2016/05/18 10:36:33 [error] 22224#22224: *80 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 50.121.214.104, server: e36v.ru, request: "GET /index.php?id=9 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "e36v.ru"
2016/05/18 12:14:18 [notice] 22224#22224: *189 "^/(.*)$" matches "/admin/", client: 89.21.79.152, server: e36v.ru, request: "GET /admin/ HTTP/1.1", host: "e36v.ru"

09 BETA
PHP7.0.1
CENTMIN32B 6.5

all worked just fine for a year until i added new domain and started editing conf redirects rules
and after few

Code:

service nginx restart;s

all my domains become unavailable except
mysql works and accessible remotely
static files are served
node and jetty proxied just fine
but no response from php scripts
i tryed to restart php-fpm(no errors)
i updated downgraded php version,
deleted domain conf, restarted and even tryed to create it again via sh console but it doesnt create this domain conf anymore despite sh console running without errors.

Code:

tryed yum update

but i get

Code:

1716 packages excluded due to repository priority protections

and randomly (after nginx restart)i get some domains work some not even made on the same cms

eva2000 · May 19, 2016

danielsl said: ↑

e36v.ru
Click to expand...

Is that the new domain with new rewrites added ? try disable that domain temporarily by backing up and removing the nginx vhost files for them at /usr/local/nginx/conf/conf.d/e36v.ru.conf and /usr/local/nginx/conf/conf.d/e36v.ru.ssl.conf and restart nginx and php-fpm and see

When you create a new nginx vhost domain via centmin.sh menu option 2 or menu option 22 or via /usr/bin/nv cli command line, you will create the Nginx vhost files and directories. You will get an outputted the path location where it will create the domain name's vhost conf file named newdomain.com.conf (and newdomain.com.ssl.conf if you selected yes to self signed SSL)

Nginx vhost conf path will be at /usr/local/nginx/conf/conf.d/newdomain.com.conf

Nginx HTTP/2 SSL vhost conf path will be at /usr/local/nginx/conf/conf.d/newdomain.com.ssl.conf

Nginx Self-Signed SSL Certificate Directory at /usr/local/nginx/conf/ssl/newdomain.com

Vhost public web root will be at /home/nginx/domains/newdomain.com/public

Vhost log directory will be at /home/nginx/domains/newdomain.com/log

Please post the contents of /usr/local/nginx/conf/conf.d/newdomain.com.conf and if applicable /usr/local/nginx/conf/conf.d/newdomain.com.ssl.conf wrapped in CODE tags (outlined at How to use forum BBCODE code tags)

eva2000 · May 19, 2016

fyi, visited the site i see 504 not 502 which is a timeout error due to php-fpm most likely with incorrect rewrite rules and/or your web app/php script setup

504 Gateway Time-out
The server didn't respond in time.
Click to expand...

danielsl · May 19, 2016

MODX CMF Manager Login i have 502
but on 4steps.ru i have 504
Daniel Ko Photography proxied to jetty just fine

this one i never edited since making it running under nginx

Code:

# redirect from non-www to www
# uncomment, save file and restart Nginx to enable
#server {
#            listen   80;
#            server_name 4steps.ru;
#            return 301 $scheme://www.4steps.ru$request_uri;
#       }

server {
  server_name 4steps.ru www.4steps.ru;

# ngx_pagespeed & ngx_pagespeed handler
#include /usr/local/nginx/conf/pagespeed.conf;
#include /usr/local/nginx/conf/pagespeedhandler.conf;
#include /usr/local/nginx/conf/pagespeedstatslog.conf;

  # limit_conn limit_per_ip 16;
  # ssi  on;

  access_log /home/nginx/domains/4steps.ru/log/access.log combined buffer=32k;
  error_log /home/nginx/domains/4steps.ru/log/error.log;

  root /home/nginx/domains/4steps.ru/public;

if (!-e $request_filename){
                        rewrite ^/(.*)$ /index.php?q=$1 last;
            }

  location / {

# block common exploits, sql injections etc
#include /usr/local/nginx/conf/block.conf;

  # Enables directory listings when index file not found
  #autoindex  on;

  # Shows file listing times as local time
  #autoindex_localtime on;

  # Enable for vBulletin usage WITHOUT vbSEO installed
  #try_files        $uri $uri/ /index.php;

  }

  include /usr/local/nginx/conf/staticfiles.conf;
  include /usr/local/nginx/conf/php.conf;
  include /usr/local/nginx/conf/drop.conf;
  include /usr/local/nginx/conf/errorpage.conf;
}

e36v
edited but then returned on default to the state it worked before the system ruined

Code:

# redirect from non-www to www
# uncomment, save file and restart Nginx to enable
#server {
#            listen   80;
#            server_name e36v.ru;
#            return 301 $scheme://www.e36v.ru$request_uri;
#       }

server {
    listen   80;
        server_name e36v.ru;
    rewrite_log on;
    index    index.php index.html;
    root /home/nginx/domains/e36v.ru/public;


#    return 301 $scheme://e36v.ru$request_uri;
#ngx_pagespeed & ngx_pagespeed handler
#include /usr/local/nginx/conf/pagespeed.conf;
#include /usr/local/nginx/conf/pagespeedhandler.conf;
#include /usr/local/nginx/conf/pagespeedstatslog.conf;

  # limit_conn limit_per_ip 16;
  # ssi  on;

  access_log /home/nginx/domains/e36v.ru/log/access.log combined buffer=32k;
  error_log /home/nginx/domains/e36v.ru/log/error.log notice;


    # редирект с ww на без-www
#   if ($host != 'e36v.ru' ) {
#       rewrite  ^/(.*)$  http://e36v.ru/$1  permanent;
#  }
#location ~ ^/(.*?)/index.html$ {
#        rewrite            ^/(.*?)/ /$1/ permanent;
#    } 
    # если нет запрошенного файла - слать в локацию rewrite
    location / {
        root /home/nginx/domains/e36v.ru/public;
        try_files    $uri $uri/ @rewrite;
    }

    # отсюда слать всё на index.php - это есть быть friendly urls
    location @rewrite {
        rewrite        ^/(.*)$ /index.php?q=$1;
    }

    # а вот и наша защита системных директорий по ip
#    location ~* ^\/(manager|core|connectors)\/(?:.*)$ {
#        allow        184.55.68.97;
#        deny        all;
#    location ~* \.php$ {
#           include        fastcgi_params;
#            fastcgi_param    SCRIPT_FILENAME $document_root$fastcgi_script_name;
#            fastcgi_pass    127.0.0.1:9000;
#        }
#    }

    # а это выполнение всех php файлов, не попавших в предыдущий блок
    location ~* \.php$ {
    try_files $uri =404;
    fastcgi_split_path_info ^(.+\.php)(/.*)$;

    #limit_req    zone=one  burst=3;
        include        fastcgi_params;
        fastcgi_param    SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_pass   127.0.0.1:9000;
    #include fastcgi_params;
        fastcgi_ignore_client_abort on;
        fastcgi_param  SERVER_NAME $http_host; 
    }

    # статику отдаём без участия MODX и php
    location ~* ^.+\.(jpg|jpeg|gif|css|png|js|ico|bmp|woff|eot)$ {
       access_log        off;
       expires            10d;
       break;
    }

    # на всякий случай запрещаем чтение .htaccess - вдруг у вас там еще и апач есть?
    location ~ /\.ht {
        deny            all;
    }

  include /usr/local/nginx/conf/staticfiles.conf;
  include /usr/local/nginx/conf/php.conf;
  include /usr/local/nginx/conf/drop.conf;
  include /usr/local/nginx/conf/errorpage.conf;
}

Also i even deleted in from conf folder, centmin.sh couldnot recreat it again via 2 menu, the folder simply didnt have it after adding

no ssl

i tryed disabling all except one or two old domains but it didnt make any change.

just after countless nginx restart e36v.ru started working but others not

eva2000 · May 19, 2016

you have to restart both nginx and php-fpm as 504 timeouts is php-fpm related and nginx con't connect/timed out
Code (Text):
nprestart

eva2000 · May 19, 2016

you also have duplicate php location matches like

Code (Text):

    # а это выполнение всех php файлов, не попавших в предыдущий блок
    location ~* \.php$ {
    try_files $uri =404;
    fastcgi_split_path_info ^(.+\.php)(/.*)$;

    #limit_req    zone=one  burst=3;
        include        fastcgi_params;
        fastcgi_param    SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_pass   127.0.0.1:9000;
    #include fastcgi_params;
        fastcgi_ignore_client_abort on;
        fastcgi_param  SERVER_NAME $http_host;
    }

you don't need them as centmin mod auto generated nginx vhosts have their own match in include file for php.conf

Code (Text):

  include /usr/local/nginx/conf/php.conf;

danielsl · May 19, 2016

i deleted all ..conf ..except those mentioned 3 domains
i deleted duplicated location as you mentioned(but it worked in the past flawlessly) in e36v.ru
i nprestartED

but my old domain no help
4steps.ru
timeout

----------------------
also when this problem first arose. i checked Htop processess. centmin.sh has been taking 30% cpu for 528Hours, and i terminated it and restarted vps

eva2000 · May 19, 2016

Might try nginx debug mode to dig deeper

Latest update on September 14, 2015 to Centmin Mod 1.2.3-eva2000.08 stable (123.08stable) and higher has added Nginx debug mode support. Nginx debug mode is disabled by default.

To enable, edit centmin.sh and change variable NGINX_DEBUG=n default to NGINX_DEBUG=y and recompile Nginx via centmin.sh menu option 4 and then setting error_log in nginx vhosts to debug as outlined at nginx.org/en/docs/debugging_log.html & wiki.nginx.org/Debugging.

You do not want to leave Nginx debug mode running forever, so after debugging, set NGINX_DEBUG=n in centmin.sh and recompile Nginx again via centmin.sh menu option 4 and remove error_log debugging mode to disable Nginx debug mode again.

To update your Centmin Mod builds code for Nginx debug mode support if you do not have a NGINX_DEBUG variable in centmin.sh, follow instructions at centminmod.com/upgrade.html and respective version threads below:

Centmin Mod 1.2.3-eva2000.08 stable install & update thread

Centmin Mod 123.09beta thread

Centmin Mod is provided as is, but you can try debugging mode for Nginx for further troubleshooting if you have problems with Nginx (i.e. segfaults / signal 11 issues) as outlined at nginx.org/en/docs/debugging_log.html & wiki.nginx.org/Debugging.

eva2000 · May 19, 2016

Also anything in /home/nginx/domains/4steps.ru/log/error.log ?

danielsl · May 19, 2016

very strange in the log request to different domain name 9ah.ru which i also have

Code:

2016/05/18 23:49:52 [error] 23421#23421: *30 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 185.83.218.184, server: 4steps.ru, request: "GET /wp-cron.php?doing_wp_cron=1 HTTP/1.0", upstream: "fastcgi://127.0.0.1:9000", host: "9ah.ru"
2016/05/18 23:50:07 [error] 23421#23421: *41 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 128.71.178.231, server: 4steps.ru, request: "GET / HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "4steps.ru"
2016/05/18 23:50:12 [error] 23421#23421: *43 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 185.83.218.184, server: 4steps.ru, request: "GET /wp-cron.php?doing_wp_cron=1 HTTP/1.0", upstream: "fastcgi://127.0.0.1:9000", host: "9ah.ru"
2016/05/18 23:53:22 [error] 23421#23421: *51 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 128.71.178.231, server: 4steps.ru, request: "GET / HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "4steps.ru"

also when i open new web link via google chrome it sometimes redirect me to previous adrress, i tryed on different PCs it sometimes happens(all right after this misfortunate accident)

----
recompiled with option 4
added

Code:

  error_log /home/nginx/domains/4steps.ru/log/error.log;
error_log /home/nginx/domains/4steps.ru/log/errord.log debug;

but
service nginx-debug start
-->unrecognized service

eva2000 · May 19, 2016

danielsl said: ↑

service nginx-debug start
Click to expand...

Just nginx service restart not nginx-debug

eva2000 · May 19, 2016

danielsl said: ↑

also when i open new web link via google chrome it sometimes redirect me to previous adrress, i tryed on different PCs it sometimes happens(all right after this misfortunate accident)
Click to expand...

clear your web browser caches and try in incognito mode

danielsl said: ↑

very strange in the log request to different domain name 9ah.ru which i also have
Click to expand...

what does your 9ah.ru nginx vhost conf files look like ?

danielsl · May 19, 2016

9ah.ru conf is not in conf folder now for debugging purposes

Code:

# Centmin Mod Getting Started Guide
# must read http://centminmod.com/getstarted.html

# redirect from non-www to www
# uncomment, save file and restart Nginx to enable
# if unsure use return 302 before using return 301
#server {
#            listen   80;
#            server_name 9ah.ru;
#            return 301 $scheme://www.9ah.ru$request_uri;
#       }

server {

  server_name 9ah.ru www.9ah.ru;

# ngx_pagespeed & ngx_pagespeed handler
#include /usr/local/nginx/conf/pagespeed.conf;
#include /usr/local/nginx/conf/pagespeedhandler.conf;
#include /usr/local/nginx/conf/pagespeedstatslog.conf;

  #add_header X-Frame-Options SAMEORIGIN;
  #add_header X-Xss-Protection "1; mode=block" always;
  #add_header X-Content-Type-Options "nosniff" always;

  # limit_conn limit_per_ip 16;
  # ssi  on;

  access_log /home/nginx/domains/9ah.ru/log/access.log combined buffer=256k flush=60m;
  error_log /home/nginx/domains/9ah.ru/log/error.log;

  root /home/nginx/domains/9ah.ru/public;
  # uncomment cloudflare.conf include if using cloudflare for
  # server and/or vhost site
  #include /usr/local/nginx/conf/cloudflare.conf;
  include /usr/local/nginx/conf/503include-main.conf;

  include /usr/local/nginx/conf/wpincludes/9ah.ru/wpcacheenabler_9ah.ru.conf;
  #include /usr/local/nginx/conf/wpincludes/9ah.ru/wpsupercache_9ah.ru.conf;
  # https://community.centminmod.com/posts/18828/
  #include /usr/local/nginx/conf/wpincludes/9ah.ru/rediscache_9ah.ru.conf;

  location / {
  include /usr/local/nginx/conf/503include-only.conf;


  # Enables directory listings when index file not found
  #autoindex  on;

  # for wordpress super cache plugin
  #try_files /wp-content/cache/supercache/$http_host/$cache_uri/index.html $uri $uri/ /index.php?q=$uri&$args;

  # for wp cache enabler plugin
  try_files $cache_enabler_uri $uri $uri/ $custom_subdir/index.php?$args;  

  # Wordpress Permalinks
  #try_files $uri $uri/ /index.php?q=$uri&$args;

  # Nginx level redis Wordpress
  # https://community.centminmod.com/posts/18828/
  #try_files $uri $uri/ /index.php?$args;

  }

location ~* /(wp-login\.php) {
    limit_req zone=xwplogin burst=1 nodelay;
    #limit_conn xwpconlimit 30;
    #auth_basic "Private";
    #auth_basic_user_file /home/nginx/domains/9ah.ru/htpasswd_wplogin;
    include /usr/local/nginx/conf/php-wpsc.conf;
    # https://community.centminmod.com/posts/18828/
    #include /usr/local/nginx/conf/php-rediscache.conf;
}

location ~* /(xmlrpc\.php) {
    limit_req zone=xwprpc burst=45 nodelay;
    #limit_conn xwpconlimit 30;
    include /usr/local/nginx/conf/php-wpsc.conf;
    # https://community.centminmod.com/posts/18828/
    #include /usr/local/nginx/conf/php-rediscache.conf;
}

  include /usr/local/nginx/conf/wpincludes/9ah.ru/wpsecure_9ah.ru.conf;
  include /usr/local/nginx/conf/php-wpsc.conf;
  # https://community.centminmod.com/posts/18828/
  #include /usr/local/nginx/conf/php-rediscache.conf;
  include /usr/local/nginx/conf/staticfiles.conf;
  include /usr/local/nginx/conf/drop.conf;
  #include /usr/local/nginx/conf/errorpage.conf;
  include /usr/local/nginx/conf/vts_server.conf;
}

Also I just opened iexplorer inprivate mode without result
nginx restarted
----------
error debug output is here also
/wp-cron.php is from 9ah.ru or icholding.ru(also deleted for testing purposes) wordpress installation. it should not be in 4steps.ru error debug path. i.m shocked

Code:

2016/05/19 00:16:00 [notice] 25983#25983: *5 "^/(.*)$" matches "/shop/feed/", client: 66.249.66.98, server: 4steps.ru, request: "GET /shop/feed/?product_tag=tee HTTP/1.1", host: "9ah.ru"
2016/05/19 00:16:00 [notice] 25983#25983: *5 rewritten data: "/index.php", args: "q=shop/feed/&product_tag=tee", client: 66.249.66.98, server: 4steps.ru, request: "GET /shop/feed/?product_tag=tee HTTP/1.1", host: "9ah.ru"
2016/05/19 00:16:23 [info] 25983#25983: *7 client 5.143.231.133 closed keepalive connection
2016/05/19 00:16:52 [notice] 25983#25983: *8 "^/(.*)$" matches "/wp-cron.php", client: 185.83.218.184, server: 4steps.ru, request: "GET /wp-cron.php?doing_wp_cron=1 HTTP/1.0", host: "9ah.ru"
2016/05/19 00:16:52 [notice] 25983#25983: *8 rewritten data: "/index.php", args: "q=wp-cron.php&doing_wp_cron=1", client: 185.83.218.184, server: 4steps.ru, request: "GET /wp-cron.php?doing_wp_cron=1 HTTP/1.0", host: "9ah.ru"
2016/05/19 00:17:12 [notice] 25983#25983: *10 "^/(.*)$" matches "/wp-cron.php", client: 185.83.218.184, server: 4steps.ru, request: "GET /wp-cron.php?doing_wp_cron=1 HTTP/1.0", host: "9ah.ru"
2016/05/19 00:17:12 [notice] 25983#25983: *10 rewritten data: "/index.php", args: "q=wp-cron.php&doing_wp_cron=1", client: 185.83.218.184, server: 4steps.ru, request: "GET /wp-cron.php?doing_wp_cron=1 HTTP/1.0", host: "9ah.ru"
2016/05/19 00:19:00 [info] 25983#25983: *5 epoll_wait() reported that client prematurely closed connection, so upstream connection is closed too while sending request to upstream, client: 66.249.66.98, server: 4steps.ru, request: "GET /shop/feed/?product_tag=tee HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "9ah.ru"
2016/05/19 00:19:52 [error] 25983#25983: *8 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 185.83.218.184, server: 4steps.ru, request: "GET /wp-cron.php?doing_wp_cron=1 HTTP/1.0", upstream: "fastcgi://127.0.0.1:9000", host: "9ah.ru"
2016/05/19 00:19:52 [info] 25983#25983: *8 client 185.83.218.184 closed keepalive connection
2016/05/19 00:20:12 [error] 25983#25983: *10 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 185.83.218.184, server: 4steps.ru, request: "GET /wp-cron.php?doing_wp_cron=1 HTTP/1.0", upstream: "fastcgi://127.0.0.1:9000", host: "9ah.ru"
2016/05/19 00:20:12 [info] 25983#25983: *10 client 185.83.218.184 closed keepalive connection
2016/05/19 00:20:35 [error] 25983#25983: *12 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 178.79.138.229, server: 4steps.ru, request: "GET / HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "www.4steps.ru"
2016/05/19 00:20:35 [info] 25983#25983: *12 client 178.79.138.229 closed keepalive connection
2016/05/19 00:20:57 [error] 25983#25983: *14 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 178.79.138.229, server: 4steps.ru, request: "GET / HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "www.4steps.ru"
2016/05/19 00:21:20 [info] 26197#26197: *3 client closed connection while waiting for request, client: 128.71.178.231, server: 0.0.0.0:80
2016/05/19 00:21:20 [info] 26197#26197: *5 client closed connection while waiting for request, client: 128.71.178.231, server: 0.0.0.0:80
2016/05/19 00:21:20 [info] 26197#26197: *4 client closed connection while waiting for request, client: 128.71.178.231, server: 0.0.0.0:80
2016/05/19 00:22:47 [notice] 26197#26197: *6 "^/(.*)$" matches "/xmlrpc.php", client: 86.105.50.40, server: 4steps.ru, request: "POST /xmlrpc.php HTTP/1.1", host: "icholding.ru"
2016/05/19 00:22:47 [notice] 26197#26197: *6 rewritten data: "/index.php", args: "q=xmlrpc.php", client: 86.105.50.40, server: 4steps.ru, request: "POST /xmlrpc.php HTTP/1.1", host: "icholding.ru"
2016/05/19 00:22:55 [info] 26197#26197: *6 epoll_wait() reported that client prematurely closed connection, so upstream connection is closed too while sending request to upstream, client: 86.105.50.40, server: 4steps.ru, request: "POST /xmlrpc.php HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "icholding.ru"
2016/05/19 00:24:01 [error] 26197#26197: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 128.71.178.231, server: 4steps.ru, request: "GET / HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "4steps.ru"
2016/05/19 00:25:07 [notice] 26197#26197: *8 "^/(.*)$" matches "/cond_calculate", client: 180.76.15.150, server: 4steps.ru, request: "GET /cond_calculate HTTP/1.1", host: "icholding.ru"
2016/05/19 00:25:07 [notice] 26197#26197: *8 rewritten data: "/index.php", args: "q=cond_calculate", client: 180.76.15.150, server: 4steps.ru, request: "GET /cond_calculate HTTP/1.1", host: "icholding.ru"
2016/05/19 00:26:01 [info] 26197#26197: *11 client closed connection while waiting for request, client: 128.71.178.231, server: 0.0.0.0:80
2016/05/19 00:26:01 [info] 26197#26197: *13 client closed connection while waiting for request, client: 128.71.178.231, server: 0.0.0.0:80
2016/05/19 00:28:07 [error] 26197#26197: *8 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 180.76.15.150, server: 4steps.ru, request: "GET /cond_calculate HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "icholding.ru"
2016/05/19 00:28:07 [info] 26197#26197: *8 recv() failed (104: Connection reset by peer) while sending response to client, client: 180.76.15.150, server: 4steps.ru, request: "GET /cond_calculate HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "icholding.ru"
2016/05/19 00:28:32 [info] 26197#26197: *16 client timed out (110: Connection timed out) while waiting for request, client: 128.71.178.231, server: 0.0.0.0:80
2016/05/19 00:28:41 [error] 26197#26197: *10 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 128.71.178.231, server: 4steps.ru, request: "GET / HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "4steps.ru"

it seems i need to reinstall server and all addons

eva2000 · May 19, 2016

yeah you'd need to troubelshoot your site vhosts and your nginx rewrite rules as alot of them are related to 4steps.ru before the connection timesout

danielsl · May 19, 2016

i never changed any rules in vhost since it worked for a year. i have this random strange problem just after adding new domain, it seems configuration got corrupted somewhere else. thanks anyway. is it possible to reinstall clean delete and fresh install of centminmod without domain directories deleting. i have db backups

eva2000 · May 19, 2016

best to backup all site data and vhost files and then reload CentOS OS clean and install fresh centmin mod 123.09beta01 (latest improvements) Beta Branch - Centmin Mod .09 beta branch Testing | Centmin Mod Community

You can do this if you want by setting up a new server i.e. VPS and install on that, then slowly move data from old to new server. This gives you chance to compare before and after configuration/settings etc

danielsl · May 22, 2016

Hi again.
after reinstalling centos 6.7 min+centminmod 09 beta01+few of web domain--> got back to the same issue.
random dissapearance. 500 error

Code:

 tail -f /var/log/php-fpm/www-error.log
[21-May-2016 14:38:03] WARNING: [pool www] server reached max_children setting (4), consider raising it
[21-May-2016 14:41:10] WARNING: [pool www] server reached max_children setting (4), consider raising it
[21-May-2016 14:42:54] WARNING: [pool www] server reached max_children setting (4), consider raising it

not sure if its a problem as i had this present config for a year

Code:

var/log/nginx

tail -f /var/log/php-fpm/www-error.log

2016/05/21 03:39:25 [error] 25390#25390: *5733 open() "/usr/local/nginx/html/contacts.html" failed (2: No such file or directory), client: 144.76.61.21, server: pan.0dan.ru, request: "GET /contacts.html HTTP/1.0", host: "f0ta.ru"
2016/05/21 03:39:26 [error] 25390#25390: *5734 open() "/usr/local/nginx/html/portfolio-album.html" failed (2: No such file or directory), client: 144.76.61.21, server: pan.0dan.ru, request: "GET /portfolio-album.html HTTP/1.0", host: "f0ta.ru"
2016/05/21 03:39:28 [error] 25390#25390: *5735 open() "/usr/local/nginx/html/replica.html" failed (2: No such file or directory), client: 144.76.61.21, server: pan.0dan.ru, request: "GET /replica.html HTTP/1.0", host: "f0ta.ru"
2016/05/21 04:08:23 [error] 25390#25390: *5908 open() "/usr/local/nginx/html/sitemap.xml" failed (2: No such file or directory), client: 66.249.66.27, server: pan.0dan.ru, request: "GET /sitemap.xml HTTP/1.1", host: "f.0dan.ru"
2016/05/21 05:07:52 [error] 25390#25390: *6261 open() "/usr/local/nginx/html/minimal-base/tours/extreme.html" failed (2: No such file or directory), client: 207.46.13.190, server: pan.0dan.ru, request: "GET /minimal-base/tours/extreme.html HTTP/1.1", host: "www.go2ch.ru"
2016/05/21 05:14:03 [error] 25390#25390: *6301 open() "/usr/local/nginx/html/vip.html" failed (2: No such file or directory), client: 220.181.108.151, server: pan.0dan.ru, request: "GET /vip.html?color=27&dir=asc&occasion=31&order=name HTTP/1.1", host: "magento.0dan.ru"
2016/05/21 06:27:52 [error] 25390#25390: *6737 open() "/usr/local/nginx/html/img/portfolio/money.jpg" failed (2: No such file or directory), client: 5.255.253.86, server: pan.0dan.ru, request: "GET /img/portfolio/money.jpg HTTP/1.1", host: "f0ta.ru"
2016/05/21 06:29:52 [error] 25390#25390: *6748 open() "/usr/local/nginx/html/img/portfolio/food3.jpg" failed (2: No such file or directory), client: 141.8.183.19, server: pan.0dan.ru, request: "GET /img/portfolio/food3.jpg HTTP/1.1", host: "f0ta.ru"
2016/05/21 06:54:41 [error] 25390#25390: *6894 "/usr/local/nginx/html/catalog/product_compare/add/product/425/uenc/aHR0cDovL21hZ2VudG8uMGRhbi5ydS8,/form_key/4x9fwz0dzllULYtP/index.php" is not found (2: No such file or directory), client: 123.125.71.14, server: pan.0dan.ru, request: "GET /catalog/product_compare/add/product/425/uenc/aHR0cDovL21hZ2VudG8uMGRhbi5ydS8,/form_key/4x9fwz0dzllULYtP/ HTTP/1.1", host: "magento.0dan.ru"
2016/05/21 07:30:12 [error] 25390#25390: *7103 "/usr/local/nginx/html/all-products/index.php" is not found (2: No such file or directory), client: 157.55.39.203, server: pan.0dan.ru, request: "GET /all-products/ HTTP/1.1", host: "9ah.ru"
2016/05/21 07:30:12 [error] 25390#25390: *7103 "/usr/local/nginx/html/shop-contact/index.php" is not found (2: No such file or directory), client: 157.55.39.203, server: pan.0dan.ru, request: "GET /shop-contact/ HTTP/1.1", host: "9ah.ru"
2016/05/21 07:30:12 [error] 25390#25390: *7103 "/usr/local/nginx/html/blog/index.php" is not found (2: No such file or directory), client: 157.55.39.203, server: pan.0dan.ru, request: "GET /blog/ HTTP/1.1", host: "9ah.ru"
2016/05/21 07:30:16 [error] 25390#25390: *7106 "/usr/local/nginx/html/123/duis-ornare-at-lobortis/index.php" is not found (2: No such file or directory), client: 207.46.13.90, server: pan.0dan.ru, request: "GET /123/duis-ornare-at-lobortis/ HTTP/1.1", host: "9ah.ru"
2016/05/21 07:30:16 [error] 25390#25390: *7106 "/usr/local/nginx/html/login/index.php" is not found (2: No such file or directory), client: 207.46.13.90, server: pan.0dan.ru, request: "GET /login/ HTTP/1.1", host: "9ah.ru"
2016/05/21 07:30:16 [error] 25390#25390: *7106 "/usr/local/nginx/html/15346/hitrices-orci-leo-dolor/index.php" is not found (2: No such file or directory), client: 207.46.13.90, server: pan.0dan.ru, request: "GET /15346/hitrices-orci-leo-dolor/ HTTP/1.1", host: "9ah.ru"
2016/05/21 07:31:08 [error] 25390#25390: *7110 "/usr/local/nginx/html/sample-page/index.php" is not found (2: No such file or directory), client: 207.46.13.136, server: pan.0dan.ru, request: "GET /sample-page/ HTTP/1.1", host: "9ah.ru"
2016/05/21 07:31:08 [error] 25390#25390: *7110 "/usr/local/nginx/html/shop-testimonials/index.php" is not found (2: No such file or directory), client: 207.46.13.136, server: pan.0dan.ru, request: "GET /shop-testimonials/ HTTP/1.1", host: "9ah.ru"
2016/05/21 07:31:09 [error] 25390#25390: *7110 "/usr/local/nginx/html/date/2012/09/19/index.php" is not found (2: No such file or directory), client: 207.46.13.136, server: pan.0dan.ru, request: "GET /date/2012/09/19/ HTTP/1.1", host: "9ah.ru"
2016/05/21 07:36:33 [error] 25390#25390: *7145 "/usr/local/nginx/html/all-products/terms-and-conditions/index.php" is not found (2: No such file or directory), client: 157.55.39.237, server: pan.0dan.ru, request: "GET /all-products/terms-and-conditions/ HTTP/1.1", host: "9ah.ru"
2016/05/21 07:36:33 [error] 25390#25390: *7145 "/usr/local/nginx/html/15138/luna-ipsum-nulla-dolor/index.php" is not found (2: No such file or directory), client: 157.55.39.237, server: pan.0dan.ru, request: "GET /15138/luna-ipsum-nulla-dolor/ HTTP/1.1", host: "9ah.ru"
2016/05/21 07:40:37 [error] 25390#25390: *7167 open() "/usr/local/nginx/html/wp-includes/wlwmanifest.xml" failed (2: No such file or directory), client: 5.255.253.67, server: pan.0dan.ru, request: "GET /wp-includes/wlwmanifest.xml HTTP/1.1", host: "9ah.ru"
2016/05/21 07:42:16 [error] 25390#25390: *7179 open() "/usr/local/nginx/html/сервисное-обслуживание" failed (2: No such file or directory), client: 144.76.8.132, server: pan.0dan.ru, request: "GET /%D1%81%D0%B5%D1%80%D0%B2%D0%B8%D1%81%D0%BD%D0%BE%D0%B5-%D0%BE%D0%B1%D1%81%D0%BB%D1%83%D0%B6%D0%B8%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5 HTTP/1.0", host: "icholding.ru"
2016/05/21 07:42:19 [error] 25390#25390: *7180 open() "/usr/local/nginx/html/install" failed (2: No such file or directory), client: 144.76.8.132, server: pan.0dan.ru, request: "GET /install HTTP/1.0", host: "icholding.ru"
2016/05/21 07:42:20 [error] 25390#25390: *7181 open() "/usr/local/nginx/html/services.html" failed (2: No such file or directory), client: 144.76.8.132, server: pan.0dan.ru, request: "GET /services.html HTTP/1.0", host: "icholding.ru"
2016/05/21 08:04:54 [error] 25390#25390: *7323 open() "/usr/local/nginx/html/img/portfolio/book_cover.jpg" failed (2: No such file or directory), client: 66.249.66.73, server: pan.0dan.ru, request: "GET /img/portfolio/book_cover.jpg HTTP/1.1", host: "f0ta.ru"
2016/05/21 08:30:45 [error] 25390#25390: *7333 open() "/usr/local/nginx/html/2November2014MartialArt" failed (2: No such file or directory), client: 89.163.148.58, server: pan.0dan.ru, request: "GET /2November2014MartialArt HTTP/1.0", host: "f.0dan.ru"
2016/05/21 08:30:47 [error] 25390#25390: *7334 open() "/usr/local/nginx/html/3011282" failed (2: No such file or directory), client: 89.163.148.58, server: pan.0dan.ru, request: "GET /3011282 HTTP/1.0", host: "f.0dan.ru"

servers and hosts are mixed somehow-> this log really puzzles me

I usually ssh and use mc to edit files via putty
at the same time sftp via far manager to edit files
and tunel mysql via plink to 3307:127.0.0.1:3306 to update upload mysql dumps

use openVZ vps

eva2000 · May 22, 2016

danielsl said: ↑

WARNING: [pool www] server reached max_children setting (4), consider raising it
Click to expand...

ah that explains it means php-fpm is overloaded and crashes and those = 504 gateway timeouts

Centmin Mod is provide as is, so short of scripted related bugs or issues, any further optimisation to the web stack components - nginx, php-fpm, mariadb mysql, csf firewall etc or web app specific configurations are left to the Centmin Mod user to deal with. So I do not provide any free support for such.

Nginx 502 or 504 Bad Gateway Errors

Bad gateway 502 /504 timeouts are usually related to Nginx timing out waiting on PHP-FPM to respond as PHP-FPM is overloaded or overwhelmed with requests, so may need to tune PHP-FPM values. It also maybe due to PHP-FPM in turn being queued and backed up waiting on MariaDB MySQL server to respond - so also need to look at MySQL.

You'll need to tune your PHP-FPM settings and this is left up to end user to do but here's a thread for starters to enable php status page output outlined at PHP-FPM - CentminMod.com LEMP Nginx web stack for CentOS and PHP-FPM - pm.max_children | Centmin Mod Community and PHP-FPM - WARNING: [pool www] server reached max_children setting (50), consider raising it | Centmin Mod Community which outlines the official PHP-FPM config documentation as well.

Checking PHP-FPM etc logs

You'll also need to check into your PHP-FPM, Nginx and MariaDB logs which you can find as outlined at How to troubleshoot Centmin Mod initial install issues

Server logs include Nginx, PHP-FPM, MariaDB MySQL error logs as well as others. You can find your Centmin Mod install/menu logs at FAQ 7 and server logs at FAQ 19 at Centmin Mod FAQ (most up to date info in FAQ so always read that first). Spoiler tag below has info too but may not be up to date.

Some of Centmin Mod's installed software will have their own access and error logs which maybe useful for diagnosing errors or give info, notes, or warning notices.

Note: There's no support provided by me for diagnosing such errors which may occur for various reasons including misconfiguration of installed php/mysql scripts or applications.

In SSH2 telnet you can use tail command to view the last X number of lines in the file.

For example for viewing last 10 lines in the file for:

For Nginx access and error logs:
Code:
tail -10 /usr/local/nginx/logs/access.log
tail -10 /usr/local/nginx/logs/error.log
For specific domainname.com access and error log:
Code:
tail -10 /home/nginx/domains/domainname.com/log/access.log
tail -10 /home/nginx/domains/domainname.com/log/error.log
For other system error logs located at /var/log:

list /var/log files in ascending time order so the most recently modified files are at the bottom
Code:
ls -lhrt /var/log
Code:
total 2.7M
-rw------- 1 root root 0 Aug 29 15:33 tallylog
-rw------- 1 root root 0 Aug 29 15:33 spooler
drwx------ 3 root root 4.0K Aug 29 15:35 samba
drwxr-xr-x 2 root root 4.0K Aug 29 15:35 mail
-rw-r--r-- 1 root 500 0 Oct 8 18:13 dmesg.old
-rw------- 1 root 500 0 Oct 8 18:13 boot.log
-rw-r--r-- 1 root 500 0 Oct 8 18:14 dmesg
drwx------ 2 root root 4.0K Oct 8 18:14 httpd
drwxr-xr-x 2 root root 4.0K Oct 8 19:08 php-fpm
-rw-rw---- 1 mysql root 2.3K Oct 9 12:38 mysqld.log
-rw------- 1 root root 9.2K Oct 26 10:48 yum.log
-rw------- 1 root utmp 94K Nov 7 22:59 btmp
drwxr-xr-x 2 root root 4.0K Nov 8 00:00 sa
-rw------- 1 root root 269K Nov 8 21:39 messages
-rw------- 1 root root 110K Nov 8 23:08 secure
-rw-rw-r-- 1 root utmp 43K Nov 8 23:08 wtmp
-rw-r--r-- 1 root root 144K Nov 8 23:08 lastlog
-rw------- 1 root root 69K Nov 8 23:08 lfd.log
-rw------- 1 root root 332K Nov 8 23:08 maillog
-rw------- 1 root 500 1.6M Nov 8 23:10 cron
For PHP-FPM error log:
Code:
tail -10 /var/log/php-fpm/www-error.log
and/or
Code:
/var/log/php-fpm/www-php.error.log
For MySQL / MariaDB error log:
Code:
tail -10 /var/log/mysqld.log
For CSF firewall LFD log:
Code:
tail -10 /var/log/lfd.log
For Mail log:
Code:
tail -10 /var/log/maillog
For Cron job logs:
Code:
tail -10 /var/log/cron
How to edit php.ini and php-fpm configuration files ?

Centmin Mod install created command short cuts outlined here to allow you to quickly edit your /usr/local/lib/php.ini file and your /usr/local/etc/php-fpm.conf file. Full list of command shortcuts below:

Edit php.ini = phpedit ( /usr/local/lib/php.ini )

Edit my.cnf = mycnf ( /etc/my.cnf )

Edit php-fpm.conf = fpmconf ( /usr/local/etc/php-fpm.conf )

Edit nginx.conf = nginxconf ( /usr/local/nginx/conf/nginx.conf )

Edit (nginx) virtual.conf = vhostconf - only edits /usr/local/nginx/conf/conf.d/virtual.conf not the additional vhost domain.com.conf files added later

Edit (nginx) php.conf = phpinc ( /usr/local/nginx/conf/php.conf )

Edit (nginx) drop.conf = dropinc ( /usr/local/nginx/conf/drop.conf )

Edit (nginx) staticfiles.conf = statfilesinc ( /usr/local/nginx/conf/staticfiles.conf )

nginx stop/start/restart = ngxstop/ngxstart/ngxrestart

php-fpm stop/start/restart = fpmstop/fpmstart/fpmrestart

mysql stop/start/restart = mysqlstop/mysqlstart/mysqlrestart

nginx + php-fpm stop/start/restart = npstop/npstart/nprestart

memcached stop/start/restart =memcachedstop/memcachedstart/memcachedrestart

csf stop/start/restart = csfstop/csfstart/csfrestart

Troubleshooting Tools

However, there's many linux tools and scripts that can help you figure out what was causing the load issues and when.

Tools and commands you will want to read up on and learn for basic system admin tasks and troubleshooting.

20 Command Line Tools to Monitor Linux Performance

13 Linux Performance Monitoring Tools - Part 2

80 Linux Monitoring Tools for SysAdmins - Server Density Blog

Amazing ! 25 Linux Performance Monitoring Tools

20 Linux System Monitoring Tools Every SysAdmin Should Know

My mysqlmymonlite.sh script can help with that mysqlmymonlite.sh Addon | Centmin Mod Community README text file has instructions for setting up a cronjob to email you reports i.e. every 10/15min or every hour.

Notes:

Centmin Mod already installed Glances tool. It can be invoked from SSH command glances or top2. Official Glances site at nicolargo.github.io/glances/ and documentation here.

However, Centmin Mod users are free to help each other out and ask questions or give answers on this community forum. My hopes are that this community forum evolves so that more veteran long time Centmin Mod users help new Centmin Mod users out

danielsl · May 22, 2016

that;s more then i expected .
Thank you very Much

danielsl · May 22, 2016

danielsl said: ↑

that;s more then i expected .
Thank you very Much
Click to expand...

but in this case , why another web name works?????
icholding.ru the one i set the last. I thought everything should go down. I.m i mistaken

Log in / Register

Forums

Get the most out of your Centmin Mod LEMP stack

Nginx 504 Gateway Timeout error

danielsl New Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

danielsl New Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

danielsl New Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

danielsl New Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

danielsl New Member

eva2000 Administrator Staff Member

danielsl New Member

eva2000 Administrator Staff Member

danielsl New Member

eva2000 Administrator Staff Member

Nginx 502 or 504 Bad Gateway Errors

Checking PHP-FPM etc logs

How to edit php.ini and php-fpm configuration files ?

Troubleshooting Tools

danielsl New Member

danielsl New Member

.

Log in / Register

Useful Searches

Get the most out of your Centmin Mod LEMP stack

Nginx 504 Gateway Timeout error

danielsl New Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

danielsl New Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

danielsl New Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

danielsl New Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

danielsl New Member

eva2000 Administrator Staff Member

danielsl New Member

eva2000 Administrator Staff Member

danielsl New Member

eva2000 Administrator Staff Member

Nginx 502 or 504 Bad Gateway Errors

Checking PHP-FPM etc logs

How to edit php.ini and php-fpm configuration files ?

Troubleshooting Tools

danielsl New Member

danielsl New Member

.