Beta Branch Centmin Mod .08 beta03+ Multiple PHP-FPM pools support added

Setting Up Multiple PHP-FPM Pools in Centmin Mod .08 beta 03+ and higher

Centmin Mod .08 beta 03+ and higher has added very basic multiple PHP-FPM pool support. It's tuning and optimisation is left up to you. However, the tip is you can only really properly utilise all PHP-FPM pools in an upstream if you have enough cpu cores/threads to support the cummulative number pm.max_children values across the main www pool + the 4 additional pool2 to pool5 that you set in each phpfpm_poolX.conf file. Otherwise, you risk overloading your server. Nevertheless, you can use multiple PHP-FPM pools in other non-upstream ways too such as dedicated PHP-FPM pool for specific parts of your web site with their own timeout and other PHP-FPM settings (see below).

Notes:

• Nov 11th, 2016 update: If your forum software gives 503 http status codes when closing the forums, you may need to add an additional workaround for multiple php-fpm pool configs outlined here.
• For existing .08 beta03 users, you need to copy over the files after updating to latest .08 beta03 as instructed below here.

PHP-FPM Pool Config Files

Rename the new php-fpm_poolX.conf-disabled files to php-fpm_poolX.conf at /usr/local/nginx/conf/phpfpmd. Update, no longer need to rename them as they default to without -disabled endings now.

Each phpfpm_poolX.conf file respectively has a listen = 127.0.0.1:9002 through to 127.0.0.1:9005, their own slowlog, pm.status_path, ping.path, own php-fpm-poolX.sock path (commented out), own error_log path at /var/log/php-fpm/www-php.error-poolX.log and named the pools as pool2, pool3, pool4, and pool5.

Code:

ls -lah /usr/local/nginx/conf/phpfpmd
total 24K
drwxr-xr-x 2 root root 4.0K Jun 26 05:57 .
drwxr-xr-x 5 root root 4.0K Jun 26 05:34 ..
-rw-r--r-- 1 root root 1.8K Jun 26 05:55 phpfpm_pool2.conf
-rw-r--r-- 1 root root 1.7K Jun 26 05:35 phpfpm_pool3.conf
-rw-r--r-- 1 root root 1.7K Jun 26 05:35 phpfpm_pool4.conf
-rw-r--r-- 1 root root 1.7K Jun 26 05:35 phpfpm_pool5.conf

Then edit /usr/local/etc/php-fpm.conf and uncomment and remove semi-colon ; from include line

from

Code:

;include=/usr/local/nginx/conf/phpfpmd/*.conf

to

Code:

include=/usr/local/nginx/conf/phpfpmd/*.conf

Restart Nginx and PHP-FPM

Nginx Upstreams

Setup upstreams as per Module ngx_http_upstream_module in your /usr/local/nginx/conf/nginx.conf file within http { context location

---

so below these directives

Code:

http {
 map_hash_bucket_size 128;
 map_hash_max_size 4096;
 server_names_hash_bucket_size 128;
 server_names_hash_max_size 2048;
 variables_hash_max_size 2048;

add

Code:

upstream phpbackend {
ip_hash;
keepalive 5;
  server 127.0.0.1:9000 weight=50;
  server 127.0.0.1:9002 weight=50;
  server 127.0.0.1:9003 weight=50;
  server 127.0.0.1:9004 weight=50;
  server 127.0.0.1:9005 weight=50;
}

For 123.09beta01 or older, edit /usr/local/nginx/conf/php.conf replace OR comment out with hash # in front

Code:

fastcgi_pass   127.0.0.1:9000;

and replace with or add

Code:

fastcgi_next_upstream error timeout http_500 http_503;
fastcgi_pass phpbackend;
fastcgi_keep_conn on;

For latest 123.09beta01 or newer, edit /usr/local/nginx/conf/php.conf replace OR comment out with hash # in front of fastcgi_pass dft_php line

Code (Text):

    fastcgi_keep_conn on;
    fastcgi_pass dft_php;
    #fastcgi_pass   127.0.0.1:9000;
    #fastcgi_pass   unix:/tmp/php5-fpm.sock;

and replace with or add

Code (Text):

    fastcgi_keep_conn on;
    fastcgi_next_upstream error timeout http_500 http_503;
    fastcgi_pass phpbackend;
    #fastcgi_pass dft_php;
    #fastcgi_pass   127.0.0.1:9000;
    #fastcgi_pass   unix:/tmp/php5-fpm.sock;

Individual PHP Include Files Usage

You can also configure additional PHP include files other than /usr/local/nginx/conf/php.conf default one which is included in each Nginx vhost. Additional PHP include files are located at /usr/local/nginx/conf/php-poolX.conf where X is pool number. Each php-poolX.conf has fastcgi_pass of it's own port at 127.0.0.1:9002, 127.0.0.1:9003, 127.0.0.1:9004, 127.0.0.1:9005 respectively for pool2, pool3, pool4, and pool5

Code:

ls -lah /usr/local/nginx/conf/ | grep php-
-rw-r--r--  1 root root 3.2K Jun 26 05:55 php-pool2.conf
-rw-r--r--  1 root root 3.2K Jun 26 05:34 php-pool3.conf
-rw-r--r--  1 root root 3.2K Jun 26 05:34 php-pool4.conf
-rw-r--r--  1 root root 3.2K Jun 26 05:34 php-pool5.conf

So you have the flexibility of using different PHP-FPM pools for different tasks or parts of your web site and can edit their timeouts and other buffers for specific tasks i.e. a dedicated pool to /admin backend may want longer timeout values for admin like tasks.

Code:

   location /admin {
     auth_basic "Private";
     auth_basic_user_file /usr/local/nginx/conf/htpasswd_admin;
        #include /usr/local/nginx/conf/php.conf;
        # use php-pool2.conf include file and pool2 on 127.0.0.1:9002 to serve /admin requests
        include /usr/local/nginx/conf/php-pool2.conf;
        allow 127.0.0.1;
        #allow YOURIPADDRESS;
        deny all;
}

PHP Status Pages

The default PHP Status Page is disabled by default and included via an include file in main hostname's Nginx vhost at /usr/local/nginx/conf/conf.d/virtual.conf. You can enable it for the main PHP-FPM pool www following instructions at PHP PHP-FPM - Centmin Mod - Menu based Nginx installer for CentOS servers It's the same also for adding the PHP status pages for multiple PHP-FPM pools too.

uncomment removing has for include file phpstatus.conf

Code:

#include /usr/local/nginx/conf/phpstatus.conf;

add additional php status location contexts to /usr/local/nginx/conf/phpstatus.conf and changing YOURIPADDRESS to your ipaddress

Code:

            location ~ ^/(phpstatus-pool2|phpping-pool2)$ {
                access_log          off;
                fastcgi_pass        127.0.0.1:9002;
                fastcgi_param     SCRIPT_FILENAME $fastcgi_script_name;
                include          fastcgi_params;
                allow              127.0.0.1;
                #allow YOURIPADDRESS;
                deny                all;
            }

            location ~ ^/(phpstatus-pool3|phpping-pool3)$ {
                access_log          off;
                fastcgi_pass        127.0.0.1:9003;
                fastcgi_param     SCRIPT_FILENAME $fastcgi_script_name;
                include          fastcgi_params;
                allow              127.0.0.1;
                #allow YOURIPADDRESS;
                deny                all;
            }

            location ~ ^/(phpstatus-pool4|phpping-pool4)$ {
                access_log          off;
                fastcgi_pass        127.0.0.1:9004;
                fastcgi_param     SCRIPT_FILENAME $fastcgi_script_name;
                include          fastcgi_params;
                allow              127.0.0.1;
                #allow YOURIPADDRESS;
                deny                all;
            }

            location ~ ^/(phpstatus-pool5|phpping-pool5)$ {
                access_log          off;
                fastcgi_pass        127.0.0.1:9005;
                fastcgi_param     SCRIPT_FILENAME $fastcgi_script_name;
                include          fastcgi_params;
                allow              127.0.0.1;
                #allow YOURIPADDRESS;
                deny                all;
            }                                    

If you can't use IP restriction, setup a password protection for each PHP status location context - you can use the same htpasswd file and user/pass details if you want.

Create the htwpaswd file at /usr/local/nginx/conf/phpstatus_htpasswd with your own USERNAME and PASSWORD

Code:

/usr/local/nginx/conf/htpasswd.sh create /usr/local/nginx/conf/phpstatus_htpasswd USERNAME PASSWORD

Code:

            location ~ ^/(phpstatus2|phpping2)$ {
                access_log          off;
                fastcgi_pass        127.0.0.1:9002;
                fastcgi_param     SCRIPT_FILENAME $fastcgi_script_name;
                include          fastcgi_params;
               auth_basic "Restricted";
               auth_basic_user_file /usr/local/nginx/conf/phpstatus_htpasswd;
            }

Then restart Nginx web server and PHP-FPM service, clear your browser cache, restart browser and check the yourmainhostname.com/phpstatus and yourmainhostname.com/phpstatus2 locations etc. You can also check via curl command

Code:

curl -I http://yourmainhostname.com/phpstatus

should return permission denied or 40x status message

Then you can view the phpstatus or phpstatus-poolX locations in browser or via curl within SSH to see PHP-FPM pool stats like below.

Code:

curl -s 127.0.0.1/phpstatus; curl -s 127.0.0.1/phpstatus-pool2; curl -s 127.0.0.1/phpstatus-pool3; curl -s 127.0.0.1/phpstatus-pool4; curl -s 127.0.0.1/phpstatus-pool5
pool:                 www
process manager:      ondemand
start time:           26/Jun/2015:18:25:38 +0000
start since:          74
accepted conn:        1
listen queue:         0
max listen queue:     0
listen queue len:     128
idle processes:       0
active processes:     1
total processes:      1
max active processes: 1
max children reached: 0
slow requests:        0
pool:                 pool2
process manager:      ondemand
start time:           26/Jun/2015:18:25:38 +0000
start since:          74
accepted conn:        1
listen queue:         0
max listen queue:     0
listen queue len:     128
idle processes:       0
active processes:     1
total processes:      1
max active processes: 1
max children reached: 0
slow requests:        0
pool:                 pool3
process manager:      ondemand
start time:           26/Jun/2015:18:25:38 +0000
start since:          74
accepted conn:        1
listen queue:         0
max listen queue:     0
listen queue len:     128
idle processes:       0
active processes:     1
total processes:      1
max active processes: 1
max children reached: 0
slow requests:        0
pool:                 pool4
process manager:      ondemand
start time:           26/Jun/2015:18:25:38 +0000
start since:          74
accepted conn:        1
listen queue:         0
max listen queue:     0
listen queue len:     128
idle processes:       0
active processes:     1
total processes:      1
max active processes: 1
max children reached: 0
slow requests:        0
pool:                 pool5
process manager:      ondemand
start time:           26/Jun/2015:18:25:38 +0000
start since:          74
accepted conn:        1
listen queue:         0
max listen queue:     0
listen queue len:     128
idle processes:       0
active processes:     1
total processes:      1
max active processes: 1
max children reached: 0
slow requests:        0

Code:

ss -nlt | grep 900
LISTEN     0      0                 127.0.0.1:9000                     *:*
LISTEN     0      0                 127.0.0.1:9002                     *:*
LISTEN     0      0                 127.0.0.1:9003                     *:*
LISTEN     0      0                 127.0.0.1:9004                     *:*
LISTEN     0      0                 127.0.0.1:9005                     *:* 

You can also tag onto /phpstatus or /phpstatus-poolX urls the arguments ?full&html for full details in html format.



Nginx Vhost Traffic Stats Module

stats courtesy of nginx vhost traffic stats module

 

Existing .08 beta03 Users

For existing .08 beta03 users who are on an older version without the required files, you need to update .08 beta03 latest code (i.e. via centmin.sh menu 23 git environment setup) and copy them over to your existing Centmin Mod server locations first.

SSH commands to copy them over - if prompted to overwrite a file, answer yes

Code:

\cp -f /usr/local/src/centminmod-123.08beta03/config/nginx/php-pool* /usr/local/nginx/conf/
cp /usr/local/src/centminmod-123.08beta03/config/nginx/phpfpmd/phpfpm_pool* /usr/local/nginx/conf/phpfpmd/

Edit: Also committed a change to .08 beta03 code to automatically check and populate these missing files at auto check and populate multiple php-fpm pool files for existing inst… · centminmod/centminmod@d5bdb75 · GitHub

 

PHP-FPM Pool Status via Nginx Lua

The multiple PHP-FPM Pools which you would of added for pool2 to pool5 in /usr/local/nginx/conf/phpstatus.conf would require individually look at /phpstatus, /phpstatus-poolX urls to see the statistics. However, with Centmin Mod .08 beta 03, Nginx Lua module and LuaJit is added and enabled by default.

This means you can do alot of fancy and creative things in Nginx Lua such as using ngx.location.capture_multi to capture the response body content of multiple urls and pass them as a single non-blocking response.

1. First create file named /usr/local/nginx/conf/php-lua.conf which will be included in main hostname's Nginx vhost at /usr/local/nginx/conf/conf.d/virtual.conf (cmd short cut = vhostconf).
2. Place the Lua code in /usr/local/nginx/conf/php-lua.conf. I am still new to Lua code so if there's a better way to do this, let me know
3. Change location /my-phppools to whatever you like to use to access it from YOURMAINHOSTNAME.COM/my-phppools and set allow IP address of your own, or use above method to password protect directory.

contents of /usr/local/nginx/conf/php-lua.conf

Code:

location /my-phppools {
    default_type 'text/plain';
    content_by_lua '
      resp1, resp2, resp3, resp4, resp5 = ngx.location.capture_multi{
          { "/phpstatus" },
          { "/phpstatus-pool2" },
          { "/phpstatus-pool3" },
          { "/phpstatus-pool4" },
          { "/phpstatus-pool5" },
        }
      if resp1.status == 200 then
      ngx.say(resp1.body)
        end
      if resp2.status == 200 then
      ngx.say(resp2.body)
        end
      if resp3.status == 200 then
      ngx.say(resp3.body)
        end
      if resp4.status == 200 then
      ngx.say(resp4.body)
        end
      if resp5.status == 200 then
      ngx.say(resp5.body)
      end             
   ';
   access_log    off;
   allow 127.0.0.1;
   #allow youripaddress;
   deny all;
}

virtual.conf partial excerpt of include file for php-lua.conf

Code:

server {
#         listen   80;
            listen   80 default_server backlog=2048 reuseport;
            server_name YOURMAINHOSTNAME.COM;
            root   html;

        access_log              /var/log/nginx/localhost.access.log     main buffer=256k flush=5m;
        error_log               /var/log/nginx/localhost.error.log      error;

# ngx_pagespeed & ngx_pagespeed handler
#include /usr/local/nginx/conf/pagespeed.conf;
#include /usr/local/nginx/conf/pagespeedhandler.conf;
#include /usr/local/nginx/conf/pagespeedstatslog.conf;

# limit_conn limit_per_ip 16;
# ssi  on;

include /usr/local/nginx/conf/php-lua.conf;

        location /nginx_status {
        stub_status on;
        access_log   off;
        allow 127.0.0.1;
        #allow youripaddress;
        deny all;
        }

Then when you access YOURMAINHOSTNAME.COM/my-phppools you will see all 5 PHP-FPM pools' status page outputted stats all on one page

Code:

pool:                 www
process manager:      ondemand
start time:           28/Jun/2015:22:13:14 +0000
start since:          1
accepted conn:        1
listen queue:         0
max listen queue:     0
listen queue len:     128
idle processes:       0
active processes:     1
total processes:      1
max active processes: 1
max children reached: 0
slow requests:        0

pool:                 pool2
process manager:      ondemand
start time:           28/Jun/2015:22:13:14 +0000
start since:          1
accepted conn:        1
listen queue:         0
max listen queue:     0
listen queue len:     128
idle processes:       0
active processes:     1
total processes:      1
max active processes: 1
max children reached: 0
slow requests:        0

pool:                 pool3
process manager:      ondemand
start time:           28/Jun/2015:22:13:14 +0000
start since:          1
accepted conn:        1
listen queue:         0
max listen queue:     0
listen queue len:     128
idle processes:       0
active processes:     1
total processes:      1
max active processes: 1
max children reached: 0
slow requests:        0

pool:                 pool4
process manager:      ondemand
start time:           28/Jun/2015:22:13:14 +0000
start since:          1
accepted conn:        1
listen queue:         0
max listen queue:     0
listen queue len:     128
idle processes:       0
active processes:     1
total processes:      1
max active processes: 1
max children reached: 0
slow requests:        0

pool:                 pool5
process manager:      ondemand
start time:           28/Jun/2015:22:13:14 +0000
start since:          1
accepted conn:        1
listen queue:         0
max listen queue:     0
listen queue len:     128
idle processes:       0
active processes:     1
total processes:      1
max active processes: 1
max children reached: 0
slow requests:        0

You can also tag onto /phpstatus or /phpstatus-poolX urls the arguments ?full for full details in non-html format. The html format doesn't come out correct in Lua as it just outputs the html source for 5 pages.

In /usr/local/nginx/conf/php-lua.conf add below the above existing code, the following

Code:

location /my-phppools-full {
        default_type 'text/plain';
    content_by_lua '
      resp1, resp2, resp3, resp4, resp5 = ngx.location.capture_multi{
          { "/phpstatus?full" },
          { "/phpstatus-pool2?full" },
          { "/phpstatus-pool3?full" },
          { "/phpstatus-pool4?full" },
          { "/phpstatus-pool5?full" },
        }
      if resp1.status == 200 then
      ngx.say(resp1.body)
          end
      if resp2.status == 200 then
      ngx.say(resp2.body)
          end
      if resp3.status == 200 then
      ngx.say(resp3.body)
          end
      if resp4.status == 200 then
      ngx.say(resp4.body)
          end
      if resp5.status == 200 then
      ngx.say(resp5.body)
      end                  
   ';
   access_log   off;
   allow 127.0.0.1;
   #allow youripaddress;
   deny all;
}

and when accessing YOURMAINHOSTNAME.COM/my-phppools-full, you will see

Code:

pool:                 www
process manager:      ondemand
start time:           30/Jun/2015:00:56:10 +0000
start since:          142
accepted conn:        2
listen queue:         0
max listen queue:     0
listen queue len:     128
idle processes:       0
active processes:     1
total processes:      1
max active processes: 1
max children reached: 0
slow requests:        0

************************
pid:                  356
state:                Running
start time:           30/Jun/2015:00:58:32 +0000
start since:          0
requests:             1
request duration:     272
request method:       GET
request URI:          /phpstatus?full
content length:       0
user:                 -
script:               -
last request cpu:     0.00
last request memory:  0

pool:                 pool2
process manager:      ondemand
start time:           30/Jun/2015:00:56:10 +0000
start since:          142
accepted conn:        2
listen queue:         0
max listen queue:     0
listen queue len:     128
idle processes:       0
active processes:     1
total processes:      1
max active processes: 1
max children reached: 0
slow requests:        0

************************
pid:                  357
state:                Running
start time:           30/Jun/2015:00:58:32 +0000
start since:          0
requests:             1
request duration:     213
request method:       GET
request URI:          /phpstatus-pool2?full
content length:       0
user:                 -
script:               -
last request cpu:     0.00
last request memory:  0

pool:                 pool3
process manager:      ondemand
start time:           30/Jun/2015:00:56:10 +0000
start since:          142
accepted conn:        2
listen queue:         0
max listen queue:     0
listen queue len:     128
idle processes:       0
active processes:     1
total processes:      1
max active processes: 1
max children reached: 0
slow requests:        0

************************
pid:                  358
state:                Running
start time:           30/Jun/2015:00:58:32 +0000
start since:          0
requests:             1
request duration:     202
request method:       GET
request URI:          /phpstatus-pool3?full
content length:       0
user:                 -
script:               -
last request cpu:     0.00
last request memory:  0

pool:                 pool4
process manager:      ondemand
start time:           30/Jun/2015:00:56:10 +0000
start since:          142
accepted conn:        2
listen queue:         0
max listen queue:     0
listen queue len:     128
idle processes:       0
active processes:     1
total processes:      1
max active processes: 1
max children reached: 0
slow requests:        0

************************
pid:                  359
state:                Running
start time:           30/Jun/2015:00:58:32 +0000
start since:          0
requests:             1
request duration:     202
request method:       GET
request URI:          /phpstatus-pool4?full
content length:       0
user:                 -
script:               -
last request cpu:     0.00
last request memory:  0

pool:                 pool5
process manager:      ondemand
start time:           30/Jun/2015:00:56:10 +0000
start since:          142
accepted conn:        2
listen queue:         0
max listen queue:     0
listen queue len:     128
idle processes:       0
active processes:     1
total processes:      1
max active processes: 1
max children reached: 0
slow requests:        0

************************
pid:                  360
state:                Running
start time:           30/Jun/2015:00:58:32 +0000
start since:          0
requests:             1
request duration:     206
request method:       GET
request URI:          /phpstatus-pool5?full
content length:       0
user:                 -
script:               -
last request cpu:     0.00
last request memory:  0

 

Disabling Multiple PHP-FPM Pools Globally or Individually

If you want disable extra pools in /usr/local/nginx/conf/phpfpmd/* globally server wide just edit /usr/local/etc/php-fpm.conf (cmd shortcut = fpmconf) and comment out the include line below

from

Code:

include=/usr/local/nginx/conf/phpfpmd/*.conf

to

Code:

;include=/usr/local/nginx/conf/phpfpmd/*.conf

If you want to disable an individual PHP-FPM pool in /usr/local/nginx/conf/phpfpmd/*, rename the extension for the pool filename from .conf to something else as the above include tries to match on *.conf. Then alter the nginx.conf upstreams accordingly.

Code:

upstream phpbackend {
ip_hash;
keepalive 5;
  server 127.0.0.1:9000 weight=50;
  server 127.0.0.1:9002 weight=50;
  server 127.0.0.1:9003 weight=50;
  server 127.0.0.1:9004 weight=50;
  server 127.0.0.1:9005 weight=50;
}

Note:

Do not delete the PHP-FPM pool files from /usr/local/nginx/conf/phpfpmd as they will be auto added back by Centmin Mod if they are detected as missing. So the above method for disabling is the recommended way.

 

Piwik Web Analytics with Multiple PHP-FPM Pools

For testing I setup a dedicated Linode KVM VPS server for a new Piwik Web Analytics server for my sites and enabled the above multiple PHP-FPM Pools deployment with latest Centmin Mod .08 beta 03 LEMP stack and updated to PHP 5.6.10 with Zend Opcache from default PHP 5.4.42 with Zend Opcache.



segment comparison between All (left) vs tablet+mobile+phablet vs mobile only

 

Thread is now open for replies

Folks may want to read this discussion thread too as some questions may have already been answered there Beta Branch - prep for PHP-FPM pools 2 to 5 support

 

The more cpu cores/threads the better if you are using them in upstream load balanced config. However, you can use them in non-upstream ways too i.e. dedicated to specific parts of your site with their own timeouts and buffers etc.

 

I always say test and try for yourself and see if it works for your usage patterns

 

Don't need to change php.conf then

yes just replace each site's php.conf include file with php-poolX.conf where X is the pool number

 

yes each php-poolX.conf has their own specific port auto assigned for their pools

Code (Text):

 for f in $(find /usr/local/nginx/conf/ -type f -name "php-pool*"); do echo $f; grep '127.0.0.1' $f;  done
/usr/local/nginx/conf/php-pool2.conf
    fastcgi_pass   127.0.0.1:9002;
/usr/local/nginx/conf/php-pool3.conf
    fastcgi_pass   127.0.0.1:9003;
/usr/local/nginx/conf/php-pool4.conf
    fastcgi_pass   127.0.0.1:9004;
/usr/local/nginx/conf/php-pool5.conf
    fastcgi_pass   127.0.0.1:9005;

or

Code (Text):

find /usr/local/nginx/conf/ -type f -name "php-pool*" -exec grep -H '127.0.0.1' {} \;
/usr/local/nginx/conf/php-pool4.conf:    fastcgi_pass   127.0.0.1:9004;
/usr/local/nginx/conf/php-pool3.conf:    fastcgi_pass   127.0.0.1:9003;
/usr/local/nginx/conf/php-pool5.conf:    fastcgi_pass   127.0.0.1:9005;
/usr/local/nginx/conf/php-pool2.conf:    fastcgi_pass   127.0.0.1:9002;

 

php.conf doesn't need any changes at all if you intend individual pools usage in non-load balanced way