Welcome to Centmin Mod Community
Register Now

Beta Branch Centmin Mod .08 beta03+ Multiple PHP-FPM pools support added

Discussion in 'Beta release code' started by eva2000, Jun 27, 2015.

  1. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    1:03 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    Setting Up Multiple PHP-FPM Pools in Centmin Mod .08 beta 03+ and higher



    Centmin Mod .08 beta 03+ and higher has added very basic multiple PHP-FPM pool support. It's tuning and optimisation is left up to you. However, the tip is you can only really properly utilise all PHP-FPM pools in an upstream if you have enough cpu cores/threads to support the cummulative number pm.max_children values across the main www pool + the 4 additional pool2 to pool5 that you set in each phpfpm_poolX.conf file. Otherwise, you risk overloading your server. Nevertheless, you can use multiple PHP-FPM pools in other non-upstream ways too such as dedicated PHP-FPM pool for specific parts of your web site with their own timeout and other PHP-FPM settings (see below).

    Notes:



    PHP-FPM Pool Config Files

    Rename the new php-fpm_poolX.conf-disabled files to php-fpm_poolX.conf at /usr/local/nginx/conf/phpfpmd. Update, no longer need to rename them as they default to without -disabled endings now.

    Each phpfpm_poolX.conf file respectively has a listen = 127.0.0.1:9002 through to 127.0.0.1:9005, their own slowlog, pm.status_path, ping.path, own php-fpm-poolX.sock path (commented out), own error_log path at /var/log/php-fpm/www-php.error-poolX.log and named the pools as pool2, pool3, pool4, and pool5.

    Code:
    ls -lah /usr/local/nginx/conf/phpfpmd
    total 24K
    drwxr-xr-x 2 root root 4.0K Jun 26 05:57 .
    drwxr-xr-x 5 root root 4.0K Jun 26 05:34 ..
    -rw-r--r-- 1 root root 1.8K Jun 26 05:55 phpfpm_pool2.conf
    -rw-r--r-- 1 root root 1.7K Jun 26 05:35 phpfpm_pool3.conf
    -rw-r--r-- 1 root root 1.7K Jun 26 05:35 phpfpm_pool4.conf
    -rw-r--r-- 1 root root 1.7K Jun 26 05:35 phpfpm_pool5.conf
    Then edit /usr/local/etc/php-fpm.conf and uncomment and remove semi-colon ; from include line

    from
    Code:
    ;include=/usr/local/nginx/conf/phpfpmd/*.conf
    to
    Code:
    include=/usr/local/nginx/conf/phpfpmd/*.conf
    Restart Nginx and PHP-FPM

    Nginx Upstreams

    Setup upstreams as per Module ngx_http_upstream_module in your /usr/local/nginx/conf/nginx.conf file within http { context location


    so below these directives
    Code:
    http {
     map_hash_bucket_size 128;
     map_hash_max_size 4096;
     server_names_hash_bucket_size 128;
     server_names_hash_max_size 2048;
     variables_hash_max_size 2048;
    
    add
    Code:
    upstream phpbackend {
    ip_hash;
    keepalive 5;
      server 127.0.0.1:9000 weight=50;
      server 127.0.0.1:9002 weight=50;
      server 127.0.0.1:9003 weight=50;
      server 127.0.0.1:9004 weight=50;
      server 127.0.0.1:9005 weight=50;
    }
    For 123.09beta01 or older, edit /usr/local/nginx/conf/php.conf replace OR comment out with hash # in front
    Code:
    fastcgi_pass   127.0.0.1:9000;
    
    and replace with or add
    Code:
    fastcgi_next_upstream error timeout http_500 http_503;
    fastcgi_pass phpbackend;
    fastcgi_keep_conn on;
    
    For latest 123.09beta01 or newer, edit /usr/local/nginx/conf/php.conf replace OR comment out with hash # in front of fastcgi_pass dft_php line
    Code (Text):
        fastcgi_keep_conn on;
        fastcgi_pass dft_php;
        #fastcgi_pass   127.0.0.1:9000;
        #fastcgi_pass   unix:/tmp/php5-fpm.sock;
    

    and replace with or add
    Code (Text):
        fastcgi_keep_conn on;
        fastcgi_next_upstream error timeout http_500 http_503;
        fastcgi_pass phpbackend;
        #fastcgi_pass dft_php;
        #fastcgi_pass   127.0.0.1:9000;
        #fastcgi_pass   unix:/tmp/php5-fpm.sock;
    


    Individual PHP Include Files Usage

    You can also configure additional PHP include files other than /usr/local/nginx/conf/php.conf default one which is included in each Nginx vhost. Additional PHP include files are located at /usr/local/nginx/conf/php-poolX.conf where X is pool number. Each php-poolX.conf has fastcgi_pass of it's own port at 127.0.0.1:9002, 127.0.0.1:9003, 127.0.0.1:9004, 127.0.0.1:9005 respectively for pool2, pool3, pool4, and pool5

    Code:
    ls -lah /usr/local/nginx/conf/ | grep php-
    -rw-r--r--  1 root root 3.2K Jun 26 05:55 php-pool2.conf
    -rw-r--r--  1 root root 3.2K Jun 26 05:34 php-pool3.conf
    -rw-r--r--  1 root root 3.2K Jun 26 05:34 php-pool4.conf
    -rw-r--r--  1 root root 3.2K Jun 26 05:34 php-pool5.conf
    So you have the flexibility of using different PHP-FPM pools for different tasks or parts of your web site and can edit their timeouts and other buffers for specific tasks i.e. a dedicated pool to /admin backend may want longer timeout values for admin like tasks.

    Code:
       location /admin {
         auth_basic "Private";
         auth_basic_user_file /usr/local/nginx/conf/htpasswd_admin;
            #include /usr/local/nginx/conf/php.conf;
            # use php-pool2.conf include file and pool2 on 127.0.0.1:9002 to serve /admin requests
            include /usr/local/nginx/conf/php-pool2.conf;
            allow 127.0.0.1;
            #allow YOURIPADDRESS;
            deny all;
    }
    PHP Status Pages

    The default PHP Status Page is disabled by default and included via an include file in main hostname's Nginx vhost at /usr/local/nginx/conf/conf.d/virtual.conf. You can enable it for the main PHP-FPM pool www following instructions at PHP PHP-FPM - Centmin Mod - Menu based Nginx installer for CentOS servers It's the same also for adding the PHP status pages for multiple PHP-FPM pools too.

    uncomment removing has for include file phpstatus.conf
    Code:
    #include /usr/local/nginx/conf/phpstatus.conf;
    add additional php status location contexts to /usr/local/nginx/conf/phpstatus.conf and changing YOURIPADDRESS to your ipaddress

    Code:
                location ~ ^/(phpstatus-pool2|phpping-pool2)$ {
                    access_log          off;
                    fastcgi_pass        127.0.0.1:9002;
                    fastcgi_param     SCRIPT_FILENAME $fastcgi_script_name;
                    include          fastcgi_params;
                    allow              127.0.0.1;
                    #allow YOURIPADDRESS;
                    deny                all;
                }
    
                location ~ ^/(phpstatus-pool3|phpping-pool3)$ {
                    access_log          off;
                    fastcgi_pass        127.0.0.1:9003;
                    fastcgi_param     SCRIPT_FILENAME $fastcgi_script_name;
                    include          fastcgi_params;
                    allow              127.0.0.1;
                    #allow YOURIPADDRESS;
                    deny                all;
                }
    
                location ~ ^/(phpstatus-pool4|phpping-pool4)$ {
                    access_log          off;
                    fastcgi_pass        127.0.0.1:9004;
                    fastcgi_param     SCRIPT_FILENAME $fastcgi_script_name;
                    include          fastcgi_params;
                    allow              127.0.0.1;
                    #allow YOURIPADDRESS;
                    deny                all;
                }
    
                location ~ ^/(phpstatus-pool5|phpping-pool5)$ {
                    access_log          off;
                    fastcgi_pass        127.0.0.1:9005;
                    fastcgi_param     SCRIPT_FILENAME $fastcgi_script_name;
                    include          fastcgi_params;
                    allow              127.0.0.1;
                    #allow YOURIPADDRESS;
                    deny                all;
                }                                    
    If you can't use IP restriction, setup a password protection for each PHP status location context - you can use the same htpasswd file and user/pass details if you want.

    Create the htwpaswd file at /usr/local/nginx/conf/phpstatus_htpasswd with your own USERNAME and PASSWORD

    Code:
    /usr/local/nginx/conf/htpasswd.sh create /usr/local/nginx/conf/phpstatus_htpasswd USERNAME PASSWORD
    Code:
                location ~ ^/(phpstatus2|phpping2)$ {
                    access_log          off;
                    fastcgi_pass        127.0.0.1:9002;
                    fastcgi_param     SCRIPT_FILENAME $fastcgi_script_name;
                    include          fastcgi_params;
                   auth_basic "Restricted";
                   auth_basic_user_file /usr/local/nginx/conf/phpstatus_htpasswd;
                }
    Then restart Nginx web server and PHP-FPM service, clear your browser cache, restart browser and check the yourmainhostname.com/phpstatus and yourmainhostname.com/phpstatus2 locations etc. You can also check via curl command

    Code:
    curl -I http://yourmainhostname.com/phpstatus
    should return permission denied or 40x status message

    Then you can view the phpstatus or phpstatus-poolX locations in browser or via curl within SSH to see PHP-FPM pool stats like below.

    Code:
    curl -s 127.0.0.1/phpstatus; curl -s 127.0.0.1/phpstatus-pool2; curl -s 127.0.0.1/phpstatus-pool3; curl -s 127.0.0.1/phpstatus-pool4; curl -s 127.0.0.1/phpstatus-pool5
    pool:                 www
    process manager:      ondemand
    start time:           26/Jun/2015:18:25:38 +0000
    start since:          74
    accepted conn:        1
    listen queue:         0
    max listen queue:     0
    listen queue len:     128
    idle processes:       0
    active processes:     1
    total processes:      1
    max active processes: 1
    max children reached: 0
    slow requests:        0
    pool:                 pool2
    process manager:      ondemand
    start time:           26/Jun/2015:18:25:38 +0000
    start since:          74
    accepted conn:        1
    listen queue:         0
    max listen queue:     0
    listen queue len:     128
    idle processes:       0
    active processes:     1
    total processes:      1
    max active processes: 1
    max children reached: 0
    slow requests:        0
    pool:                 pool3
    process manager:      ondemand
    start time:           26/Jun/2015:18:25:38 +0000
    start since:          74
    accepted conn:        1
    listen queue:         0
    max listen queue:     0
    listen queue len:     128
    idle processes:       0
    active processes:     1
    total processes:      1
    max active processes: 1
    max children reached: 0
    slow requests:        0
    pool:                 pool4
    process manager:      ondemand
    start time:           26/Jun/2015:18:25:38 +0000
    start since:          74
    accepted conn:        1
    listen queue:         0
    max listen queue:     0
    listen queue len:     128
    idle processes:       0
    active processes:     1
    total processes:      1
    max active processes: 1
    max children reached: 0
    slow requests:        0
    pool:                 pool5
    process manager:      ondemand
    start time:           26/Jun/2015:18:25:38 +0000
    start since:          74
    accepted conn:        1
    listen queue:         0
    max listen queue:     0
    listen queue len:     128
    idle processes:       0
    active processes:     1
    total processes:      1
    max active processes: 1
    max children reached: 0
    slow requests:        0
    Code:
    ss -nlt | grep 900
    LISTEN     0      0                 127.0.0.1:9000                     *:*
    LISTEN     0      0                 127.0.0.1:9002                     *:*
    LISTEN     0      0                 127.0.0.1:9003                     *:*
    LISTEN     0      0                 127.0.0.1:9004                     *:*
    LISTEN     0      0                 127.0.0.1:9005                     *:* 
    You can also tag onto /phpstatus or /phpstatus-poolX urls the arguments ?full&html for full details in html format.

    phpstatus_full_html_00.png

    Nginx Vhost Traffic Stats Module

    stats courtesy of nginx vhost traffic stats module

    vhost-stats-phppools_00.png
     
    Last edited: Nov 11, 2016
  2. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    1:03 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    Existing .08 beta03 Users



    For existing .08 beta03 users who are on an older version without the required files, you need to update .08 beta03 latest code (i.e. via centmin.sh menu 23 git environment setup) and copy them over to your existing Centmin Mod server locations first.

    SSH commands to copy them over - if prompted to overwrite a file, answer yes
    Code:
    \cp -f /usr/local/src/centminmod-123.08beta03/config/nginx/php-pool* /usr/local/nginx/conf/
    cp /usr/local/src/centminmod-123.08beta03/config/nginx/phpfpmd/phpfpm_pool* /usr/local/nginx/conf/phpfpmd/
    Edit: Also committed a change to .08 beta03 code to automatically check and populate these missing files at auto check and populate multiple php-fpm pool files for existing inst… · centminmod/centminmod@d5bdb75 · GitHub
     
    Last edited: Jun 27, 2015
  3. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    1:03 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    PHP-FPM Pool Status via Nginx Lua



    The multiple PHP-FPM Pools which you would of added for pool2 to pool5 in /usr/local/nginx/conf/phpstatus.conf would require individually look at /phpstatus, /phpstatus-poolX urls to see the statistics. However, with Centmin Mod .08 beta 03, Nginx Lua module and LuaJit is added and enabled by default.

    This means you can do alot of fancy and creative things in Nginx Lua such as using ngx.location.capture_multi to capture the response body content of multiple urls and pass them as a single non-blocking response.

    1. First create file named /usr/local/nginx/conf/php-lua.conf which will be included in main hostname's Nginx vhost at /usr/local/nginx/conf/conf.d/virtual.conf (cmd short cut = vhostconf).
    2. Place the Lua code in /usr/local/nginx/conf/php-lua.conf. I am still new to Lua code so if there's a better way to do this, let me know :)
    3. Change location /my-phppools to whatever you like to use to access it from YOURMAINHOSTNAME.COM/my-phppools and set allow IP address of your own, or use above method to password protect directory.
    contents of /usr/local/nginx/conf/php-lua.conf
    Code:
    location /my-phppools {
        default_type 'text/plain';
        content_by_lua '
          resp1, resp2, resp3, resp4, resp5 = ngx.location.capture_multi{
              { "/phpstatus" },
              { "/phpstatus-pool2" },
              { "/phpstatus-pool3" },
              { "/phpstatus-pool4" },
              { "/phpstatus-pool5" },
            }
          if resp1.status == 200 then
          ngx.say(resp1.body)
            end
          if resp2.status == 200 then
          ngx.say(resp2.body)
            end
          if resp3.status == 200 then
          ngx.say(resp3.body)
            end
          if resp4.status == 200 then
          ngx.say(resp4.body)
            end
          if resp5.status == 200 then
          ngx.say(resp5.body)
          end             
       ';
       access_log    off;
       allow 127.0.0.1;
       #allow youripaddress;
       deny all;
    }
    virtual.conf partial excerpt of include file for php-lua.conf
    Code:
    server {
    #         listen   80;
                listen   80 default_server backlog=2048 reuseport;
                server_name YOURMAINHOSTNAME.COM;
                root   html;
    
            access_log              /var/log/nginx/localhost.access.log     main buffer=256k flush=5m;
            error_log               /var/log/nginx/localhost.error.log      error;
    
    # ngx_pagespeed & ngx_pagespeed handler
    #include /usr/local/nginx/conf/pagespeed.conf;
    #include /usr/local/nginx/conf/pagespeedhandler.conf;
    #include /usr/local/nginx/conf/pagespeedstatslog.conf;
    
    # limit_conn limit_per_ip 16;
    # ssi  on;
    
    include /usr/local/nginx/conf/php-lua.conf;
    
            location /nginx_status {
            stub_status on;
            access_log   off;
            allow 127.0.0.1;
            #allow youripaddress;
            deny all;
            }
    
    Then when you access YOURMAINHOSTNAME.COM/my-phppools you will see all 5 PHP-FPM pools' status page outputted stats all on one page :)

    Code:
    pool:                 www
    process manager:      ondemand
    start time:           28/Jun/2015:22:13:14 +0000
    start since:          1
    accepted conn:        1
    listen queue:         0
    max listen queue:     0
    listen queue len:     128
    idle processes:       0
    active processes:     1
    total processes:      1
    max active processes: 1
    max children reached: 0
    slow requests:        0
    
    pool:                 pool2
    process manager:      ondemand
    start time:           28/Jun/2015:22:13:14 +0000
    start since:          1
    accepted conn:        1
    listen queue:         0
    max listen queue:     0
    listen queue len:     128
    idle processes:       0
    active processes:     1
    total processes:      1
    max active processes: 1
    max children reached: 0
    slow requests:        0
    
    pool:                 pool3
    process manager:      ondemand
    start time:           28/Jun/2015:22:13:14 +0000
    start since:          1
    accepted conn:        1
    listen queue:         0
    max listen queue:     0
    listen queue len:     128
    idle processes:       0
    active processes:     1
    total processes:      1
    max active processes: 1
    max children reached: 0
    slow requests:        0
    
    pool:                 pool4
    process manager:      ondemand
    start time:           28/Jun/2015:22:13:14 +0000
    start since:          1
    accepted conn:        1
    listen queue:         0
    max listen queue:     0
    listen queue len:     128
    idle processes:       0
    active processes:     1
    total processes:      1
    max active processes: 1
    max children reached: 0
    slow requests:        0
    
    pool:                 pool5
    process manager:      ondemand
    start time:           28/Jun/2015:22:13:14 +0000
    start since:          1
    accepted conn:        1
    listen queue:         0
    max listen queue:     0
    listen queue len:     128
    idle processes:       0
    active processes:     1
    total processes:      1
    max active processes: 1
    max children reached: 0
    slow requests:        0
    
    You can also tag onto /phpstatus or /phpstatus-poolX urls the arguments ?full for full details in non-html format. The html format doesn't come out correct in Lua as it just outputs the html source for 5 pages.

    In /usr/local/nginx/conf/php-lua.conf add below the above existing code, the following
    Code:
    location /my-phppools-full {
            default_type 'text/plain';
        content_by_lua '
          resp1, resp2, resp3, resp4, resp5 = ngx.location.capture_multi{
              { "/phpstatus?full" },
              { "/phpstatus-pool2?full" },
              { "/phpstatus-pool3?full" },
              { "/phpstatus-pool4?full" },
              { "/phpstatus-pool5?full" },
            }
          if resp1.status == 200 then
          ngx.say(resp1.body)
              end
          if resp2.status == 200 then
          ngx.say(resp2.body)
              end
          if resp3.status == 200 then
          ngx.say(resp3.body)
              end
          if resp4.status == 200 then
          ngx.say(resp4.body)
              end
          if resp5.status == 200 then
          ngx.say(resp5.body)
          end                  
       ';
       access_log   off;
       allow 127.0.0.1;
       #allow youripaddress;
       deny all;
    }
    and when accessing YOURMAINHOSTNAME.COM/my-phppools-full, you will see
    Code:
    pool:                 www
    process manager:      ondemand
    start time:           30/Jun/2015:00:56:10 +0000
    start since:          142
    accepted conn:        2
    listen queue:         0
    max listen queue:     0
    listen queue len:     128
    idle processes:       0
    active processes:     1
    total processes:      1
    max active processes: 1
    max children reached: 0
    slow requests:        0
    
    ************************
    pid:                  356
    state:                Running
    start time:           30/Jun/2015:00:58:32 +0000
    start since:          0
    requests:             1
    request duration:     272
    request method:       GET
    request URI:          /phpstatus?full
    content length:       0
    user:                 -
    script:               -
    last request cpu:     0.00
    last request memory:  0
    
    pool:                 pool2
    process manager:      ondemand
    start time:           30/Jun/2015:00:56:10 +0000
    start since:          142
    accepted conn:        2
    listen queue:         0
    max listen queue:     0
    listen queue len:     128
    idle processes:       0
    active processes:     1
    total processes:      1
    max active processes: 1
    max children reached: 0
    slow requests:        0
    
    ************************
    pid:                  357
    state:                Running
    start time:           30/Jun/2015:00:58:32 +0000
    start since:          0
    requests:             1
    request duration:     213
    request method:       GET
    request URI:          /phpstatus-pool2?full
    content length:       0
    user:                 -
    script:               -
    last request cpu:     0.00
    last request memory:  0
    
    pool:                 pool3
    process manager:      ondemand
    start time:           30/Jun/2015:00:56:10 +0000
    start since:          142
    accepted conn:        2
    listen queue:         0
    max listen queue:     0
    listen queue len:     128
    idle processes:       0
    active processes:     1
    total processes:      1
    max active processes: 1
    max children reached: 0
    slow requests:        0
    
    ************************
    pid:                  358
    state:                Running
    start time:           30/Jun/2015:00:58:32 +0000
    start since:          0
    requests:             1
    request duration:     202
    request method:       GET
    request URI:          /phpstatus-pool3?full
    content length:       0
    user:                 -
    script:               -
    last request cpu:     0.00
    last request memory:  0
    
    pool:                 pool4
    process manager:      ondemand
    start time:           30/Jun/2015:00:56:10 +0000
    start since:          142
    accepted conn:        2
    listen queue:         0
    max listen queue:     0
    listen queue len:     128
    idle processes:       0
    active processes:     1
    total processes:      1
    max active processes: 1
    max children reached: 0
    slow requests:        0
    
    ************************
    pid:                  359
    state:                Running
    start time:           30/Jun/2015:00:58:32 +0000
    start since:          0
    requests:             1
    request duration:     202
    request method:       GET
    request URI:          /phpstatus-pool4?full
    content length:       0
    user:                 -
    script:               -
    last request cpu:     0.00
    last request memory:  0
    
    pool:                 pool5
    process manager:      ondemand
    start time:           30/Jun/2015:00:56:10 +0000
    start since:          142
    accepted conn:        2
    listen queue:         0
    max listen queue:     0
    listen queue len:     128
    idle processes:       0
    active processes:     1
    total processes:      1
    max active processes: 1
    max children reached: 0
    slow requests:        0
    
    ************************
    pid:                  360
    state:                Running
    start time:           30/Jun/2015:00:58:32 +0000
    start since:          0
    requests:             1
    request duration:     206
    request method:       GET
    request URI:          /phpstatus-pool5?full
    content length:       0
    user:                 -
    script:               -
    last request cpu:     0.00
    last request memory:  0
    
     
    Last edited: Jul 1, 2015
  4. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    1:03 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    Disabling Multiple PHP-FPM Pools Globally or Individually



    If you want disable extra pools in /usr/local/nginx/conf/phpfpmd/* globally server wide just edit /usr/local/etc/php-fpm.conf (cmd shortcut = fpmconf) and comment out the include line below

    from
    Code:
    include=/usr/local/nginx/conf/phpfpmd/*.conf
    
    to
    Code:
    ;include=/usr/local/nginx/conf/phpfpmd/*.conf
    
    If you want to disable an individual PHP-FPM pool in /usr/local/nginx/conf/phpfpmd/*, rename the extension for the pool filename from .conf to something else as the above include tries to match on *.conf. Then alter the nginx.conf upstreams accordingly.

    Code:
    upstream phpbackend {
    ip_hash;
    keepalive 5;
      server 127.0.0.1:9000 weight=50;
      server 127.0.0.1:9002 weight=50;
      server 127.0.0.1:9003 weight=50;
      server 127.0.0.1:9004 weight=50;
      server 127.0.0.1:9005 weight=50;
    }

    Note:



    Do not delete the PHP-FPM pool files from /usr/local/nginx/conf/phpfpmd as they will be auto added back by Centmin Mod if they are detected as missing. So the above method for disabling is the recommended way.
     
    Last edited: Jun 29, 2015
  5. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    1:03 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    Piwik Web Analytics with Multiple PHP-FPM Pools



    For testing I setup a dedicated Linode KVM VPS server for a new Piwik Web Analytics server for my sites and enabled the above multiple PHP-FPM Pools deployment with latest Centmin Mod .08 beta 03 LEMP stack and updated to PHP 5.6.10 with Zend Opcache from default PHP 5.4.42 with Zend Opcache.

    piwik_vhost_traffic_00.png

    segment comparison between All (left) vs tablet+mobile+phablet vs mobile only

    piwik_segment_compare_00.png
    piwik_realtime_map_00.png
    nodequery_00.png
    nodequery_01.png
     
  6. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    1:03 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Last edited: Jun 30, 2015
  7. Jemekite

    Jemekite New Member

    12
    5
    3
    Jun 10, 2015
    Indonesia
    Ratings:
    +9
    Local Time:
    10:03 PM
    1.9.2
    MariaDB 10
    what is minimal requirement for running multiple pool?
     
  8. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    1:03 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    The more cpu cores/threads the better if you are using them in upstream load balanced config. However, you can use them in non-upstream ways too i.e. dedicated to specific parts of your site with their own timeouts and buffers etc.
     
  9. Mask

    Mask Active Member

    108
    31
    28
    Nov 10, 2014
    Ratings:
    +37
    Local Time:
    8:03 PM
    Nginx 1.9.1
    MariaDB 10.0.19
    @eva2000

    This is just pure love :)
    Awesome thing to add.

    PS. Don't you think it be a good idea to run multiple pool with tcp and unix socket??
     
  10. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    1:03 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    I always say test and try for yourself and see if it works for your usage patterns :)
     
  11. rdan

    rdan Well-Known Member

    5,443
    1,402
    113
    May 25, 2014
    Ratings:
    +2,194
    Local Time:
    11:03 PM
    Mainline
    10.2
    I'm more into Unix Socket now :).
     
  12. Mask

    Mask Active Member

    108
    31
    28
    Nov 10, 2014
    Ratings:
    +37
    Local Time:
    8:03 PM
    Nginx 1.9.1
    MariaDB 10.0.19
    Details please :)
     
  13. rdan

    rdan Well-Known Member

    5,443
    1,402
    113
    May 25, 2014
    Ratings:
    +2,194
    Local Time:
    11:03 PM
    Mainline
    10.2
  14. Mask

    Mask Active Member

    108
    31
    28
    Nov 10, 2014
    Ratings:
    +37
    Local Time:
    8:03 PM
    Nginx 1.9.1
    MariaDB 10.0.19
  15. rdan

    rdan Well-Known Member

    5,443
    1,402
    113
    May 25, 2014
    Ratings:
    +2,194
    Local Time:
    11:03 PM
    Mainline
    10.2
    Just using standalone UNIX :).
     
  16. Jimmy

    Jimmy Well-Known Member

    1,788
    390
    83
    Oct 24, 2015
    East Coast USA
    Ratings:
    +990
    Local Time:
    10:03 AM
    Question, sorry if this is a stupid question.

    Let's say someone had a server with multiple websites and wanted to use a pool for each website on the site.

    In the instructions in the OP you have people adjusting the php.conf and replacing

    Code:
    fastcgi_pass   127.0.0.1:9000;
    with

    Code:
    fastcgi_next_upstream error timeout http_500 http_503;
    fastcgi_pass phpbackend;
    fastcgi_keep_conn on;
    Would this also need to be added to each php-pool*.conf file or just the php.conf? Or should each php-pool*.conf file retain the line below (port number is changed per pool conf file)?

    Code:
    fastcgi_pass   127.0.0.1:9002;
    Also, for each site conf file. The include line at the bottom would need to be changed, correct?

    Code:
    include /usr/local/nginx/conf/php.conf;
    to

    Code:
    include /usr/local/nginx/conf/php-pool2.conf;
     
  17. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    1:03 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Don't need to change php.conf then
    yes just replace each site's php.conf include file with php-poolX.conf where X is the pool number
     
  18. Jimmy

    Jimmy Well-Known Member

    1,788
    390
    83
    Oct 24, 2015
    East Coast USA
    Ratings:
    +990
    Local Time:
    10:03 AM
    So, each php-poolX.conf would retain the

    Code:
    fastcgi_pass   127.0.0.1:9002;
    and only the php.conf would have

    Code:
    # fastcgi_pass  127.0.0.1:9000;
    fastcgi_next_upstream error timeout http_500 http_503;
    fastcgi_pass phpbackend;
    fastcgi_keep_conn on;
     
  19. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    1:03 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    yes each php-poolX.conf has their own specific port auto assigned for their pools
    Code (Text):
     for f in $(find /usr/local/nginx/conf/ -type f -name "php-pool*"); do echo $f; grep '127.0.0.1' $f;  done
    /usr/local/nginx/conf/php-pool2.conf
        fastcgi_pass   127.0.0.1:9002;
    /usr/local/nginx/conf/php-pool3.conf
        fastcgi_pass   127.0.0.1:9003;
    /usr/local/nginx/conf/php-pool4.conf
        fastcgi_pass   127.0.0.1:9004;
    /usr/local/nginx/conf/php-pool5.conf
        fastcgi_pass   127.0.0.1:9005;

    or
    Code (Text):
    find /usr/local/nginx/conf/ -type f -name "php-pool*" -exec grep -H '127.0.0.1' {} \;
    /usr/local/nginx/conf/php-pool4.conf:    fastcgi_pass   127.0.0.1:9004;
    /usr/local/nginx/conf/php-pool3.conf:    fastcgi_pass   127.0.0.1:9003;
    /usr/local/nginx/conf/php-pool5.conf:    fastcgi_pass   127.0.0.1:9005;
    /usr/local/nginx/conf/php-pool2.conf:    fastcgi_pass   127.0.0.1:9002;
    
     
  20. eva2000

    eva2000 Administrator Staff Member

    54,106
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    1:03 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    php.conf doesn't need any changes at all if you intend individual pools usage in non-load balanced way