PHP-FPM pool recommendations script

Well such a script won't work on default Centmin Mod PHP-FPM as PHP-FPM uses ondemand so ps output won't show any PHP pools if there is no active PHP requests as ondemand ends the PHP pool processes if idle for certain time to save memory.

example with default PHP-FPM set to use ondemand process manager

Code (Text):

ps aux | grep "php-fpm" | grep -v grep
root      1141  0.0  0.4 567440  9116 ?        S<s  17:22   0:01 php-fpm: master process (/usr/local/etc/php-fpm.conf)

example where PHP-FPM is switched from ondemand to static process manager so php pools and children run even at idle - uses more memory but is faster than ondemand

Code (Text):

ps aux | grep "php-fpm" | grep -v grep
root     34291  0.0  0.0 1243676 7772 ?        S<s  Jan06   0:05 php-fpm: master process (/usr/local/etc/php-fpm.conf)
nginx    34292  0.0  0.0 1328920 18848 ?       S<   Jan06   0:00 php-fpm: pool www
nginx    34293  0.0  0.0 1329136 19152 ?       S<   Jan06   0:00 php-fpm: pool www
nginx    34294  0.0  0.0 1328896 18608 ?       S<   Jan06   0:00 php-fpm: pool www
nginx    34295  0.0  0.0 1330520 17872 ?       S<   Jan06   0:00 php-fpm: pool www
nginx    34296  0.0  0.0 1331272 18668 ?       S<   Jan06   0:00 php-fpm: pool www
nginx    34297  0.0  0.0 1331088 18216 ?       S<   Jan06   0:00 php-fpm: pool www
nginx    34298  0.0  0.0 1331184 18644 ?       S<   Jan06   0:00 php-fpm: pool www
nginx    34299  0.0  0.0 1331312 18620 ?       S<   Jan06   0:00 php-fpm: pool www
nginx    34300  0.0  0.0 1330572 18040 ?       S<   Jan06   0:00 php-fpm: pool www
nginx    34301  0.0  0.0 1330532 17932 ?       S<   Jan06   0:00 php-fpm: pool www
nginx    34302  0.0  0.0 1331292 19028 ?       S<   Jan06   0:00 php-fpm: pool www
nginx    34303  0.0  0.0 1328376 17676 ?       S<   Jan06   0:00 php-fpm: pool www
nginx    34304  0.0  0.0 1328636 17868 ?       S<   Jan06   0:00 php-fpm: pool www
nginx    34305  0.0  0.0 1328952 18268 ?       S<   Jan06   0:00 php-fpm: pool www
nginx    34306  0.0  0.0 1329276 18624 ?       S<   Jan06   0:00 php-fpm: pool www
nginx    34307  0.0  0.0 1329040 18252 ?       S<   Jan06   0:00 php-fpm: pool www
nginx    34308  0.0  0.0 1328636 17896 ?       S<   Jan06   0:00 php-fpm: pool www
nginx    34309  0.0  0.0 1328472 17804 ?       S<   Jan06   0:00 php-fpm: pool www
nginx    34310  0.0  0.0 1329056 18460 ?       S<   Jan06   0:00 php-fpm: pool www
nginx    34311  0.0  0.0 1329136 18548 ?       S<   Jan06   0:00 php-fpm: pool www

Also setting number of PHP children based on available memory is inherently flawed as you do not take into account number of cpu threads available. Setting PHP children to high above cpu threads available will result in slower PHP response time latency.

Best way to work out optimal usage, is through monitoring via phpstatus output and re-tuning over time - see https://community.centminmod.com/threads/how-to-troubleshoot-optimize-php-fpm-server.15317/

 

With that said, thought it would be a nice addition to cminfo top command output so will add these 2 command lines to cminfo top output later today

Will only work if you have changed php-fpm process management from default ondemand to static management which will increase php-fpm memory usage as php-fpm children will remain running even if they're idle with no requests unlike ondemand

Code (Text):

f=$(free -wk | awk '/Mem:/ {print $8}')
smem -P 'php-fpm: pool' | egrep -v 'python|Command' | awk -v f=$f '{swap+=$6; uss+=$7; pss+=$8; rss+=$9} END {print "Free Memory (KB): "f"\n""Estimated Max PHP Children: "(f+rss)/(rss/NR) "\nPHP-FPM Total Used Memory (KB): ""swap:"swap, "uss:"uss, "pss:"pss, "rss:"rss"\n""PHP-FPM Average Per Child (KB): ""swap:"swap/NR, "uss:"uss/NR, "pss:"pss/NR, "rss:"rss/NR}'

Which output the following

Code (Text):

Free Memory (KB): 947060
Estimated Max PHP Children: 124.009
PHP-FPM Total Used Memory (KB): swap:0 uss:7044 pss:14630 rss:302224
PHP-FPM Average Per Child (KB): swap:0 uss:234.8 pss:487.667 rss:10074.1

better version

Code (Text):

f=$(free -wk | awk '/Mem:/ {print $8}')
smem -P 'php-fpm: pool' | egrep -v 'python|Command' | awk -v f=$f '{swap+=$6; uss+=$7; pss+=$8; rss+=$9} END {print "Current Free Memory (KB): "f"\n""PHP-FPM Available Memory (KB): "f+rss"\n""Estimated Max PHP Children: "(f+rss)/(rss/NR) "\nPHP-FPM Total Used Memory (KB): ""swap:"swap, "uss:"uss, "pss:"pss, "rss:"rss"\n""PHP-FPM Average Per Child (KB): ""swap:"swap/NR, "uss:"uss/NR, "pss:"pss/NR, "rss:"rss/NR}'

Current Free Memory (KB): 947060
PHP-FPM Available Memory (KB): 1244644
Estimated Max PHP Children: 125.475
PHP-FPM Total Used Memory (KB): swap:0 uss:6120 pss:13034 rss:297584
PHP-FPM Average Per Child (KB): swap:0 uss:204 pss:434.467 rss:9919.47

cminfo top output will look something like this by end of today when I update cminfo - if you have php-fpm process manager switched to static instead of default ondemand

Code (Text):

------------------------------------------------------------------
php-fpm stats

Current Free Memory (KB): 938604
PHP-FPM Available Memory (KB): 1236188
Estimated Max PHP Children: 124.622
PHP-FPM Total Used Memory (KB): swap:0 uss:6120 pss:13004 rss:297584
PHP-FPM Average Per Child (KB): swap:0 uss:204 pss:433.467 rss:9919.47
uss = user set size
pss = process set size
rss = resident set size

Processes active: 0, idle: 30, Requests: 0, slow: 0, Traffic: 0req/sec
pool:                 www
process manager:      static
start time:           12/Jan/2020:01:02:46 +0000
start since:          218
accepted conn:        2
listen queue:         0
max listen queue:     0
listen queue len:     65535
idle processes:       29
active processes:     1
total processes:      30
max active processes: 1
max children reached: 0
slow requests:        0

 

123.09beta01 has updated cminfo now see https://community.centminmod.com/threads/cminfo-command-explained.11399/#post-80834

Example of dedicated server with 32GB memory installed, this illustrates why calculating PHP-FPM max children processes by dividing total available memory by average PHP-FPM process size is flawed. Based on dedicated servers total PHP-FPM available memory divide by average PHP-FPM per child size of 13,706 KB you will end up with estimated max PHP-FPM children value of = 1965. But that is insanely high for a 4 cpu / 8 cpu thread server and result in much slower PHP response times as there's a 245:1 ratio of PHP-FPM children to cpu threads. You'd drive up cpu load insanely high with such settings !

Code (Text):

cminfo phpmem

Current Free Memory (KB): 26305692
PHP-FPM Available Memory (KB): 26936168
Estimated Max PHP Children: 1965.28
PHP-FPM Total Used Memory (KB): swap:12924 uss:125636 pss:164296 rss:630476
PHP-FPM Average Per Child (KB): swap:280.957 uss:2731.22 pss:3571.65 rss:13706
uss = user set size
pss = process set size
rss = resident set size

Updated cminfo phpmem output with PHP-FPM max children estimated to cpu thread ratio too. Below are other ridiculous examples of how flawed such calculations on their own are without taking into account how many cpu threads will service those PHP-FPM children.

On 8 cpu thread, 32GB memory server

Code (Text):

cminfo phpmem

Current Free Memory (KB): 26199316
PHP-FPM Available Memory (KB): 26440460
Estimated Max PHP Children: 3289.38
Estimated Max PHP Children To CPU Thread Ratio: 411.172
PHP-FPM Total Used Memory (KB): swap:0 uss:7820 pss:12814 rss:241144
PHP-FPM Average Per Child (KB): swap:0 uss:260.667 pss:427.133 rss:8038.13
uss = user set size
pss = process set size
rss = resident set size

On 48 cpu thread server with 256GB memory

Code (Text):

cminfo phpmem

Current Free Memory (KB): 175076956
PHP-FPM Available Memory (KB): 185749528
Estimated Max PHP Children: 2784.7
Estimated Max PHP Children To CPU Thread Ratio: 58.0146
PHP-FPM Total Used Memory (KB): swap:0 uss:7666384 pss:7685017 rss:10672572
PHP-FPM Average Per Child (KB): swap:0 uss:47914.9 pss:48031.4 rss:66703.6
uss = user set size
pss = process set size
rss = resident set size

On 24 cpu thread AMD Ryzen 3900X with 64GB memory

Code (Text):

cminfo phpmem

Current Free Memory (KB): 55084620
PHP-FPM Available Memory (KB): 55275964
Estimated Max PHP Children: 5777.65
Estimated Max PHP Children To CPU Thread Ratio: 240.736
PHP-FPM Total Used Memory (KB): swap:0 uss:4388 pss:27296 rss:191344
PHP-FPM Average Per Child (KB): swap:0 uss:219.4 pss:1364.8 rss:9567.2
uss = user set size
pss = process set size
rss = resident set size

 

Yeah cminfo command is meant to allow folks to deep dive into server usage stats etc. I've updated to add a phpstats command to cminfo for when using pm = static process manager.

• Update cminfo top and cminfo phpmem routines to also report how many PHP-FPM masters/pools & how many PHP-FPM children there are and list their PHP-FPM pool names
• Add cminfo phpstats command to combine systemd PHP-FPM status stats with phpmem output
• Again this will only display for CentOS 7 systems with pm = static process manager and not the default PHP-FPM pm = ondemand

Code (Text):

------------------------------------------------------------------
Total PHP-FPM Master Processes: 1
PHP-FPM Pool Names:
www
------------------------------------------------------------------
27335 php-fpm: master /usr/local/etc/php-fpm.conf
------------------------------------------------------------------
Current Free Memory (KB): 26181960
PHP-FPM Available Memory (KB): 26423228
Estimated Max PHP Children: 3285.54
Estimated Max PHP Children To CPU Thread Ratio: 410.693
PHP-FPM Total Children: 30 from 1 PHP-FPM master(s)
PHP-FPM Total Used Memory (KB): swap:0 uss:7824 pss:12832 rss:241268
PHP-FPM Average Per Child (KB): swap:0 uss:260.8 pss:427.733 rss:8042.27
uss = user set size
pss = process set size
rss = resident set size

Ready to handle connections
pool:                 www
process manager:      static
start time:           12/Jan/2020:15:22:22 +0000
start since:          9
accepted conn:        2
listen queue:         0
max listen queue:     0
listen queue len:     65535
idle processes:       29
active processes:     1
total processes:      30
max active processes: 1
max children reached: 0
slow requests:        0

If pm = ondemand and not pm = static, then memory info is not displayed

Code (Text):

PHP-FPM pm = ondemand in /usr/local/etc/php-fpm.conf
PHP-FPM memory usage only viewable when pm = static

Processes active: 0, idle: 0, Requests: 0, slow: 0, Traffic: 0req/sec
pool:                 www
process manager:      ondemand
start time:           12/Jan/2020:15:21:47 +0000
start since:          17
accepted conn:        2
listen queue:         0
max listen queue:     0
listen queue len:     65535
idle processes:       0
active processes:     1
total processes:      1
max active processes: 1
max children reached: 0
slow requests:        0

 

Actually it will for dynamic and static, just i was focused on ondemand differences

example with pm = dynamic

Code (Text):

cminfo phpstats
------------------------------------------------------------------
Total PHP-FPM Master Processes: 1
PHP-FPM Pool Names:
www
------------------------------------------------------------------
19717 php-fpm: master /usr/local/etc/php-fpm.conf
------------------------------------------------------------------
Current Free Memory (KB): 26195380
PHP-FPM Available Memory (KB): 26259692
Estimated Max PHP Children: 3266.54
Estimated Max PHP Children To CPU Thread Ratio: 408.317
PHP-FPM Total Children: 8 from 1 PHP-FPM master(s)
PHP-FPM Total Used Memory (KB): swap:0 uss:2084 pss:7232 rss:64312
PHP-FPM Average Per Child (KB): swap:0 uss:260.5 pss:904 rss:8039
uss = user set size
pss = process set size
rss = resident set size
Ready to handle connections
pool:                 www
process manager:      dynamic
start time:           12/Jan/2020:19:56:44 +0000
start since:          8
accepted conn:        2
listen queue:         0
max listen queue:     0
listen queue len:     65535
idle processes:       7
active processes:     1
total processes:      8
max active processes: 1
max children reached: 0
slow requests:        0

 

start a thread in this forum at https://community.centminmod.com/forums/nginx-php-fpm-mariadb-mysql.21/ with info https://community.centminmod.com/threads/how-to-troubleshoot-optimize-php-fpm-server.15317/