Welcome to Centmin Mod Community
Become a Member

PHP-FPM WARNING: [pool www] server reached max_children setting (50), consider raising it

Discussion in 'Nginx, PHP-FPM & MariaDB MySQL' started by rdan, Jul 10, 2014.

  1. eva2000

    eva2000 Administrator Staff Member

    40,624
    9,016
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +13,883
    Local Time:
    10:24 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    just did, above was trying to quickly edit the above with name first in case you beat me to quoting it without it hehe
     
  2. Floren

    Floren Active Member

    148
    77
    28
    Jun 6, 2014
    Ratings:
    +77
    Local Time:
    8:24 AM
    @eva2000, wait you set it to run though Siege? I just test it now, nothing happened on a one time execution.
    Code:
    int(1407640255) int(1407640265)
     
  3. eva2000

    eva2000 Administrator Staff Member

    40,624
    9,016
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +13,883
    Local Time:
    10:24 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    yup in high concurrency environment simulation through siege

    all i needed was
    Code:
    siege -c50 -d30 -t1M URLTO/howdy_sleep.php
    
    to hit limits

    howdy.php
    PHP:
    <html>
    <head>
      <title>PHP test script</title>
    </head>
    <body>
    <?php echo '<p>Howdy Folks</p>';?>
    </body>
    </html>
    howdy_sleep.php
    PHP:
    <html>
    <head>
      <title>PHP test script</title>
    </head>
    <body>
    <?php echo '<p>Howdy Folks</p>'sleep(10); exit;?>
    </body>
    </html>
     
    style="display:inline-block;min-width:400px;max-width:970px;width:95%;height:90px" data-ad-client="ca-pub-6669518204467592" data-ad-slot="4024536743" data-ad-format="auto">
  4. Floren

    Floren Active Member

    148
    77
    28
    Jun 6, 2014
    Ratings:
    +77
    Local Time:
    8:24 AM
    @eva2000, let me run it through 500 users. It is insane what we are doing now, btw. :D
    There is no PHP script that will take an execution time on 10 seconds, I'm testing this:
    # siege -c 500 -d 30 -t 1M

    Yep, I got the warning:
    [09-Aug-2014 23:26:49] WARNING: [pool www] server reached pm.max_children setting (8), consider raising it

    It is an invalid test, this will never happen in real life. :)
    Code:
    # siege -c 500 -d 30 -t 1M
    ** SIEGE 3.0.5
    ** Preparing 500 concurrent users for battle.
    The server is now under siege...
    HTTP/1.1 200  10.49 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  10.48 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  10.48 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  10.48 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  10.48 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  10.48 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  10.97 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  10.97 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  19.48 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  19.48 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  19.48 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  19.48 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  19.48 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  19.48 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  19.97 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  20.97 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  29.48 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  29.48 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  29.48 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  29.48 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  29.48 secs:      41 bytes ==> GET  /time.php
    HTTP/1.1 200  29.48 secs:      41 bytes ==> GET  /time.php
    ---
    HTTP/1.1 504  29.97 secs:     565 bytes ==> GET  /time.php
    HTTP/1.1 504  29.97 secs:     565 bytes ==> GET  /time.php
    HTTP/1.1 504  29.97 secs:     565 bytes ==> GET  /time.php
    HTTP/1.1 504  29.97 secs:     565 bytes ==> GET  /time.php
    HTTP/1.1 504  29.97 secs:     565 bytes ==> GET  /time.php
    HTTP/1.1 504  29.97 secs:     565 bytes ==> GET  /time.php
    HTTP/1.1 504  29.97 secs:     565 bytes ==> GET  /time.php
    Did not take long until it busted PHP. Even if you 300,000 as setting it will die. :D
    The test also added 700MB to the used memory, LOL
     
    Last edited: Aug 10, 2014
  5. eva2000

    eva2000 Administrator Staff Member

    40,624
    9,016
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +13,883
    Local Time:
    10:24 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    less optimised web PHP apps might

    try with 1 second or 1/2 second and see as well :)

    Guess we both live for this testing stuff :D
     
  6. Floren

    Floren Active Member

    148
    77
    28
    Jun 6, 2014
    Ratings:
    +77
    Local Time:
    8:24 AM
    There is no such thing as 0.5 seconds, a PHP script takes 0.01 seconds to execute.
     
  7. eva2000

    eva2000 Administrator Staff Member

    40,624
    9,016
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +13,883
    Local Time:
    10:24 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    ah quoted my typo = More meant to say less optimised web PHP apps might heh

    for 100MB PHP file upload ? and say via 3G mobile net connection ?

    remember in context of RoldanLT's server his users are on average on very slow net connections too
     
    • Like Like x 1
  8. Floren

    Floren Active Member

    148
    77
    28
    Jun 6, 2014
    Ratings:
    +77
    Local Time:
    8:24 AM
    @eva2000, I only have 1MB uploads, go ahead and try it first. You started this, not me. :D
    I'm rebooting the server to clear the used memory you forced me to ruin it. :D
     
  9. eva2000

    eva2000 Administrator Staff Member

    40,624
    9,016
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +13,883
    Local Time:
    10:24 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    remember in context of RoldanLT's server his users are on average on very slow net connections too

    whole point of this thread :D

    But yeah I don't have such user and upload size traffic like RoldanLT does

    haha, you're welcome :)
     
  10. Floren

    Floren Active Member

    148
    77
    28
    Jun 6, 2014
    Ratings:
    +77
    Local Time:
    8:24 AM
    There nothing you can do about a slow connection, no? There are no miracles. You want me to pull my 56K modem? :D


    Also, you cannot do miracles with a small server, period.
    I did not know his site is busy? To me, it is not.... he has like 130 real users online:
    Total: 578 (members: 131, guests: 399, robots: 48)

    You don't count guests, they are cached. Last time I remember, he got this server:
    Server SYS-IP-2 - So you Start

    How powerful is that processor?
     
    Last edited: Aug 10, 2014
  11. eva2000

    eva2000 Administrator Staff Member

    40,624
    9,016
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +13,883
    Local Time:
    10:24 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    hehe yeah but slow connections are still reality - checking my Google Analytic stats from Asian visitors can clearly see still alot of folks on slower connections.
     
    • Like Like x 1
  12. eva2000

    eva2000 Administrator Staff Member

    40,624
    9,016
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +13,883
    Local Time:
    10:24 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
  13. Floren

    Floren Active Member

    148
    77
    28
    Jun 6, 2014
    Ratings:
    +77
    Local Time:
    8:24 AM
    @eva2000, here it is some useful info I found deep down on the Internet:

    Accelerated upload support
    The essence: the support of the web server Category: Optimization
    This feature, as the name implies, allows faster processing of large POST requests, including file uploads. Acceleration is achieved by ensuring that the body of the request is recorded into a temporary file and fastcgi protocol passes file path instead of the request body. At the moment, as far as I know, only nginx 0.5.9 + supports this functionality. Obviously, such a scheme will only work if php is a host with a Web server.
    Sample nginx configuration:

    Code:
    location ~\.php$ {
       fastcgi_pass_request_body off;
       client_body_in_file_only clean;
       fastcgi_param  REQUEST_BODY_FILE  $request_body_file; 
       ... 
       fastcgi_pass ...;
     }
    In php you do not need to configure anything. If php gets option REQUEST_BODY_FILE - it reads the request body of the file. If there is no option - usually read request body of a protocol fastcgi.
    tmpfs (linux): Along with this feature it makes sense to use "memory-based filesystem for temporary files, such as tmpfs (linux):

    Code:
    client_body_temp_path /dev/shm/client_body_temp;
    Are you going to benchmark this? They talk about POST also, not just uploads.
     
    • Like Like x 1
  14. eva2000

    eva2000 Administrator Staff Member

    40,624
    9,016
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +13,883
    Local Time:
    10:24 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    where did you find that nugget of info ?

    Yeah probably set aside some time to test later on, working with clients now :D
     
  15. Floren

    Floren Active Member

    148
    77
    28
    Jun 6, 2014
    Ratings:
    +77
    Local Time:
    8:24 AM
    @eva2000, I ran a 500 users comparison test on my Nginx configuration, with and without those settings. Nginx and PHP-FPM temp dirs were already on tmpfs, cache was disabled, no connection limits and the following URL's:
    Code:
    # tail -n 6 /etc/siege/urls.txt
    https://www.axivo.com/
    https://www.axivo.com/forums/
    https://www.axivo.com/members/
    https://www.axivo.com/forums/software.8/
    https://www.axivo.com/threads/notifications-of-uptream-releases.56/
    https://www.axivo.com/articles/google-ranks-ssl-sites-better.23/
    Accelerated support enabled (reboot performed, waited 10min for OS to stabilize, load 0):
    Code:
    # siege -c 500 -d 30 -t 1M
    Transactions:               1769 hits
    Availability:             100.00 %
    Elapsed time:              59.28 secs
    Data transferred:          17.46 MB
    Response time:               0.09 secs
    Transaction rate:          29.84 trans/sec
    Throughput:               0.29 MB/sec
    Concurrency:               2.62
    Successful transactions:        1769
    Failed transactions:              0
    Longest transaction:           5.16
    Shortest transaction:           0.02
    # reboot
    The server load was practically ZERO, it does not makes sense?

    Accelerated support disabled (waited 10min after reboot to let the OS stabilize, load 0):
    Code:
    top - 00:37:09 up 11 min,  2 users,  load average: 1.28, 0.57, 0.20
    
    # siege -c 500 -d 30 -t 1M
    Transactions:               1776 hits
    Availability:             100.00 %
    Elapsed time:              59.10 secs
    Data transferred:          17.38 MB
    Response time:               0.08 secs
    Transaction rate:          30.05 trans/sec
    Throughput:               0.29 MB/sec
    Concurrency:               2.55
    Successful transactions:        1776
    Failed transactions:              0
    Longest transaction:           0.85
    Shortest transaction:           0.02
    
    #reboot
    Take a look a number comparison, the winner is accelerated support disabled:
    Code:
          Date & Time,  Trans,  Elap Time,  Data Trans,  Resp Time,  Trans Rate,  Throughput,  Concurrent,    OKAY,   Failed
    2014-08-10 00:23:55,   1769,      59.28,          17,       0.09,       29.84,        0.29,        2.62,    1769,       0
    2014-08-10 00:37:09,   1776,      59.10,          17,       0.08,       30.05,        0.29,        2.55,    1776,       0
    There are 2 factors that changed during the tests: server load and longest transaction. With acceleration enabled, the server load stayed very low (0.2), idle was 75% and the longest transaction was very high (5.16 seconds). With acceleration disabled, the server load plummeted to 1.3, idle was 75% and the longest transaction was very low (0.85 seconds). During the test with acceleration enabled, I noticed several long transactions.

    What's your take on all this? The most surprising part for myself is the server load with acceleration enabled. The 6 PHP-FPM processes I saw running were using 10% CPU each for both tests. In other words: with acceleration enabled you get no server load with high transaction time (not good) and with it disabled you get a high server load but excellent transaction times.

    Personally, I'm inclined to low transaction times (acceleration disabled), since the idle was the same. Keep in mind that I performed only GET's, no POST's involved. It is clear the load I saw is related to slow disks. I don't have sar or iostat installed to take a deeper look, but the experience is telling me this.
     
    Last edited: Aug 10, 2014
    • Like Like x 1
  16. eva2000

    eva2000 Administrator Staff Member

    40,624
    9,016
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +13,883
    Local Time:
    10:24 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    interesting what was the php status output like for the 2 different configs (accelerated and not accelerated) ?

    wonder how random the siege file url.s.txt listing is as i see a difference in total data transferred sizes between the 2 configs

    maybe isolate it to a single url to see first

    edit: also guess it depends on what they mean by large POST request size, how large is large ?
     
    Last edited: Aug 10, 2014
  17. Floren

    Floren Active Member

    148
    77
    28
    Jun 6, 2014
    Ratings:
    +77
    Local Time:
    8:24 AM
    @eva2000, what do you mean by "php status output"?
    > I see a difference in total data transferred sizes.
    No, there is no difference.
     
  18. eva2000

    eva2000 Administrator Staff Member

    40,624
    9,016
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +13,883
    Local Time:
    10:24 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
  19. Floren

    Floren Active Member

    148
    77
    28
    Jun 6, 2014
    Ratings:
    +77
    Local Time:
    8:24 AM
    @eva2000, you are using the location wrong:
    location ~ ^/(phpstatus|phpping)$

    That is an invalid statement, the OR does not take effect. It is not reported as config error because it is a valid regex. Please test it and let me know.
     
    Last edited: Aug 11, 2014
  20. eva2000

    eva2000 Administrator Staff Member

    40,624
    9,016
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +13,883
    Local Time:
    10:24 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    works for me been like that for 3yrs now and all my monitor scripts use them to gather stats without problem
    Code:
     curl -S localhost/phpping
    pong
    
    Code:
    curl -S localhost/phpstatus
    pool:                 www
    process manager:      dynamic
    start time:           09/Aug/2014:21:07:11 -0700
    start since:          7075
    accepted conn:        194
    listen queue:         0
    max listen queue:     7
    listen queue len:     65535
    idle processes:       1
    active processes:     1
    total processes:      2
    max active processes: 2
    max children reached: 1
    slow requests:        0
     
    • Like Like x 1