/ Register

Learn about Centmin Mod LEMP Stack today
Register Now

SSL Caddy HTTP/2 server & benchmarks - Part 2

Discussion in 'Domains, DNS, Email & SSL Certificates' started by eva2000, Sep 16, 2017.

Tags:
Thread Status:
Not open for further replies.
Previous Thread Next Thread
Loading...
  1. Sep 16, 2017 #1
    eva2000

    eva2000 Administrator Staff Member

    53,607
    12,139
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,683
    Local Time:
    2:25 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    This is Part 2 continuation from Caddy 0.10.9 benchmarks here.

    h2load Load Testing



    nghttp2's h2load load tester is a HTTP/2 based HTTPS load testing tool which fully supports HTTP/2 unlike other benchmarking tools (siege, apache bench, wkr, etc). I am using h2load version 1.26.0-dev via my Ubuntu Docker image using native networking to remove the Docker networking overhead. Only testing against source built Caddy 0.10.9 binary as the new Caddy EULA license agreement prohibits sharing benchmark results for official Caddy provided binaries Caddy EULA section 3.1 h - benchmarking info clarification?. FYI, my custom built Caddy binaries are already faster than official Caddy provided binary.

    h2load test server:

    The h2load test server is a dedicated server from OVH BHS, Canada
    • OVH BHS Dedicated
    • Intel i7 4790K @4.0-4.40Ghz
    • 32GB RAM
    • 2x240GB SSD
    • CentOS 7.4 64bit
    • Centmin Mod 123.09beta01
    Test server for Nginx/Caddy:

    The Nginx/Caddy test server is a 2 cpu core OpenVZ VPS server from HostUS.us. So the h2load tests are testing how well Centmin Mod Nginx and Caddy handle load on a low end OpenVZ VPS server.

    • HostUS.us Washington, DC
    • OpenVZ 2 CPU Core @Intel Xeon E5-1650v3 @3.5-3.8Ghz
    • 2GB RAM
    • 50GB SSD
    • CentOS 7.4
    • CentOS 7.4 64bit
    • Centmin Mod 123.09beta01
    Code (Text):
    h2load --version
h2load nghttp2/1.26.0-DEV


    Configuration



    I setup Centmin Mod 123.09beta01 Nginx 1.13.5 HTTP/2 vhost using Letsencrypt integration and defaults to RSA 2048bit SSL certs to match Caddy's default RSA 2048bit SSL certificate issuance and to match the negotiated SSL cipher. Check negotiated SSL ciphers for h2load tests with low load 2 thread, 2 concurrent user and 2 request test. Ensuring both Centmin Mod Nginx HTTP/2 HTTPS and Caddy HTTP/2 HTTPS negotiate using same SSL ciphers = ECDHE-RSA-AES256-GCM-SHA384 and test over HTTP/2 via = Application protocol: h2

    Vhost Config Files

    For Centmin Mod Nginx 1.13.5
    Code (Text):
    curl -I -H 'Accept-Encoding: gzip' https://nginxbench.domain.com:443

    Code (Text):
    curl -I -H 'Accept-Encoding: gzip' https://nginxbench.domain.com:443
HTTP/2 200
date: Sat, 16 Sep 2017 11:19:25 GMT
content-type: text/html; charset=utf-8
last-modified: Sat, 16 Sep 2017 09:26:40 GMT
vary: Accept-Encoding
etag: W/"59bcee50-17cf"
server: nginx centminmod
x-powered-by: centminmod
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
content-encoding: gzip

    Code (Text):
    /usr/local/bin/h2load -t2 -c2 -m100 -n2 -v https://nginxbench.domain.com:443

    Code (Text):
    /usr/local/bin/h2load -t2 -c2 -m100 -n2 -v https://nginxbench.domain.com:443
starting benchmark...
spawning thread #0: 1 total client(s). 1 total requests
spawning thread #1: 1 total client(s). 1 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 100% done

finished in 134.55ms, 14.86 req/s, 92.83KB/s
requests: 2 total, 2 started, 2 done, 2 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 2 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 12.49KB (12790) total, 466B (466) headers (space savings 27.64%), 11.90KB (12190) data
                     min         max         mean         sd        +/- sd
time for request:    31.95ms     32.59ms     32.27ms       452us   100.00%
time for connect:    98.22ms    101.60ms     99.91ms      2.39ms   100.00%
time to 1st byte:   129.85ms    133.85ms    131.85ms      2.83ms   100.00%
req/s           :       7.45        7.68        7.57        0.16   100.00%


    For Caddy 0.10.9

    Code (Text):
    curl -I -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443

    Code (Text):
    curl -I -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443
HTTP/2 200
accept-ranges: bytes
content-encoding: gzip
content-type: text/html; charset=utf-8
etag: W/"owd8wg4pb"
last-modified: Sat, 16 Sep 2017 09:26:40 GMT
server: Caddy
vary: Accept-Encoding
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-powered-by: Caddy via CentminMod
x-xss-protection: 1; mode=block
content-length: 23
date: Sat, 16 Sep 2017 11:21:06 GMT

    Code (Text):
    /usr/local/bin/h2load -t2 -c2 -m100 -n2 -v https://caddybench.domain.com:443

    Code (Text):
    /usr/local/bin/h2load -t2 -c2 -m100 -n2 -v https://caddybench.domain.com:443
starting benchmark...
spawning thread #0: 1 total client(s). 1 total requests
spawning thread #1: 1 total client(s). 1 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 100% done

finished in 183.77ms, 10.88 req/s, 67.11KB/s
requests: 2 total, 2 started, 2 done, 2 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 2 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 12.33KB (12628) total, 274B (274) headers (space savings 43.15%), 11.90KB (12190) data
                     min         max         mean         sd        +/- sd
time for request:    35.46ms     63.10ms     49.28ms     19.54ms   100.00%
time for connect:   120.24ms    123.52ms    121.88ms      2.32ms   100.00%
time to 1st byte:   158.77ms    183.35ms    171.06ms     17.38ms   100.00%
req/s           :       5.45        6.29        5.87        0.59   100.00%
     
    Last edited: Sep 17, 2017
  2. Sep 16, 2017 #2
    eva2000

    eva2000 Administrator Staff Member

    53,607
    12,139
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,683
    Local Time:
    2:25 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    h2load Load Testing Results



    Now for HTTP/2 based load testing via nghttp2's h2load load testing tool.

    Did 4 sets of h2load tests are different levels of load all with h2load passing Accept-Encoding: gzip header for gzip compressed requests for:
    1. 2 threads, 500 concurrent users, 50,000 requests with max concurrent HTTP/2 streams set at 100
    2. 2 threads, 400 concurrent users, 40,000 requests with max concurrent HTTP/2 streams set at 100
    3. 2 threads, 300 concurrent users, 30,000 requests with max concurrent HTTP/2 streams set at 100
    4. 2 threads, 200 concurrent users, 10,000 requests with max concurrent HTTP/2 streams set at 100
    caddy-vs-nginx-h2load-http2-loadtests-160917-01b.png

    Centmin Mod Nginx 1.13.5 HTTP/2



    7x runs of h2load test with 2 threads, 500 concurrent users, 50,000 requests with max concurrent HTTP/2 streams set at 100 with average of 7412.84 reques/sec with 100% completion rate

    Code (Text):
    echo "CentminMod.com Nginx 1.13.5 HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c500 -m100 -n50000 -v -H 'Accept-Encoding: gzip' https://nginxbench.domain.com:443 > h2load.$i.nginx.log; cat h2load.$i.nginx.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;

    Code (Text):
    echo "CentminMod.com Nginx 1.13.5 HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c500 -m100 -n50000 -v -H 'Accept-Encoding: gzip' https://nginxbench.domain.com:443 > h2load.$i.nginx.log; cat h2load.$i.nginx.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;
CentminMod.com Nginx 1.13.5 HTTP/2 h2load stress test
8001.94 req/s 100% completed
6268.60 req/s 100% completed
7545.21 req/s 100% completed
8088.38 req/s 100% completed
7660.50 req/s 100% completed
5787.04 req/s 100% completed
8538.20 req/s 100% completed

    Caddy 0.10.9 Source Built HTTP/2



    h2load test with 2 threads, 500 concurrent users, 50,000 requests with max concurrent HTTP/2 streams set at 100 with average of 2,793.13 reques/sec with 87.36% completion rate.

    Code (Text):
    echo "Caddy 0.10.9 Source Build HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c500 -m100 -n50000 -v -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;

    Code (Text):
    echo "Caddy 0.10.9 Source Build HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c500 -m100 -n50000 -v -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443 > h2load.$i.log; cat h2load.$i.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;
Caddy 0.10.9 Source Build HTTP/2 h2load stress test
3510.35 req/s 95.6% completed
3407.05 req/s 89.8% completed
2456.65 req/s 82.496% completed
2191.76 req/s 75.6% completed
2367.59 req/s 95% completed
2540.23 req/s 90.2% completed
3078.25 req/s 82.8% completed


    Centmin Mod Nginx 1.13.5 HTTP/2



    7x runs of h2load test with 2 threads, 400 concurrent users, 40,000 requests with max concurrent HTTP/2 streams set at 100 with average of 7,315.63 reques/sec with 100% completion rate

    Code (Text):
    echo "CentminMod.com Nginx 1.13.5 HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c400 -m100 -n40000 -v -H 'Accept-Encoding: gzip' https://nginxbench.domain.com:443 > h2load.$i.nginx.log; cat h2load.$i.nginx.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;

    Code (Text):
    echo "CentminMod.com Nginx 1.13.5 HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c400 -m100 -n40000 -v -H 'Accept-Encoding: gzip' https://nginxbench.domain.com:443 > h2load.$i.nginx.log; cat h2load.$i.nginx.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;
CentminMod.com Nginx 1.13.5 HTTP/2 h2load stress test
7188.68 req/s 100% completed
7904.49 req/s 100% completed
6016.62 req/s 100% completed
6876.74 req/s 100% completed
7905.02 req/s 100% completed
7336.11 req/s 100% completed
7981.76 req/s 100% completed

    Caddy 0.10.9 Source Built HTTP/2



    h2load test with 2 threads, 400 concurrent users, 40,000 requests with max concurrent HTTP/2 streams set at 100 with average of 2,867.45 reques/sec with 91.84% completion rate.

    Code (Text):
    echo "Caddy 0.10.9 Source Build HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c400 -m100 -n40000 -v -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;

    Code (Text):
    echo "Caddy 0.10.9 Source Build HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c400 -m100 -n40000 -v -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;
Caddy 0.10.9 Source Build HTTP/2 h2load stress test
3257.08 req/s 91% completed
2831.80 req/s 91.5% completed
2176.95 req/s 89.25% completed
3172.83 req/s 98% completed
3214.47 req/s 97.5% completed
2730.83 req/s 79.61% completed
2688.22 req/s 96% completed


    Centmin Mod Nginx 1.13.5 HTTP/2



    7x runs of h2load test with 2 threads, 300 concurrent users, 30,000 requests with max concurrent HTTP/2 streams set at 100 with average of 7,174.14 reques/sec with 100% completion rate

    Code (Text):
    echo "CentminMod.com Nginx 1.13.5 HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c300 -m100 -n30000 -v -H 'Accept-Encoding: gzip' https://nginxbench.domain.com:443 > h2load.$i.nginx.log; cat h2load.$i.nginx.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;

    Code (Text):
    echo "CentminMod.com Nginx 1.13.5 HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c300 -m100 -n30000 -v -H 'Accept-Encoding: gzip' https://nginxbench.domain.com:443 > h2load.$i.nginx.log; cat h2load.$i.nginx.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;
CentminMod.com Nginx 1.13.5 HTTP/2 h2load stress test
7651.89 req/s 100% completed
7913.35 req/s 100% completed
5572.42 req/s 100% completed
6957.35 req/s 100% completed
7260.92 req/s 100% completed
7536.88 req/s 100% completed
7326.19 req/s 100% completed

    Caddy 0.10.9 Source Built HTTP/2



    h2load test with 2 threads, 300 concurrent users, 30,000 requests with max concurrent HTTP/2 streams set at 100 with average of 3,335.63 reques/sec with 92.61% completion rate.

    Code (Text):
    echo "Caddy 0.10.9 Source Build HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c300 -m100 -n30000 -v -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;

    Code (Text):
    echo "Caddy 0.10.9 Source Build HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c300 -m100 -n30000 -v -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;
Caddy 0.10.9 Source Build HTTP/2 h2load stress test
3266.61 req/s 90.3333% completed
3300.61 req/s 97.3333% completed
3564.28 req/s 96% completed
3327.11 req/s 93.6667% completed
3545.27 req/s 85.62% completed
3328.88 req/s 92% completed
3016.64 req/s 93.3333% completed


    Centmin Mod Nginx 1.13.5 HTTP/2



    7x runs of h2load test with 2 threads, 200 concurrent users, 10,000 requests with max concurrent HTTP/2 streams set at 100 with average of 4,604.68 reques/sec with 100% completion rate

    Code (Text):
    echo "CentminMod.com Nginx 1.13.5 HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -v -H 'Accept-Encoding: gzip' https://nginxbench.domain.com:443 > h2load.$i.nginx.log; cat h2load.$i.nginx.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;

    Code (Text):
    echo "CentminMod.com Nginx 1.13.5 HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -v -H 'Accept-Encoding: gzip' https://nginxbench.domain.com:443 > h2load.$i.nginx.log; cat h2load.$i.nginx.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;
CentminMod.com Nginx 1.13.5 HTTP/2 h2load stress test
4342.71 req/s 100% completed
4489.34 req/s 100% completed
4735.29 req/s 100% completed
4543.58 req/s 100% completed
4872.51 req/s 100% completed
4666.24 req/s 100% completed
4583.11 req/s 100% completed


    example log for 5th run
    Code (Text):
    cat h2load.5.nginx.log
starting benchmark...
spawning thread #0: 100 total client(s). 5000 total requests
spawning thread #1: 100 total client(s). 5000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 2.05s, 4872.51 req/s, 10.22MB/s
requests: 10000 total, 10000 started, 10000 done, 10000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 10000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 20.97MB (21989800) total, 2.19MB (2300000) headers (space savings 25.32%), 18.60MB (19500000) data
                     min         max         mean         sd        +/- sd
time for request:    32.02ms    752.52ms    384.73ms    137.39ms    65.07%
time for connect:    99.23ms       1.79s    812.66ms    623.51ms    37.50%
time to 1st byte:   131.28ms       1.84s    964.01ms    581.92ms    51.00%
req/s           :      24.38       79.40       46.59       20.69    53.50%


    Caddy 0.10.9 Source Built HTTP/2



    h2load test with 2 threads, 200 concurrent users, 10,000 requests with max concurrent HTTP/2 streams set at 100 with average of 2,786.89 reques/sec with 100% completion rate.

    Code (Text):
    echo "Caddy 0.10.9 Source Build HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -v -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;

    Code (Text):
    echo "Caddy 0.10.9 Source Build HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -v -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;
Caddy 0.10.9 Source Build HTTP/2 h2load stress test
2736.89 req/s 100% completed
2700.29 req/s 100% completed
2441.27 req/s 100% completed
2993.02 req/s 100% completed
3244.75 req/s 100% completed
2439.18 req/s 100% completed
2952.80 req/s 100% completed


    example log for 5th run
    Code (Text):
    cat h2load.5.caddy.log
starting benchmark...
spawning thread #0: 100 total client(s). 5000 total requests
spawning thread #1: 100 total client(s). 5000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 3.08s, 3244.75 req/s, 6.13MB/s
requests: 10000 total, 10000 started, 10000 done, 10000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 10000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 18.89MB (19812113) total, 176.87KB (181113) headers (space savings 94.66%), 18.54MB (19440000) data
                     min         max         mean         sd        +/- sd
time for request:   125.10ms       2.37s       1.05s    337.91ms    64.39%
time for connect:   133.30ms       2.69s       1.02s    719.40ms    69.00%
time to 1st byte:   282.48ms       2.95s       1.74s    795.92ms    63.00%
req/s           :      16.23       50.30       25.20        9.63    75.50%


    Summary



    caddy-vs-nginx-h2load-http2-loadtests-160917-01b.png

    Nginx completed all h2load HTTP/2 HTTPS tests with 100% completion while at higher concurrency and number of requests, it seems Caddy choked with incomplete requests. Could be related to OpenVZ VPS being tested against as you can't do any TCP level tuning like you can on non-OpenVZ VPS/Servers.

    Doing a manual run against Caddy to see what's going on.
    Code (Text):
    /usr/local/bin/h2load -t2 -c500 -m100 -n50000 -v -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443

    h2load Results
    Code (Text):
    /usr/local/bin/h2load -t2 -c500 -m100 -n50000 -v -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443
starting benchmark...
spawning thread #0: 250 total client(s). 25000 total requests
spawning thread #1: 250 total client(s). 25000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done

finished in 13.97s, 3384.84 req/s, 6.39MB/s
requests: 50000 total, 50000 started, 47300 done, 47300 succeeded, 2700 failed, 2700 errored, 0 timeout
status codes: 47300 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 89.26MB (93592801) total, 745.06KB (762944) headers (space savings 95.24%), 87.69MB (91951200) data
                     min         max         mean         sd        +/- sd
time for request:    73.14ms       8.27s       2.88s       1.93s    64.71%
time for connect:   138.59ms      13.35s       5.50s       4.12s    56.60%
time to 1st byte:   327.98ms      13.60s       7.28s       3.76s    60.68%
req/s           :       0.00       70.44       11.48        7.22    86.60%

    From requests line, doesn't seem that any requests timed out though, they just failed.
    Code (Text):
    requests: 50000 total, 50000 started, 47300 done, 47300 succeeded, 2700 failed, 2700 errored, 0 timeout
     
    Last edited: Sep 17, 2017
  3. Sep 17, 2017 #3
    eva2000

    eva2000 Administrator Staff Member

    53,607
    12,139
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,683
    Local Time:
    2:25 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    Caddy & Nginx Vhost Creation



    For creating Caddy vhost config file, used my caddy.sh script which is in development for Caddy integration into Centmin Mod eventually. It runs Caddy on default HTTP port 8888 and HTTPS on port 448 self-signed SSL cert for testing, so need to manually change it to HTTP 80 and HTTPS 443 for testing after Caddy vhost is created and restart Caddy for it to issue a valid Letsencrypt SSL certificate. The caddy.sh script can auto generate both Caddy vhost + Centmin Mod Nginx vhost config files for the same domain - allowing users to switch between Nginx and Caddy web servers. With planned integrations for OpenLiteSpeed, Litespeed, Apache 2.4 and Caddy/H2O web servers with existing Nginx web servers, the end goal is to be able auto generate one domain Vhost with has equivalent vhosts for each respective web server - sharing the same the same web roots. This will allow you to switch between web servers or concurrently run different web servers, once Haproxy load balancer integration is added.

    output snippet from caddy.sh addsite command
    Code (Text):
    ./caddy.sh addsite

Add a new Caddy site domain vhost
Enter domain name wihtout www.: caddybench.domain.com

Caddy integration into Centmin Mod LEMP runs Nginx on port 80 & 443
Caddy server runs on 8888 & 448
Do you want to create the Nginx vhost for caddybench.domain.com as well ? [y/n]: y
Create self-signed SSL cert for Nginx vhost caddybench.domain.com ? recommended [y/n]: y

-------------------------------------------------------------
ok: /usr/local/src/centminmod/addons/acmetool.sh
/usr/local/src/centminmod/addons/acmetool.sh issue caddybench.domain.com


    Using Centmin Mod 123.09beta01 which has optional Letsencrypt integration which is disabled by default. To enable Letsencrypt integration, set in /etc/centminmod/custom_config.inc following settings to enable letsencrypt detectioion.
    Code (Text):
    LETSENCRYPT_DETECT='y'

    Install or update underlying acme.sh client via addons/acmetool.sh
    Code (Text):
    /usr/local/src/centminmod/addons/acmetool.sh acmeupdate

    Then create Centmin Mod Nginx HTTP/2 based HTTPS default site with free Letsencrypt SSL certificate using addons/acmetool.sh via /usr/bin/nv command line setting your own desired Pure-FTPD virtual ftp username i.e. DESIREDFTPUSERNAME
    Code (Text):
    nv

Usage: /bin/nv [-d yourdomain.com] [-s y|n|yd|le|led|lelive|lelived] [-u ftpusername]

  -d  yourdomain.com or subdomain.yourdomain.com
  -s  ssl self-signed create = y or n or https only vhost = yd
  -s  le - letsencrypt test cert or led test cert with https default
  -s  lelive - letsencrypt live cert or lelived live cert with https default
  -u  your FTP username

  example:

  /bin/nv -d yourdomain.com -s y -u ftpusername
  /bin/nv -d yourdomain.com -s n -u ftpusername
  /bin/nv -d yourdomain.com -s yd -u ftpusername
  /bin/nv -d yourdomain.com -s le -u ftpusername
  /bin/nv -d yourdomain.com -s led -u ftpusername
  /bin/nv -d yourdomain.com -s lelive -u ftpusername
  /bin/nv -d yourdomain.com -s lelived -u ftpusername

    So SSH command to create a Nginx HTTP/2 HTTPS default vhost site with Letsencrypt SSL certificate is as follows:
    Code (Text):
    nv -d nginxbench.domain.com -s lelived -u DESIREDFTPUSERNAME

    output snippet shows all the auto generated config files and letsencrypt SSL issued certificate
    Code (Text):
    -------------------------------------------------------------
FTP hostname : ipaddress
FTP port : 21
FTP mode : FTP (explicit SSL)
FTP Passive (PASV) : ensure is checked/enabled
FTP username created for nginxbench.domain.com : DESIREDFTPUSERNAME
FTP password created for nginxbench.domain.com : ****
-------------------------------------------------------------
vhost for nginxbench.domain.com created successfully

domain: http://nginxbench.domain.com
vhost conf file for nginxbench.domain.com created: /usr/local/nginx/conf/conf.d/nginxbench.domain.com.conf

vhost ssl for nginxbench.domain.com created successfully

domain: https://nginxbench.domain.com
vhost ssl conf file for nginxbench.domain.com created: /usr/local/nginx/conf/conf.d/nginxbench.domain.com.ssl.conf
/usr/local/nginx/conf/ssl/nginxbench.domain.com/nginxbench.domain.com.crt.key.conf created
/usr/local/nginx/conf/ssl_include.conf created
Self-signed SSL Certificate: /usr/local/nginx/conf/ssl/nginxbench.domain.com/nginxbench.domain.com.crt
SSL Private Key: /usr/local/nginx/conf/ssl/nginxbench.domain.com/nginxbench.domain.com.key
SSL CSR File: /usr/local/nginx/conf/ssl/nginxbench.domain.com/nginxbench.domain.com.csr
Backup SSL Private Key: /usr/local/nginx/conf/ssl/nginxbench.domain.com/nginxbench.domain.com-backup.key
Backup SSL CSR File: /usr/local/nginx/conf/ssl/nginxbench.domain.com/nginxbench.domain.com-backup.csr

upload files to /home/nginx/domains/nginxbench.domain.com/public
vhost log files directory is /home/nginx/domains/nginxbench.domain.com/log

-------------------------------------------------------------
Current vhost listing at: /usr/local/nginx/conf/conf.d/
                     
Sep 12  10:59   1.1K   demodomain.com.conf
Sep 15  17:53   1.6K   virtual.conf
Sep 16  09:28   3.4K   nginxbench.domain.com.ssl.conf

-------------------------------------------------------------
Current vhost ssl files listing at: /usr/local/nginx/conf/ssl/nginxbench.domain.com

                     
Sep 16  09:26   1.7K   nginxbench.domain.com.key
Sep 16  09:26   997    nginxbench.domain.com.csr
Sep 16  09:26   1.2K   nginxbench.domain.com.crt
Sep 16  09:26   1.7K   nginxbench.domain.com-backup.key
Sep 16  09:26   997    nginxbench.domain.com-backup.csr
Sep 16  09:27   45     hpkp-info-primary-pin.txt
Sep 16  09:27   45     hpkp-info-secondary-pin.txt
Sep 16  09:27   424    dhparam.pem
Sep 16  09:28   408    nginxbench.domain.com.crt.key.conf
Sep 16  09:28   408    acme-vhost-config.txt
Sep 16  09:28   3.2K   nginxbench.domain.com-acme.cer
Sep 16  09:28   302    nginxbench.domain.com-acme.key
Sep 16  09:28   3.2K   nginxbench.domain.com-fullchain-acme.key

-------------------------------------------------------------
Commands to remove nginxbench.domain.com

 pure-pw userdel DESIREDFTPUSERNAME
 rm -rf /usr/local/nginx/conf/conf.d/nginxbench.domain.com.conf
 rm -rf /usr/local/nginx/conf/conf.d/nginxbench.domain.com.ssl.conf
 rm -rf /usr/local/nginx/conf/ssl/nginxbench.domain.com/nginxbench.domain.com.crt
 rm -rf /usr/local/nginx/conf/ssl/nginxbench.domain.com/nginxbench.domain.com.key
 rm -rf /usr/local/nginx/conf/ssl/nginxbench.domain.com/nginxbench.domain.com.csr
 rm -rf /usr/local/nginx/conf/ssl/nginxbench.domain.com
 rm -rf /home/nginx/domains/nginxbench.domain.com
 rm -rf /root/.acme.sh/nginxbench.domain.com
 service nginx restart

-------------------------------------------------------------
vhost for nginxbench.domain.com setup successfully
nginxbench.domain.com setup info log saved at:
/root/centminlogs/centminmod_160917-092640_nginx_addvhost_nv.log
-------------------------------------------------------------


    Vhost Config Files
     
  4. Sep 17, 2017 #4
    eva2000

    eva2000 Administrator Staff Member

    53,607
    12,139
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,683
    Local Time:
    2:25 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    HTTP/2 HTTPS Header Compression Testing



    Doing more HTTP/2 HTTPS tests for Centmin Mod Nginx and Caddy source compiled binaries. Not all HTTP/2 feature sets are equal.
    • Caddy has full HTTP/2 HPACK encoding / header compression support, while Nginx out of box only has partial HTTP/2 HPACK encoding support. So over subsequent HTTP/2 HTTPS requests, Caddy will have more header size savings due to higher levels of compression compared to Nginx. However, Cloudflare folks have open sourced a Nginx patch to enable full HTTP/2 HPACK encoding support in Nginx. You can see Cloudflare's blog article about the benefits here.
    • Caddy has HTTP/2 Server Push support, while Nginx doesn't have HTTP/2 Server Push officially as yet.
    Centmin Mod 123.09beta01's Nginx server has optional support to enable the Cloudflare HTTP/2 HPACK full encoding support patch. So let's see how h2load tests fair when both Centmin Mod Nginx and Caddy are using full HTTP/2 HPACK encoding support :)

    If you look carefully, h2load tester also reports the amount of header savings due to HTTP/2 HPACK encoding support / header compression. For Nginx defaults it's usually around 20-25% due to partial support. Here, you see headers (space savings 25.32%). With full HTTP/2 HPACK encoding support, header savings should be around 90-97% - getting closer to 99% as you handle more HTTP/2 HTTPS requests.
    Code (Text):
    /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -v -H 'Accept-Encoding: gzip' https://nginxbench.domain.com:443

starting benchmark...
spawning thread #0: 100 total client(s). 5000 total requests
spawning thread #1: 100 total client(s). 5000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 2.05s, 4872.51 req/s, 10.22MB/s
requests: 10000 total, 10000 started, 10000 done, 10000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 10000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 20.97MB (21989800) total, 2.19MB (2300000) headers (space savings 25.32%), 18.60MB (19500000) data
                    min         max         mean         sd        +/- sd
time for request:    32.02ms    752.52ms    384.73ms    137.39ms    65.07%
time for connect:    99.23ms       1.79s    812.66ms    623.51ms    37.50%
time to 1st byte:   131.28ms       1.84s    964.01ms    581.92ms    51.00%
req/s           :      24.38       79.40       46.59       20.69    53.50%


    Before Cloudflare patch, Nginx config is compiled against LibreSSL 2.5.5 (which actually is less performant than OpenSSL 1.0.2/1.1), so will do tests against OpenSSL as well later.
    Then enable Cloudflare patch, setting NGINX_HPACK='y' in persistent config file at /etc/centminmod/custom_config.inc and then running centmin.sh menu option 4 to recompile Nginx.

    After Cloudflare patch applied, Nginx config - notice --with-http_v2_hpack_enc indicating that Cloudflare's HTTP/2 HPACK full encoding patch has been added.
    retesting h2load testing and notice the header space saving is now increased from ~25% to 94.83%
    Code (Text):
    /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -v -H 'Accept-Encoding: gzip' https://nginxbench.domain.com:443

starting benchmark...
spawning thread #0: 100 total client(s). 5000 total requests
spawning thread #1: 100 total client(s). 5000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 2.14s, 4676.12 req/s, 8.85MB/s
requests: 10000 total, 10000 started, 10000 done, 10000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 10000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 18.93MB (19849000) total, 155.47KB (159200) headers (space savings 94.83%), 18.60MB (19500000) data
                     min         max         mean         sd        +/- sd
time for request:    32.39ms    571.10ms    237.62ms    115.17ms    69.44%
time for connect:   105.86ms       1.90s       1.00s    564.89ms    59.00%
time to 1st byte:   143.36ms       1.98s       1.09s    540.39ms    57.00%
req/s           :      23.39       83.86       44.78       19.24    61.50%


    Let's switch Centmin Mod Nginx from using LibreSSL 2.5.5 to OpenSSL 1.1.0f setting LIBRESSL_SWITCH='n' in persistent config file at /etc/centminmod/custom_config.inc so the persistent config file now looks like this
    Code (Text):
    NGINX_HPACK='y'
LIBRESSL_SWITCH='n'
LETSENCRYPT_DETECT='y'

    Run centmin.sh menu option 4 to recompile Nginx with OpenSSL 1.1.0f
    retesting h2load testing with header space saving is still around 94.83% though req/s is down a bit.
    Code (Text):
    /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -v -H 'Accept-Encoding: gzip' https://nginxbench.domain.com:443

starting benchmark...
spawning thread #0: 100 total client(s). 5000 total requests
spawning thread #1: 100 total client(s). 5000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 2.18s, 4593.27 req/s, 8.69MB/s
requests: 10000 total, 10000 started, 10000 done, 10000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 10000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 18.93MB (19849000) total, 155.47KB (159200) headers (space savings 94.83%), 18.60MB (19500000) data
                     min         max         mean         sd        +/- sd
time for request:    32.48ms    574.35ms    296.40ms    115.78ms    67.03%
time for connect:   103.28ms       1.93s    942.41ms    575.89ms    50.00%
time to 1st byte:   138.22ms       2.05s       1.06s    571.18ms    56.50%
req/s           :      22.98       96.53       47.19       22.95    72.00%


    Centmin Mod Nginx was compiled using Clang 3.4.2 compiler, but the VPS is using Intel Xeon E5-1650v3 2 cpu cores so may lend well to using a newer compiler for the new cpu. So switching Nginx compile from Clang 3.4.2 to GCC 6.3.1 instead setting in persistent config file at /etc/centminmod/custom_config.inc additional 3 options at bottom to switch from Clang to GCC compiler and use GCC 6.3.1 via devtoolset-6. Centmin Mod 123.09beta01 can compile Nginx via additional persistent config file options to support Clang 3.4, 4.0.1, 5.0 and GCC native 4.4/4.8, 5.3, 6.3 and 7.1.
    Code (Text):
    NGINX_HPACK='y'
LIBRESSL_SWITCH='n'
LETSENCRYPT_DETECT='y'

DEVTOOLSETSIX='y'
NGINX_DEVTOOLSETGCC='y'
CLANG='n'

    Run centmin.sh menu option 4 to recompile Nginx with OpenSSL 1.1.0f + GCC 6.3.1 + Cloudflare HTTP/2 HPACK Full encoding patch.
    retesting h2load testing with header space saving is still around 94.83% though req/s is up a bit at 4,796.58 req/s
    Code (Text):
    /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -v -H 'Accept-Encoding: gzip' https://nginxbench.domain.com:443

starting benchmark...
spawning thread #0: 100 total client(s). 5000 total requests
spawning thread #1: 100 total client(s). 5000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 2.08s, 4796.58 req/s, 9.08MB/s
requests: 10000 total, 10000 started, 10000 done, 10000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 10000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 18.93MB (19849000) total, 155.47KB (159200) headers (space savings 94.83%), 18.60MB (19500000) data
                     min         max         mean         sd        +/- sd
time for request:    31.78ms    683.64ms    377.83ms    143.25ms    67.21%
time for connect:   103.95ms       1.71s    876.70ms    542.49ms    48.50%
time to 1st byte:   140.36ms       1.89s       1.04s    573.50ms    54.00%
req/s           :      24.00      100.69       46.48       23.81    81.00%

    Switching from GCC 6.3.1 to GCC 7.1 compiled Nginx is still in private testing but can be used if devtoolset-7 SCL is installed and setting in persistent config file at /etc/centminmod/custom_config.inc the following
    Code (Text):
    NGINX_HPACK='y'
LIBRESSL_SWITCH='n'
LETSENCRYPT_DETECT='y'

#DEVTOOLSETSIX='y'
DEVTOOLSETSEVEN='y'
NGINX_DEVTOOLSETGCC='y'
CLANG='n'

    Run centmin.sh menu option 4 to recompile Nginx with OpenSSL 1.1.0f + GCC 7.1 + Cloudflare HTTP/2 HPACK Full encoding patch.
    retesting h2load testing with header space saving is still around 94.83% but req/s is higher at 4929.19 req/s
    Code (Text):
    /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -v -H 'Accept-Encoding: gzip' https://nginxbench.domain.com:443

starting benchmark...
spawning thread #0: 100 total client(s). 5000 total requests
spawning thread #1: 100 total client(s). 5000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 2.03s, 4929.19 req/s, 9.33MB/s
requests: 10000 total, 10000 started, 10000 done, 10000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 10000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 18.93MB (19849000) total, 155.47KB (159200) headers (space savings 94.83%), 18.60MB (19500000) data
                     min         max         mean         sd        +/- sd
time for request:    31.72ms    794.90ms    468.84ms    170.18ms    64.76%
time for connect:    97.66ms       1.71s    778.32ms    576.91ms    56.50%
time to 1st byte:   130.49ms       1.83s    967.94ms    565.99ms    55.50%
req/s           :      24.66       73.87       42.20       16.71    67.50%


    Compared to Caddy's HTTP/2 header compression savings ~94.64%
    Code (Text):
    /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -v -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443

starting benchmark...
spawning thread #0: 100 total client(s). 5000 total requests
spawning thread #1: 100 total client(s). 5000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 3.77s, 2650.73 req/s, 5.01MB/s
requests: 10000 total, 10000 started, 10000 done, 10000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 10000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 18.89MB (19812711) total, 177.45KB (181711) headers (space savings 94.64%), 18.54MB (19440000) data
                     min         max         mean         sd        +/- sd
time for request:   378.20ms       3.40s       1.69s    635.10ms    71.59%
time for connect:   261.01ms       2.70s       1.24s    958.16ms    58.50%
time to 1st byte:   641.85ms       3.61s       2.58s    784.14ms    60.50%
req/s           :      13.26       30.88       16.27        3.41    84.50%


    Also will look into compiling Caddy binaries with Clang and GCC versions to see if there's a difference :)

    Summary

    caddy-vs-nginx-h2load-http2-loadtests-160917-02.png

    Updated:     Built Caddy custom source binary with GCC 7.1.1 instead of system GCC 4.8.5 to compare. Nice gains :)

    Code (Text):
    echo "Caddy 0.10.9 Source Build HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -v -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;   
Caddy 0.10.9 Source Build HTTP/2 h2load stress test
2836.69 req/s 100% completed
2992.75 req/s 100% completed
2751.78 req/s 100% completed
2886.93 req/s 100% completed
2996.70 req/s 100% completed
3004.66 req/s 100% completed
2474.80 req/s 99% completed


    caddy-vs-nginx-h2load-http2-loadtests-160917-03.png
     
    Last edited: Sep 18, 2017
  5. Sep 18, 2017 #5
    eva2000

    eva2000 Administrator Staff Member

    53,607
    12,139
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,683
    Local Time:
    2:25 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    Caddy http.cache Proxy Caching



    Caddy also has a http.cache plugin for proxy caching. Testing Caddy source built with GCC 7.1.1 binary with the following http.cache config
    Code (Text):
        proxy / caddybench.domain.com:80
    cache {
        match_path /
        match_header Content-Type image/jpg image/png text/html
        status_header X-Cache-Status
        default_max_age 5m
        path /tmp/caddy-cache
    }

    Caddy proxy cache miss
    Code (Text):
    curl -I https://caddybench.domain.com
HTTP/2 200
accept-ranges: bytes
content-type: text/html; charset=utf-8
date: Sun, 17 Sep 2017 23:06:08 GMT
etag: "owd8wg4pb"
last-modified: Sat, 16 Sep 2017 09:26:40 GMT
server: Caddy
server: Caddy
x-cache-status: miss
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-powered-by: Caddy via CentminMod
x-xss-protection: 1; mode=block
content-length: 6095

    Caddy proxy cache hit
    Code (Text):
    curl -I https://caddybench.domain.com
HTTP/2 200
accept-ranges: bytes
content-type: text/html; charset=utf-8
date: Sun, 17 Sep 2017 23:06:08 GMT
etag: "owd8wg4pb"
last-modified: Sat, 16 Sep 2017 09:26:40 GMT
server: Caddy
server: Caddy
x-cache-status: hit
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-powered-by: Caddy via CentminMod
x-xss-protection: 1; mode=block
content-length: 6095


    h2load test against HTTP/2 HTTPS proxy cached site
    Code (Text):
    echo "Caddy 0.10.9 Source Build HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;
Caddy 0.10.9 Source Build HTTP/2 h2load stress test

4436.63 req/s 100% completed
4838.55 req/s 100% completed
4737.82 req/s 100% completed
5068.30 req/s 100% completed
3881.18 req/s 100% completed
4568.86 req/s 100% completed
4099.06 req/s 100% completed

    h2load test against non-HTTPS HTTP/1.1 backend directly on port 80
    Code (Text):
    echo "Caddy 0.10.9 Source Build HTTP/1.1 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load --h1 -t2 -c200 -m100 -n10000 -H 'Accept-Encoding: gzip' http://caddybench.domain.com:80 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;
Caddy 0.10.9 Source Build HTTP/1.1 h2load stress test
2884.80 req/s 100% completed
3753.52 req/s 100% completed
3985.83 req/s 100% completed
4374.14 req/s 100% completed
3670.77 req/s 100% completed
3881.24 req/s 100% completed
3813.97 req/s 100% completed

    Retest Nginx 1.13.5 built with GCC 7.1.1 for 7x h2load runs = 4,956 req/s average
    Code (Text):
    echo "CentminMod.com Nginx 1.13.5 HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -H 'Accept-Encoding: gzip' https://nginxbench.domain.com:443 > h2load.$i.nginx.log; cat h2load.$i.nginx.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;
CentminMod.com Nginx 1.13.5 HTTP/2 h2load stress test
5171.32 req/s 100% completed
4654.93 req/s 100% completed
5028.61 req/s 100% completed
4695.41 req/s 100% completed
5101.92 req/s 100% completed
5039.42 req/s 100% completed
5001.25 req/s 100% completed


    Summary compared to previous results in the following table. So Caddy http.cache proxy caching improves non-HTTPS requests by ~20% and HTTP/2 HTTPS requests by 58.6%. However, even with Caddy proxy caching via http.cache plugin, Caddy is still slower than Nginx without any caching.

    caddy-vs-nginx-h2load-http2-loadtests-160917-06.png
     
  6. Sep 19, 2017 #6
    eva2000

    eva2000 Administrator Staff Member

    53,607
    12,139
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,683
    Local Time:
    2:25 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    Caddy Source Build: GCC 7.1.1 vs Clang 4.0.1



    Previously Caddy was tested with GCC 7.1.1 compiler as well. This time also built Caddy 0.10.9 binary with Clang 4.0.1 compiler to compare. From the results, Caddy built with GCC 7.1.1 was overall faster especially for non-HTTPS port 80 backend and Caddy http.cache proxy caching. While for straight Caddy HTTP/2 HTTPS tests, Caddy built with GCC 7.1.1 and Clang 4.0.1 were pretty close.

    Caddy h2load HTTP/2 test with GCC 7.1.1
    Code (Text):
    echo "Caddy 0.10.9 Source GCC 7.1.1 Build HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;

    avg: 2812 req/s
    Code (Text):
    echo "Caddy 0.10.9 Source GCC 7.1.1 Build HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;
Caddy 0.10.9 Source GCC 7.1.1 Build HTTP/2 h2load stress test
2696.12 req/s 100% completed
2906.17 req/s 100% completed
2719.78 req/s 100% completed
2718.14 req/s 100% completed
2912.49 req/s 100% completed
2736.60 req/s 100% completed
2995.76 req/s 100% completed

    Caddy h2load HTTP/2 test with Clang 4.0.1
    Code (Text):
    echo "Caddy 0.10.9 Source Clang 4.0.1 Build HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;

    avg: 2813 req/s
    Code (Text):
    echo "Caddy 0.10.9 Source Clang 4.0.1 Build HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;
Caddy 0.10.9 Source Clang 4.0.1 Build HTTP/2 h2load stress test
3148.24 req/s 100% completed
2964.38 req/s 100% completed
2836.92 req/s 100% completed
2984.65 req/s 100% completed
2347.34 req/s 100% completed
2843.19 req/s 100% completed
2569.00 req/s 100% completed

    Caddy port 80 backend with Clang 4.0.1
    Code (Text):
    echo "Caddy 0.10.9 Source Clang 4.0.1 Build HTTP/1.1 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load --h1 -t2 -c200 -m100 -n10000 -H 'Accept-Encoding: gzip' http://caddybench.domain.com:80 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;

    avg: 2543 req/s
    Code (Text):
    echo "Caddy 0.10.9 Source Clang 4.0.1 Build HTTP/1.1 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load --h1 -t2 -c200 -m100 -n10000 -H 'Accept-Encoding: gzip' http://caddybench.domain.com:80 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;
2499.96 req/s 100% completed
2995.45 req/s 100% completed
1135.85 req/s 100% completed
2985.12 req/s 100% completed
2368.40 req/s 100% completed
3091.10 req/s 100% completed
2727.03 req/s 100% completed

    Caddy HTTP/2 HTTPS http.cache proxy caching to 80 with Clang 4.0.1
    Code (Text):
    echo "Caddy 0.10.9 Source Clang 4.0.1 Build HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;

    avg: 3962 req/s
    Code (Text):
    echo "Caddy 0.10.9 Source Clang 4.0.1 Build HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;
Caddy 0.10.9 Source Clang 4.0.1 Build HTTP/2 h2load stress test
4476.26 req/s 100% completed
3759.10 req/s 100% completed
3205.71 req/s 100% completed
3684.59 req/s 100% completed
4043.06 req/s 100% completed
4655.74 req/s 100% completed
3906.90 req/s 100% completed


    caddy-vs-nginx-h2load-http2-loadtests-190917-07.png
     
  7. Sep 21, 2017 #7
    eva2000

    eva2000 Administrator Staff Member

    53,607
    12,139
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,683
    Local Time:
    2:25 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    Caddy + Varnish Cache 5.2 Sandwiched Proxies



    Testing Caddy 0.10.9 GCC 7.1.1 build binary with Varnish Cache 5.2 where Caddy HTTPS frontend acts as a SSL proxy for SSL termination (port 443) to Varnish Cache backend (port 6081) and then Varnish Cache talks to Caddy HTTP (port 80) as backend. This is following Caddy + Varnish guide setup on official forums here. It seems it worked well for users testing non-HTTP/2 HTTPS loads i.e. with wrk load testing tool. Unfortunately, for h2load HTTP/2 HTTPS load testing was pretty bad - much worse than Caddy http.cache proxy caching performance. HTTP/2 issues might be related to how Varnish Cache is handling HTTP/2 H/2 threads with these bugs:
    Varnish Cache has experimental HTTP/2 support since 5.0 but disabled by default so need to enable it.
    Code (Text):
    varnishadm param.show feature
feature
        Value is: +http2
        Default is: none

        Enable/Disable various minor features.
           none                       Disable all features.

        Use +/- prefix to enable/disable individual feature:
           short_panic                Short panic message.
           wait_silo                  Wait for persistent silo.
           no_coredump                No coredumps.
           esi_ignore_https           Treat HTTPS as HTTP in
                                      ESI:includes
           esi_disable_xml_check      Don't check of body looks like
                                      XML
           esi_ignore_other_elements  Ignore non-esi XML-elements
           esi_remove_bom             Remove UTF-8 BOM
           https_scheme               Also split https URIs
           http2                      Support HTTP/2 protocol

    Varnish Cache 5.2 was ran with following options
    Code (Text):
    DAEMON_OPTS="-p feature=+http2 -p thread_pools=2 -p thread_pool_add_delay=0.02 -p listen_depth=4096 -p lru_interval=2 -p cli_timeout=10 -p thread_pool_min=500 -p thread_pool_max=1500 -p thread_pool_timeout=300 -p tcp_fastopen=on"


    Caddy proxy to Varnish 6081 port
    Code (Text):
         proxy / localhost:6081 {
         transparent
     }

    Code (Text):
    curl -I https://caddybench.domain.com
HTTP/2 200
accept-ranges: bytes
age: 6
content-type: text/html; charset=utf-8
date: Thu, 21 Sep 2017 01:09:21 GMT
etag: W/"owd8wg4pb"
last-modified: Sat, 16 Sep 2017 09:26:40 GMT
server: Caddy
server: Caddy
vary: Accept-Encoding
via: 1.1 varnish (Varnish/5.2)
x-cache: HIT
x-cache-hits: 1
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-powered-by: Caddy via CentminMod
x-varnish: 32770 3
x-xss-protection: 1; mode=block

    Code (Text):
    echo "Caddy 0.10.9 Source GCC 7.1.1 Build HTTP/2 h2load stress test + Varnish 5.2"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;

    avg: 145.57 req/s 31.99%
    Code (Text):
    echo "Caddy 0.10.9 Source GCC 7.1.1 Build HTTP/2 h2load stress test + Varnish 5.2"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c200 -m100 -n10000 -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;
Caddy 0.10.9 Source GCC 7.1.1 Build HTTP/2 h2load stress test + Varnish 5.2
699.98 req/s 94.49% completed
166.57 req/s 46.03% completed
146.46 req/s 59.46% completed
56.99 req/s 23.93% completed
0.00 req/s 0% completed
0.00 req/s 0% completed
0.00 req/s 0% completed

    caddy-vs-nginx-h2load-http2-loadtests-varnish-210917-01.png
     
    Last edited: Sep 21, 2017
  8. Jan 28, 2018 #8
    eva2000

    eva2000 Administrator Staff Member

    53,607
    12,139
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,683
    Local Time:
    2:25 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    Nginx 1.13.8 vs Caddy 0.10.10



    Revisiting benchmarks with updated Nginx 1.13.8 compiled this time with GCC 8 compiler versus Caddy 0.10.10 compiled with GCC 4.8.5 default.

    h2load Load Testing



    nghttp2's h2load load tester is a HTTP/2 based HTTPS load testing tool which fully supports HTTP/2 unlike other benchmarking tools (siege, apache bench, wkr, etc). I am using h2load version 1.30-dev via my Ubuntu Docker image using native networking to remove the Docker networking overhead. Only testing against source built Caddy 0.10.10 binary as the new Caddy EULA license agreement prohibits sharing benchmark results for official Caddy provided binaries Caddy EULA section 3.1 h - benchmarking info clarification?. FYI, my custom built Caddy binaries are already faster than official Caddy provided binary.

    h2load test server:

    The h2load test server is a dedicated server from OVH BHS, Canada
    • OVH BHS Dedicated
    • Intel i7 4790K @4.0-4.40Ghz
    • 32GB RAM
    • 2x240GB SSD
    • CentOS 7.4 64bit
    • Centmin Mod 123.09beta01
    Test server for Nginx/Caddy:

    The Nginx/Caddy test server is a 2 cpu core OpenVZ VPS server from HostUS.us. So the h2load tests are testing how well Centmin Mod Nginx and Caddy handle load on a low end OpenVZ VPS server.

    • HostUS.us Washington, DC
    • OpenVZ 2 CPU Core @Intel Xeon E5-1650v3 @3.5-3.8Ghz
    • 2GB RAM
    • 50GB SSD
    • CentOS 7.4 64bit
    • Centmin Mod 123.09beta01
    Code (Text):
    h2load --version
h2load nghttp2/1.30.0-DEV


    Configuration



    I setup Centmin Mod 123.09beta01 Nginx 1.13.8 HTTP/2 vhost using Letsencrypt integration and defaults to RSA 2048bit SSL certs to match Caddy's default RSA 2048bit SSL certificate issuance and to match the negotiated SSL cipher. Check negotiated SSL ciphers for h2load tests with low load 2 thread, 2 concurrent user and 2 request test. Ensuring both Centmin Mod Nginx HTTP/2 HTTPS and Caddy HTTP/2 HTTPS negotiate using same SSL ciphers = ECDHE-RSA-AES256-GCM-SHA384 and test over HTTP/2 via = Application protocol: h2

    Vhost Config Files

    For Centmin Mod Nginx 1.13.8

    Code (Text):
    curl -I -H 'Accept-Encoding: gzip' https://nginxbench.domain.com:443

    Code (Text):
    curl -I -H 'Accept-Encoding: gzip' https://nginxbench.domain.com:443
HTTP/2 200
date: Sun, 28 Jan 2018 10:42:12 GMT
content-type: text/html; charset=utf-8
content-length: 6095
last-modified: Sun, 28 Jan 2018 10:15:37 GMT
vary: Accept-Encoding
etag: "5a6da2c9-17cf"
server: nginx centminmod
x-powered-by: centminmod
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
accept-ranges: bytes

    Code (Text):
    h2load -t2 -c2 -m100 -n2 -v https://nginxbench.domain.com:443
starting benchmark...
spawning thread #0: 1 total client(s). 1 total requests
spawning thread #1: 1 total client(s). 1 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
[stream_id=1] :status: 200
[stream_id=1] date: Sun, 28 Jan 2018 10:44:45 GMT
[stream_id=1] content-type: text/html; charset=utf-8
[stream_id=1] content-length: 6095
[stream_id=1] last-modified: Sun, 28 Jan 2018 10:15:37 GMT
[stream_id=1] vary: Accept-Encoding
[stream_id=1] etag: "5a6da2c9-17cf"
[stream_id=1] server: nginx centminmod
[stream_id=1] x-powered-by: centminmod
[stream_id=1] x-frame-options: SAMEORIGIN
[stream_id=1] x-xss-protection: 1; mode=block
[stream_id=1] x-content-type-options: nosniff
[stream_id=1] accept-ranges: bytes
progress: 100% done
[stream_id=1] :status: 200
[stream_id=1] date: Sun, 28 Jan 2018 10:44:45 GMT
[stream_id=1] content-type: text/html; charset=utf-8
[stream_id=1] content-length: 6095
[stream_id=1] last-modified: Sun, 28 Jan 2018 10:15:37 GMT
[stream_id=1] vary: Accept-Encoding
[stream_id=1] etag: "5a6da2c9-17cf"
[stream_id=1] server: nginx centminmod
[stream_id=1] x-powered-by: centminmod
[stream_id=1] x-frame-options: SAMEORIGIN
[stream_id=1] x-xss-protection: 1; mode=block
[stream_id=1] x-content-type-options: nosniff
[stream_id=1] accept-ranges: bytes

finished in 133.39ms, 14.99 req/s, 93.65KB/s
requests: 2 total, 2 started, 2 done, 2 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 2 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 12.49KB (12792) total, 468B (468) headers (space savings 27.33%), 11.90KB (12190) data
                     min         max         mean         sd        +/- sd
time for request:    32.60ms     32.93ms     32.77ms       227us   100.00%
time for connect:   100.23ms    100.25ms    100.24ms         9us   100.00%
time to 1st byte:   132.78ms    133.11ms    132.94ms       233us   100.00%
req/s           :       7.51        7.52        7.51        0.01   100.00%


    For Caddy 0.10.10

    Code (Text):
    caddy -version
Caddy 0.10.10 (+106d62b Sun Jan 28 06:11:52 UTC 2018) (unofficial)
1 file changed, 31 insertions(+)
caddy/caddymain/run.go

    Code (Text):
    curl -I -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443

    Code (Text):
    curl -I -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443
HTTP/2 200
accept-ranges: bytes
content-type: text/html; charset=utf-8
etag: "p39dzu4pb"
last-modified: Sun, 28 Jan 2018 09:21:30 GMT
server: Caddy
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-powered-by: Caddy via CentminMod
x-xss-protection: 1; mode=block
content-length: 6095
date: Sun, 28 Jan 2018 10:42:51 GMT

    Code (Text):
    h2load -t2 -c2 -m100 -n2 -v https://caddybench.domain.com:443  
starting benchmark...
spawning thread #0: 1 total client(s). 1 total requests
spawning thread #1: 1 total client(s). 1 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
[stream_id=1] :status: 200[stream_id=
1] [stream_id=:status1: ] accept-ranges: bytes
200
[stream_id=1] accept-ranges: bytes
[stream_id=1] content-type[stream_id=: text/html; charset=utf-8
1] content-type: text/html; charset=utf-8[stream_id=
[stream_id=11] etag: "p39dzu4pb"
] [stream_id=1] last-modified: Sun, 28 Jan 2018 09:21:30 GMT
etag[stream_id=1: ] server: Caddy
"p39dzu4pb"
[stream_id=[stream_id=1] 1last-modified] x-content-type-options: nosniff
: [stream_id=1] x-frame-options: SAMEORIGIN
Sun, 28 Jan 2018 09:21:30 GMT
[stream_id=1] server: Caddy[stream_id=1] x-powered-by: Caddy via CentminMod

[stream_id=[stream_id=1] x-content-type-options: nosniff
[stream_id=1] x-frame-options: 1SAMEORIGIN] x-xss-protection: 1; mode=block

[stream_id=1] content-length: 6095
[stream_id=1] x-powered-by: Caddy via CentminMod
[stream_id=1] date: Sun, 28 Jan 2018 10:46:29 GMT
[stream_id=1] x-xss-protection: 1; mode=block
[stream_id=1] content-length: 6095
[stream_id=1] date: Sun, 28 Jan 2018 10:46:29 GMT
progress: 100% done

finished in 157.23ms, 12.72 req/s, 79.18KB/s
requests: 2 total, 2 started, 2 done, 2 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 2 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 12.45KB (12748) total, 394B (394) headers (space savings 33.89%), 11.90KB (12190) data
                     min         max         mean         sd        +/- sd
time for request:    37.79ms     38.93ms     38.36ms       807us   100.00%
time for connect:   118.06ms    119.25ms    118.66ms       844us   100.00%
time to 1st byte:   156.82ms    156.83ms    156.82ms         3us   100.00%
req/s           :       6.37        6.37        6.37        0.00   100.00%


    h2load Load Testing Results



    Did 1 sets of h2load tests with different levels of load all with h2load passing Accept-Encoding: gzip header for gzip compressed requests for:
    1. 2 threads, 500 concurrent users, 50,000 requests with max concurrent HTTP/2 streams set at 100

    For Caddy 0.10.0



    h2load 2 threads, 500 concurrent users, 50,000 requests with max concurrent HTTP/2 streams set at 100
    • 2726.93 req/s average
    • 94.054% average completed requests
    Code (Text):
    echo "Caddy 0.10.10 Source Build HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c500 -m100 -n50000 -v -H 'Accept-Encoding: gzip' https://caddybench.domain.com:443 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;

3143.02 req/s 94.2% completed
3199.79 req/s 95.8% completed
3356.13 req/s 88.982% completed
2658.94 req/s 98.2% completed
2331.08 req/s 89.996% completed
2173.81 req/s 98.2% completed
2225.74 req/s 93% completed


    For Nginx 1.13.8



    h2load 2 threads, 500 concurrent users, 50,000 requests with max concurrent HTTP/2 streams set at 100
    • 7,297.17 req/s average
    • 100% average completed requests
    Code (Text):
    echo "Nginx 1.13.8 Source Build HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c500 -m100 -n50000 -v -H 'Accept-Encoding: gzip' https://nginxbench.domain.com:443 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;

5507.08 req/s 100% completed
8380.65 req/s 100% completed
8561.74 req/s 100% completed
7910.94 req/s 100% completed
7088.63 req/s 100% completed
5280.55 req/s 100% completed
8350.60 req/s 100% completed


    Then lets enable optional Cloudflare Zlib performance patch for Centmin Mod Nginx as outlined in guide at Insight Guide - How to boost Centmin Mod LEMP stack performance.

    h2load 2 threads, 500 concurrent users, 50,000 requests with max concurrent HTTP/2 streams set at 100
    • 7,572.18 req/s average
    • 100% average completed requests
    Code (Text):
    echo "Nginx 1.13.8 Source Build HTTP/2 h2load stress test"; sleep 30; for i in {1..7}; do /usr/local/bin/h2load -t2 -c500 -m100 -n50000 -v -H 'Accept-Encoding: gzip' https://nginxbench.domain.com:443 > h2load.$i.caddy.log; cat h2load.$i.caddy.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;

5972.79 req/s 100% completed
9203.27 req/s 100% completed
5514.54 req/s 100% completed
5638.55 req/s 100% completed
9026.86 req/s 100% completed
9215.60 req/s 100% completed
8433.63 req/s 100% completed


    Summary Table

    Looks like Cloudflare zlib patched Centmin Mod Nginx helped for higher peak requests rate but averages were just that bit higher.

    Web Server h2load -t2 -c500 -m100 -n50000 7 run avg req/s 7 run avg % completed
    Centmin Mod built Nginx 1.13.8 + Cloudflare zlib GCC 8.0 7572.18 100%
    Centmin Mod built Nginx 1.13.8 GCC 8.0 7297.17 100%
    Centmin Mod built Caddy 0.10.10 GCC 4.8.5 2726.93 94.054%
     
    Last edited: Jan 28, 2018
  9. Mar 5, 2019 #9
    eva2000

    eva2000 Administrator Staff Member

    53,607
    12,139
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,683
    Local Time:
    2:25 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
Previous Thread Next Thread
Loading...
Thread Status:
Not open for further replies.

.