Welcome to Centmin Mod Community
Become a Member

Nginx Nginx with Cloudflare zlib fork VS nxg_brotli compression level tests

Discussion in 'Nginx and PHP-FPM news & discussions' started by eva2000, Jan 24, 2018.

  1. eva2000

    eva2000 Administrator Staff Member

    33,733
    7,466
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,479
    Local Time:
    6:50 PM
    Nginx 1.13.x
    MariaDB 5.5
    Quick wrk load testing comparison benchmarks between Centmin Mod Nginx 1.13.8 using cloudflare zlib performance forked library for HTTPS based compressed static index file tests versus Nginx 1.13.8 with Google ngx_brotli module for brotli compression instead. First set of compression level 1-9 tests are for cloudflare zlib and second set are for ngx_brotli based. You can see ngx_brotli scales badly at high compression levels. FYI, Centmin Mod defaults to compression level 5 for both zlib and brotli usually. Though ngx_brotli has smaller file sizes than zlib/gzip based compression in Nginx.

    Tests done on OVH Core i7 4790K server with Centmin Mod compiled with GCC 8 20180114 snapshot.

    Centmin Mod Ngixn 1.13.8 + cloudflare zlib



    Code (Text):
    domain=https://domain.com
    for l in {1..9}; do
    sleep 5;
    echo -e "\ntest gzip_comp_level $l";
    sed -ie "s|BROTLI=.*|BROTLI='n'|" /usr/local/bin/curltest
    /usr/local/bin/curltest $domain;
    sed -ie "s|gzip_comp_level .*|gzip_comp_level $l;|" /usr/local/nginx/conf/nginx.conf | grep gzip_comp_level
    ngxrestart >/dev/null 2>&1;
    echo "wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: gzip' -s scripts/setup.lua --latency $domain 2>&1 | egrep -A1 'Thread Stats|Requests/sec';"
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: gzip' -s scripts/setup.lua --latency $domain 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
    done;
    sed -ie "s|gzip_comp_level .*|gzip_comp_level 5;|" /usr/local/nginx/conf/nginx.conf | grep gzip_comp_level
    


    Code (Text):
    test gzip_comp_level 1
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.53 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: gzip' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency     3.64ms    2.78ms  91.47ms   82.86%
    --
    Requests/sec:  57328.78
    Transfer/sec:    117.00MB
    
    test gzip_comp_level 2
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.60 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: gzip' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency     3.72ms    2.80ms  89.40ms   80.65%
    --
    Requests/sec:  56372.51
    Transfer/sec:    114.14MB
    
    test gzip_comp_level 3
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.59 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: gzip' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency     3.68ms    2.31ms  23.55ms   76.99%
    --
    Requests/sec:  55660.12
    Transfer/sec:    112.11MB
    
    test gzip_comp_level 4
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.58 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: gzip' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency     4.14ms    2.51ms  40.27ms   80.86%
    --
    Requests/sec:  49335.99
    Transfer/sec:     97.63MB
    
    test gzip_comp_level 5
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.54 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: gzip' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency     4.38ms    2.88ms  28.23ms   75.97%
    --
    Requests/sec:  46941.20
    Transfer/sec:     92.31MB
    
    test gzip_comp_level 6
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.53 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: gzip' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency     4.47ms    3.63ms 109.23ms   87.82%
    --
    Requests/sec:  46594.92
    Transfer/sec:     91.67MB
    
    test gzip_comp_level 7
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.53 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: gzip' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency     4.45ms    2.74ms  34.57ms   79.50%
    --
    Requests/sec:  46192.65
    Transfer/sec:     90.88MB
    
    test gzip_comp_level 8
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.53 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: gzip' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency     4.50ms    2.88ms  30.34ms   75.30%
    --
    Requests/sec:  45833.67
    Transfer/sec:     90.18MB
    
    test gzip_comp_level 9
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.53 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: gzip' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency     4.37ms    2.76ms  31.79ms   75.67%
    --
    Requests/sec:  46975.52
    Transfer/sec:     92.42MB
    
     
  2. eva2000

    eva2000 Administrator Staff Member

    33,733
    7,466
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,479
    Local Time:
    6:50 PM
    Nginx 1.13.x
    MariaDB 5.5

    Centmin Mod Ngixn 1.13.8 ngx_brotli



    Code (Text):
    domain=https://domain.com
    for l in {1..9}; do
    sleep 5;
    echo -e "\ntest brotli_comp_level $l";
    sed -ie "s|BROTLI=.*|BROTLI='y'|" /usr/local/bin/curltest
    /usr/local/bin/curltest $domain;
    sed -ie "s|brotli_comp_level .*|brotli_comp_level $l;|" /usr/local/nginx/conf/brotli_inc.conf | grep brotli_comp_level
    ngxrestart >/dev/null 2>&1;
    echo "wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: br' -s scripts/setup.lua --latency $domain 2>&1 | egrep -A1 'Thread Stats|Requests/sec';"
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: br' -s scripts/setup.lua --latency $domain 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
    done;
    sed -ie "s|brotli_comp_level .*|brotli_comp_level 5;|" /usr/local/nginx/conf/brotli_inc.conf | grep brotli_comp_level
    

    Code (Text):
    test brotli_comp_level 1
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.27 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: br' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency     3.10ms    2.36ms  21.26ms   80.09%
    --
    Requests/sec:  69216.96
    Transfer/sec:    149.85MB
    
    test brotli_comp_level 2
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.73 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: br' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency     4.15ms    3.62ms 109.56ms   88.92%
    --
    Requests/sec:  51881.04
    Transfer/sec:    106.28MB
    
    test brotli_comp_level 3
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.61 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: br' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency     4.89ms    2.91ms  38.35ms   81.21%
    --
    Requests/sec:  41836.26
    Transfer/sec:     84.74MB
    
    test brotli_comp_level 4
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.59 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: br' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency     6.12ms    3.29ms  47.08ms   81.60%
    --
    Requests/sec:  33020.04
    Transfer/sec:     59.58MB
    
    test brotli_comp_level 5
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.36 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: br' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency     6.99ms    3.24ms  31.60ms   75.34%
    --
    Requests/sec:  28453.59
    Transfer/sec:     48.84MB
    
    test brotli_comp_level 6
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.27 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: br' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency     7.18ms    3.40ms  70.95ms   84.31%
    --
    Requests/sec:  27902.55
    Transfer/sec:     47.90MB
    
    test brotli_comp_level 7
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.27 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: br' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency     7.48ms    2.79ms  29.54ms   77.60%
    --
    Requests/sec:  26428.62
    Transfer/sec:     45.24MB
    
    test brotli_comp_level 8
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.27 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: br' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency     7.53ms    2.56ms  36.67ms   82.18%
    --
    Requests/sec:  26221.07
    Transfer/sec:     44.89MB
    
    test brotli_comp_level 9
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.27 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: br' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency    21.16ms    7.13ms  69.63ms   58.05%
    --
    Requests/sec:   9278.82
    Transfer/sec:     15.87MB
    
     
  3. eva2000

    eva2000 Administrator Staff Member

    33,733
    7,466
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,479
    Local Time:
    6:50 PM
    Nginx 1.13.x
    MariaDB 5.5
    Specifically comparing levels 5 & 6 compression for Cloudflare zlib and Google ngx_brotli

    Cloudflare zlib
    Code (Text):
    test gzip_comp_level 5
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.54 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: gzip' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
       Latency     4.38ms    2.88ms  28.23ms   75.97%
    --
    Requests/sec:  46941.20
    Transfer/sec:     92.31MB
    
    test gzip_comp_level 6
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.53 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: gzip' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
       Latency     4.47ms    3.63ms 109.23ms   87.82%
    --
    Requests/sec:  46594.92
    Transfer/sec:     91.67MB
    

    Google ngx_brotli
    Code (Text):
    test brotli_comp_level 5
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.36 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: br' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
       Latency     6.99ms    3.24ms  31.60ms   75.34%
    --
    Requests/sec:  28453.59
    Transfer/sec:     48.84MB
    
    test brotli_comp_level 6
    URI: https://domain.com
    Uncompressed size : 3.71 KiB
    Compressed size   : 1.27 KiB
    wrk-cmm -t3 -c200 -d5s --breakout -H 'Accept-Encoding: br' -s scripts/setup.lua --latency https://domain.com 2>&1 | egrep -A1 'Thread Stats|Requests/sec';
      Thread Stats   Avg      Stdev     Max   +/- Stdev
       Latency     7.18ms    3.40ms  70.95ms   84.31%
    --
    Requests/sec:  27902.55
    Transfer/sec:     47.90MB
    
     
  4. RoldanLT

    RoldanLT Well-Known Member

    4,157
    1,007
    113
    May 25, 2014
    Phillipines
    Ratings:
    +1,421
    Local Time:
    4:50 PM
    1.11
    10.2
    So it's faster to deliver to each visitor.
    But how much difference we are talking?
     
  5. eva2000

    eva2000 Administrator Staff Member

    33,733
    7,466
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,479
    Local Time:
    6:50 PM
    Nginx 1.13.x
    MariaDB 5.5
    see the latency numbers in above benchmarks, probably not enough to notice really though throughput has zlib winning
     
  6. RoldanLT

    RoldanLT Well-Known Member

    4,157
    1,007
    113
    May 25, 2014
    Phillipines
    Ratings:
    +1,421
    Local Time:
    4:50 PM
    1.11
    10.2
    But Cloudflare itself uses brotli, so it's worth using I think.
     
  7. eva2000

    eva2000 Administrator Staff Member

    33,733
    7,466
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,479
    Local Time:
    6:50 PM
    Nginx 1.13.x
    MariaDB 5.5
    most like they they take backend brotli compressed assets and store them temporarily so using brotli pre-compressed served files which don't add as much overhead as on the fly brotli compression. Just like pre-gzipped static files vs on the fly gzip compressed files. So yes maybe for Cloudflare enabled Brotli compression, then it would be fine.
     
..