Beta Branch Nginx HTTP/2 & OpenSSL 1.1.0 patch updates

eva2000 · Jun 23, 2017

Nginx HTTP/2 & OpenSSL 1.1.0 patch updates

- Update inc/nginx_patch.inc for HTTP/2 full HPACK encoding optional support when using OpenSSL 1.0.2 branch not OpenSSL 1.1.x branch i.e. OPENSSL_VERSION='1.0.2l'. When Nginx 1.13.1 or newer + NGINX_PATCH='y' default + LIBRESSL_SWITCH='n' + NGINX_HPACK='y' set in persistent config file at /etc/centminmod/custom_config.inc (default is disabled), then optional HTTP/2 full HPACK encoding support for nginx is enabled as per RFC7541 specs when you run centmin.sh menu option 4 to compile Nginx. Full HTTP/2 HPACK encoding support is provided via Cloudflare patch which improves header compression ratio by 5-10% for the first response, and by 40-95% for consequential responses on the connection. Currently, not fully working with Nginx 1.13.1 so this is just prep work right now so keep NGINX_PATCH='n' set which is current default.
- Cloudflare Patch enabled full HTTP/2 HPACK encoding versus Nginx partial HTTP/2 HPACK encoding details at HPACK: the silent killer (feature) of HTTP/2
- Update patches/openssl/chacha20-smarter.patch switch to cloudflare's openssl__1.1.0_chacha20_poly1305.patch patch

Continue reading...

123.09beta01 branch

Branch: GitHub - centminmod/centminmod at 123.09beta01

Commit History: Commits · centminmod/centminmod · GitHub

eva2000 · Jun 23, 2017

Updated to support both OpenSSL 1.1.x and 1.0.2 branches now

example when Nginx 1.13.1 or newer + NGINX_PATCH='y' default

+ LIBRESSL_SWITCH='n' + NGINX_HPACK='y' set in persistent config file at /etc/centminmod/custom_config.inc
Code (Text):
patching nginx http/2 full HPACK encoding support
https://github.com/cloudflare/sslconfig/raw/hpack_1.13.1/patches/nginx_1.13.1_http2_hpack.patch
2017-06-23 10:58:33 URL:https://raw.githubusercontent.com/cloudflare/sslconfig/hpack_1.13.1/patches/nginx_1.13.1_http2_hpack.patch [34561/34561] -> "nginx_1.13.1_http2_hpack.patch" [1]
patch -p1 < nginx_1.13.1_http2_hpack.patch
patching file auto/modules
patching file auto/options
patching file src/core/ngx_murmurhash.c
patching file src/core/ngx_murmurhash.h
patching file src/http/v2/ngx_http_v2.c
patching file src/http/v2/ngx_http_v2.h
patching file src/http/v2/ngx_http_v2_filter_module.c
patching file src/http/v2/ngx_http_v2_table.c

patching nginx http/2 full HPACK encoding for nginx 1.13.1+
nginx -V
nginx version: nginx/1.13.1
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC)
built with OpenSSL 1.1.0f 25 May 2017
TLS SNI support enabled
configure arguments: --with-ld-opt='-ljemalloc -Wl,-z,relro -Wl,-rpath,/usr/local/lib' --with-cc-opt='-m64 -march=native -DTCP_FASTOPEN=23 -g -O3 -fstack-protector-strong -fuse-ld=gold --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -gsplit-dwarf' --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-compat --with-http_stub_status_module --with-http_secure_link_module --with-libatomic --with-http_gzip_static_module --with-http_sub_module --with-http_addition_module --with-http_image_filter_module=dynamic --with-http_geoip_module --with-stream_geoip_module --with-stream_realip_module --with-stream_ssl_preread_module --with-threads --with-stream=dynamic --with-stream_ssl_module --with-http_realip_module --add-dynamic-module=../ngx-fancyindex-0.4.0 --add-module=../ngx_cache_purge-2.3 --add-module=../ngx_devel_kit-0.3.0 --add-module=../set-misc-nginx-module-0.31 --add-module=../echo-nginx-module-0.60 --add-module=../redis2-nginx-module-0.14 --add-module=../ngx_http_redis-0.3.7 --add-module=../memc-nginx-module-0.18 --add-module=../srcache-nginx-module-0.31 --add-module=../headers-more-nginx-module-0.32 --with-pcre=../pcre-8.40 --with-pcre-jit --with-zlib=../zlib-1.2.11 --with-http_ssl_module --with-http_v2_module --with-openssl=../openssl-1.1.0f
Click to expand...

eva2000 · Jun 23, 2017

HPACK: the silent killer (feature) of HTTP/2

Current implementation of nginx, as well edge networks and CDNs using it, do not support the full HPACK implementation. We have, however, implemented the full HPACK in nginx, and upstreamed the part that performs Huffman encoding.
Click to expand...

HPACK
HTTP/2 supports a new dedicated header compression algorithm, called HPACK. HPACK was developed with attacks like CRIME in mind, and is therefore considered safe to use.

HPACK is resilient to CRIME, because it does not use partial backward string matches and dynamic Huffman codes like DEFLATE. Instead, it uses these three methods of compression:

Static Dictionary: A predefined dictionary of 61 commonly used header fields, some with predefined values.

Dynamic Dictionary: A list of actual headers that were encountered during the connection. This dictionary has limited size, and when new entries are added, old entries might be evicted.

Huffman Encoding: A static Huffman code can be used to encode any string: name or value. This code was computed specifically for HTTP Response/Request headers - ASCII digits and lowercase letters are given shorter encodings. The shortest encoding possible is 5 bits long, therefore the highest compression ratio achievable is 8:5 (or 37.5% smaller).

Click to expand...

As per article to test full HPACK encoding for header compression you can use nghttp2 tool's h2load to check header compression savings i.e. 4 runs from cloudflare blog
Code (Text):
url=https://blog.cloudflare.com
for i in $(seq 1 4); do echo "h2load run $i"; h2load $url -n $i | tail -6 | head -1; done                     
h2load run 1
traffic: 27.83KB (28502) total, 541B (541) headers (space savings 27.96%), 27.21KB (27858) data
h2load run 2
traffic: 55.12KB (56443) total, 570B (570) headers (space savings 62.05%), 54.41KB (55716) data
h2load run 3
traffic: 82.41KB (84384) total, 599B (599) headers (space savings 73.41%), 81.62KB (83574) data
h2load run 4
traffic: 109.69KB (112327) total, 630B (630) headers (space savings 79.03%), 108.82KB (111432) data
Tested Centmin Mod 123.09beta01 updated with Nginx 1.13.1 + NGINX_HPACK='y' with OpenSSL 1.0.2l and 1.1.0f and not seeing such HPACK HTTP/2 header savings on subsequent h2load runs though ?

Test again test site i.e. Nginx HTTP/2 HTTPS based domain.com defined by variable url
Code (Text):
url=https://domain.com
Code (Text):
curl -I $url
HTTP/2 200
date: Fri, 23 Jun 2017 11:36:50 GMT
content-type: text/html; charset=utf-8
content-length: 6090
last-modified: Fri, 23 Jun 2017 11:24:24 GMT
vary: Accept-Encoding
etag: "594cfa68-17ca"
server: nginx centminmod
x-powered-by: centminmod
expires: Sat, 24 Jun 2017 11:36:50 GMT
cache-control: max-age=86400
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
accept-ranges: bytes
h2load reported header space savings due to full HPACK encoding patch enabled support doesn't change on subsequent h2load runs #2-4 ?
Code (Text):
for i in $(seq 1 4); do echo "h2load run $i"; h2load $url -n $i | tail -6 | head -1; done
h2load run 1
traffic: 6.33KB (6481) total, 324B (324) headers (space savings 15.62%), 5.95KB (6090) data
h2load run 2
traffic: 12.61KB (12913) total, 648B (648) headers (space savings 15.62%), 11.89KB (12180) data
h2load run 3
traffic: 18.89KB (19345) total, 972B (972) headers (space savings 15.62%), 17.84KB (18270) data
h2load run 4
traffic: 25.17KB (25777) total, 1.27KB (1296) headers (space savings 15.62%), 23.79KB (24360) dat
without cloudflare HTTP/2 HPACK Full encoding patch strange get higher compression levels/header space savings ?
Code (Text):
for i in $(seq 1 4); do echo "h2load run $i"; h2load $url -n $i | tail -6 | head -1; done
h2load run 1
traffic: 6.29KB (6442) total, 285B (285) headers (space savings 25.78%), 5.95KB (6090) data
h2load run 2
traffic: 12.53KB (12835) total, 570B (570) headers (space savings 25.78%), 11.89KB (12180) data
h2load run 3
traffic: 18.78KB (19228) total, 855B (855) headers (space savings 25.78%), 17.84KB (18270) data
h2load run 4
traffic: 25.02KB (25621) total, 1.11KB (1140) headers (space savings 25.78%), 23.79KB (24360) data
nginx -V
nginx version: nginx/1.13.1
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC)
built with OpenSSL 1.0.2l 25 May 2017
TLS SNI support enabled
configure arguments: --with-ld-opt='-ljemalloc -Wl,-z,relro -Wl,-rpath,/usr/local/lib' --with-cc-opt='-m64 -march=native -DTCP_FASTOPEN=23 -g -O3 -fstack-protector-strong -fuse-ld=gold --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -gsplit-dwarf' --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-compat --with-http_stub_status_module --with-http_secure_link_module --with-libatomic --with-http_gzip_static_module --with-http_sub_module --with-http_addition_module --with-http_image_filter_module=dynamic --with-http_geoip_module --with-stream_geoip_module --with-stream_realip_module --with-stream_ssl_preread_module --with-threads --with-stream=dynamic --with-stream_ssl_module --with-http_realip_module --add-dynamic-module=../ngx-fancyindex-0.4.0 --add-module=../ngx_cache_purge-2.3 --add-module=../ngx_devel_kit-0.3.0 --add-module=../set-misc-nginx-module-0.31 --add-module=../echo-nginx-module-0.60 --add-module=../redis2-nginx-module-0.14 --add-module=../ngx_http_redis-0.3.7 --add-module=../memc-nginx-module-0.18 --add-module=../srcache-nginx-module-0.31 --add-module=../headers-more-nginx-module-0.32 --with-pcre=../pcre-8.40 --with-pcre-jit --with-zlib=../zlib-1.2.11 --with-http_ssl_module --with-http_v2_module --with-openssl=../openssl-1.0.2l
Click to expand...

submitted an issue for cloudflare sslconfig github repo at Nginx 1.13.1 + HTTP/2 Full HPACK encoding patch verification? · Issue #72 · cloudflare/sslconfig · GitHub

edit: fixed add missing --with-http_v2_hpack_enc configure option and confirmed if i remove -O2 or -O3 during Nginx compile when HPACK patch detected in centmin.sh menu option 4 runs, then yes Nginx 1.13.1 compiles with full HPACK encoding patch !

confirm the improved header savings in h2load tests
Code (Text):
url=https://domain.com
for i in $(seq 1 4); do echo "h2load run $i"; h2load $url -n $i | tail -6 | head -1; done
h2load run 1
traffic: 6.29KB (6441) total, 284B (284) headers (space savings 26.04%), 5.95KB (6090) data
h2load run 2
traffic: 12.45KB (12752) total, 487B (487) headers (space savings 36.59%), 11.89KB (12180) data
h2load run 3
traffic: 18.62KB (19063) total, 690B (690) headers (space savings 40.10%), 17.84KB (18270) data
h2load run 4
traffic: 24.78KB (25374) total, 893B (893) headers (space savings 41.86%), 23.79KB (24360) data
nginx -V
nginx version: nginx/1.13.1
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC)
built with OpenSSL 1.0.2l 25 May 2017
TLS SNI support enabled
configure arguments: --with-ld-opt='-ljemalloc -Wl,-z,relro -Wl,-rpath,/usr/local/lib' --with-cc-opt='-m64 -march=native -DTCP_FASTOPEN=23 -g -fstack-protector-strong -fuse-ld=gold --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -gsplit-dwarf' --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-compat --with-http_stub_status_module --with-http_secure_link_module --with-libatomic --with-http_gzip_static_module --with-http_sub_module --with-http_addition_module --with-http_image_filter_module=dynamic --with-http_geoip_module --with-stream_geoip_module --with-stream_realip_module --with-stream_ssl_preread_module --with-threads --with-stream=dynamic --with-stream_ssl_module --with-http_realip_module --add-dynamic-module=../ngx-fancyindex-0.4.0 --add-module=../ngx_cache_purge-2.3 --add-module=../ngx_devel_kit-0.3.0 --add-module=../set-misc-nginx-module-0.31 --add-module=../echo-nginx-module-0.60 --add-module=../redis2-nginx-module-0.14 --add-module=../ngx_http_redis-0.3.7 --add-module=../memc-nginx-module-0.18 --add-module=../srcache-nginx-module-0.31 --add-module=../headers-more-nginx-module-0.32 --with-pcre=../pcre-8.40 --with-pcre-jit --with-zlib=../zlib-1.2.11 --with-http_ssl_module --with-http_v2_module --with-http_v2_hpack_enc --with-openssl=../openssl-1.0.2l
Click to expand...

eva2000 · Jun 23, 2017

more tests with Cloudflare HTTP/2 HPACK full encoding patch with Centmin Mod 123.09beta01's Nginx 1.13.1 with persistent config file /etc/centminmod/custom_config.inc set with
Code (Text):
NGINX_HPACK='y'
LIBRESSL_SWITCH='n'
CLANG='n'
NGINX_DEVTOOLSETGCC='y'
DEVTOOLSETSIX='y'
which will enable Cloudflare HTTP/2 HPACK patch, switch from LibreSSL to OpenSSL crypto library, use GCC instead of Clang compiler and set Nginx GCC version to devtoolset-4's provided GCC 6.2.1

nginx -V
nginx version: nginx/1.13.1
built by gcc 6.2.1 20160916 (Red Hat 6.2.1-3) (GCC)
built with OpenSSL 1.1.0f 25 May 2017
TLS SNI support enabled
configure arguments: --with-ld-opt='-ljemalloc -Wl,-z,relro -Wl,-rpath,/usr/local/lib' --with-cc-opt='-m64 -march=native -DTCP_FASTOPEN=23 -g -fstack-protector-strong -flto -fuse-ld=gold --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -gsplit-dwarf' --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-compat --with-http_stub_status_module --with-http_secure_link_module --with-libatomic --with-http_gzip_static_module --with-http_sub_module --with-http_addition_module --with-http_image_filter_module=dynamic --with-http_geoip_module --with-stream_geoip_module --with-stream_realip_module --with-stream_ssl_preread_module --with-threads --with-stream=dynamic --with-stream_ssl_module --with-http_realip_module --add-dynamic-module=../ngx-fancyindex-0.4.0 --add-module=../ngx_cache_purge-2.3 --add-module=../ngx_devel_kit-0.3.0 --add-module=../set-misc-nginx-module-0.31 --add-module=../echo-nginx-module-0.60 --add-module=../redis2-nginx-module-0.14 --add-module=../ngx_http_redis-0.3.7 --add-module=../memc-nginx-module-0.18 --add-module=../srcache-nginx-module-0.31 --add-module=../headers-more-nginx-module-0.32 --with-pcre=../pcre-8.40 --with-pcre-jit --with-zlib=../zlib-1.2.11 --with-http_ssl_module --with-http_v2_module --with-http_v2_hpack_enc --with-openssl=../openssl-1.1.0f
Click to expand...

h2load header check test over 1-8 runs with up to 44% HTTP/2 header compression savings on subsequent HTTP/2 requests
Code (Text):
url=https://domain.com

for i in $(seq 1 8); do echo "h2load run $i"; h2load $url -n $i | tail -6 | head -1; done
h2load run 1
traffic: 6.29KB (6441) total, 284B (284) headers (space savings 26.04%), 5.95KB (6090) data
h2load run 2
traffic: 12.45KB (12752) total, 487B (487) headers (space savings 36.59%), 11.89KB (12180) data
h2load run 3
traffic: 18.62KB (19063) total, 690B (690) headers (space savings 40.10%), 17.84KB (18270) data
h2load run 4
traffic: 24.78KB (25374) total, 893B (893) headers (space savings 41.86%), 23.79KB (24360) data
h2load run 5
traffic: 30.94KB (31685) total, 1.07KB (1096) headers (space savings 42.92%), 29.74KB (30450) data
h2load run 6
traffic: 37.11KB (37996) total, 1.27KB (1299) headers (space savings 43.62%), 35.68KB (36540) data
h2load run 7
traffic: 43.27KB (44307) total, 1.47KB (1502) headers (space savings 44.12%), 41.63KB (42630) data
h2load run 8
traffic: 49.43KB (50618) total, 1.67KB (1705) headers (space savings 44.50%), 47.58KB (48720) data

buik · Jun 24, 2017

Hmmwha HPACK: the silent killer (feature) of HTTP/2 haha.
Looking at Github it took hours to implement this in Centminmod, with only minimal profit.

Even if you get 80% benefit, header optimization is still a mini mini mini piece section of optimization.
They better released their HTTP/2 server push patch or their OpenSSL with BoringSSL enhancements.

To really take great steps and no mice steps like now.
As Cloudflare already released this defacto patch in 2015.

(Nothing to do with this experimental module)

eva2000 · Jun 24, 2017

heh.. well yeah hopefully step by step the full HTTP/2 feature set is made available in Nginx

updated centminmod.com Nginx 1.13.1 with the Cloudflare HTTP/2 HPACK Full encoding patch

Code (Text):

url=https://centminmod.com

for i in $(seq 1 8); do echo "h2load run $i"; h2load $url -n $i | tail -6 | head -1; done
h2load run 1
traffic: 76.02KB (77847) total, 335B (335) headers (space savings 18.89%), 75.55KB (77364) data
h2load run 2
traffic: 151.94KB (155591) total, 616B (616) headers (space savings 25.42%), 151.10KB (154728) data
h2load run 3
traffic: 227.87KB (233335) total, 897B (897) headers (space savings 27.60%), 226.65KB (232092) data
h2load run 4
traffic: 303.79KB (311079) total, 1.15KB (1178) headers (space savings 28.69%), 302.20KB (309456) data
h2load run 5
traffic: 379.71KB (388823) total, 1.42KB (1459) headers (space savings 29.35%), 377.75KB (386820) data
h2load run 6
traffic: 455.63KB (466567) total, 1.70KB (1740) headers (space savings 29.78%), 453.30KB (464184) data
h2load run 7
traffic: 531.55KB (544311) total, 1.97KB (2021) headers (space savings 30.09%), 528.86KB (541548) data
h2load run 8
traffic: 607.48KB (622055) total, 2.25KB (2302) headers (space savings 30.33%), 604.41KB (618912) data

Code (Text):

url=https://centminmod.com/js/jquery.min.js

for i in $(seq 1 8); do echo "h2load run $i"; h2load $url -n $i | tail -6 | head -1; done
h2load run 1
traffic: 82.78KB (84771) total, 360B (360) headers (space savings 22.91%), 82.27KB (84245) data
h2load run 2
traffic: 165.44KB (169406) total, 651B (651) headers (space savings 30.15%), 164.54KB (168490) data
h2load run 3
traffic: 248.10KB (254051) total, 943B (943) headers (space savings 32.55%), 246.81KB (252735) data
h2load run 4
traffic: 330.78KB (338719) total, 1.23KB (1258) headers (space savings 32.51%), 329.08KB (336980) data
h2load run 5
traffic: 413.42KB (423341) total, 1.49KB (1527) headers (space savings 34.46%), 411.35KB (421225) data
h2load run 6
traffic: 496.08KB (507986) total, 1.78KB (1819) headers (space savings 34.94%), 493.62KB (505470) data
h2load run 7
traffic: 578.78KB (592666) total, 2.09KB (2137) headers (space savings 34.55%), 575.89KB (589715) data
h2load run 8
traffic: 661.41KB (677288) total, 2.35KB (2406) headers (space savings 35.51%), 658.16KB (673960) data

but looks like Cloudflare's implementation has more savings, centmin.sh domain is centminmod.com mirror using Cloudflare free plan = HPACK'd up to 84% savings ! And look at data size, Cloudflare probaby using higher levels of compression

Code (Text):

url=https://centmin.sh

for i in $(seq 1 8); do echo "h2load run $i"; h2load $url -n $i | tail -6 | head -1; done
h2load run 1
traffic: 49.78KB (50976) total, 473B (473) headers (space savings 28.01%), 49.19KB (50373) data
h2load run 2
traffic: 99.09KB (101470) total, 513B (513) headers (space savings 60.96%), 98.38KB (100746) data
h2load run 3
traffic: 148.40KB (151960) total, 549B (549) headers (space savings 72.15%), 147.58KB (151119) data
h2load run 4
traffic: 289.92KB (296877) total, 674B (674) headers (space savings 74.35%), 288.79KB (295722) data
h2load run 5
traffic: 369.93KB (378808) total, 705B (705) headers (space savings 78.54%), 368.66KB (377505) data
h2load run 6
traffic: 419.26KB (429318) total, 752B (752) headers (space savings 80.92%), 417.85KB (427878) data
h2load run 7
traffic: 438.09KB (448609) total, 1.01KB (1035) headers (space savings 77.50%), 436.37KB (446841) data
h2load run 8
traffic: 456.44KB (467398) total, 825B (825) headers (space savings 84.30%), 454.89KB (465804) data

buik · Jun 24, 2017

eva2000 said: ↑

heh.. well yeah hopefully step by step the full HTTP/2 feature set is made available in Nginx
Click to expand...

Cloudflare released this patch already in 2015.
[PATCH] HTTP/2: HPACK Huffman encoding
They are now making everyone happy with an old gift.

eva2000 · Jun 24, 2017

haha.. well anything is better than nothing

buik · Jun 24, 2017

eva2000 said: ↑

haha.. well anything is better than nothing
Click to expand...

Valentin is going to look (as time permits) to implement this to upstream.
[PATCH] HTTP/2: add support for HPACK encoding

eva2000 · Jun 24, 2017

bassie said: ↑

Valentin is going to look (as time permits) to implement this to upstream.
[PATCH] HTTP/2: add support for HPACK encoding
Click to expand...

Ah that is good news

eva2000 · Jun 24, 2017

testing with OpenSSL 1.1.1-dev TLS v1.3 draft-18 branch + HPACK patch

nginx -V
nginx version: nginx/1.13.1
built by gcc 6.2.1 20160916 (Red Hat 6.2.1-3) (GCC)
built with OpenSSL 1.1.1-dev xx XXX xxxx
TLS SNI support enabled
configure arguments: --with-ld-opt='-ljemalloc -Wl,-z,relro -Wl,-rpath,/usr/local/lib' --with-cc-opt='-m64 -march=native -DTCP_FASTOPEN=23 -g -O3 -Wno-error=strict-aliasing -fstack-protector-strong -flto -fuse-ld=gold --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -gsplit-dwarf' --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-compat --with-http_stub_status_module --with-http_secure_link_module --with-libatomic --with-http_gzip_static_module --with-http_sub_module --with-http_addition_module --with-http_image_filter_module=dynamic --with-http_geoip_module --with-stream_geoip_module --with-stream_realip_module --with-stream_ssl_preread_module --with-threads --with-stream=dynamic --with-stream_ssl_module --with-http_realip_module --add-dynamic-module=../ngx-fancyindex-0.4.0 --add-module=../ngx_cache_purge-2.3 --add-module=../ngx_devel_kit-0.3.0 --add-module=../set-misc-nginx-module-0.31 --add-module=../echo-nginx-module-0.60 --add-module=../redis2-nginx-module-0.14 --add-module=../ngx_http_redis-0.3.7 --add-module=../memc-nginx-module-0.18 --add-module=../srcache-nginx-module-0.31 --add-module=../headers-more-nginx-module-0.32 --with-pcre=../pcre-8.40 --with-pcre-jit --with-zlib=../zlib-1.2.11 --with-http_ssl_module --with-http_v2_module --with-http_v2_hpack_enc --with-openssl=../openssl-tls1.3 --with-openssl-opt='enable-tls1_3'
Click to expand...

interesting header savings are much better at up to 88% savings ! seems the HPACK patch was updated with a few fixes
Code (Text):
url=https://domain.com
for i in $(seq 1 8); do echo "h2load run $i"; h2load $url -n $i | tail -6 | head -1; done
h2load run 1
traffic: 4.02KB (4117) total, 249B (249) headers (space savings 34.99%), 3.71KB (3801) data
h2load run 2
traffic: 7.76KB (7951) total, 264B (264) headers (space savings 65.54%), 7.42KB (7602) data
h2load run 3
traffic: 11.51KB (11785) total, 279B (279) headers (space savings 75.72%), 11.14KB (11403) data
h2load run 4
traffic: 15.25KB (15619) total, 294B (294) headers (space savings 80.81%), 14.85KB (15204) data
h2load run 5
traffic: 18.00KB (19453) total, 309B (309) headers (space savings 83.86%), 18.56KB (19005) data
h2load run 6
traffic: 22.74KB (23287) total, 324B (324) headers (space savings 85.90%), 22.27KB (22806) data
h2load run 7
traffic: 26.49KB (27121) total, 339B (339) headers (space savings 87.36%), 25.98KB (26607) data
h2load run 8
traffic: 30.23KB (30955) total, 354B (354) headers (space savings 88.45%), 29.70KB (30408) data
with OpenSSL 1.1.0f

for i in $(seq 1 8); do echo "h2load run $i"; h2load $url -n $i | tail -6 | head -1; done
h2load run 1
traffic: 4.02KB (4116) total, 248B (248) headers (space savings 35.25%), 3.71KB (3801) data
h2load run 2
traffic: 7.76KB (7950) total, 263B (263) headers (space savings 65.67%), 7.42KB (7602) data
h2load run 3
traffic: 11.51KB (11784) total, 278B (278) headers (space savings 75.81%), 11.14KB (11403) data
h2load run 4
traffic: 15.25KB (15618) total, 293B (293) headers (space savings 80.87%), 14.85KB (15204) data
h2load run 5
traffic: 18.00KB (19452) total, 308B (308) headers (space savings 83.92%), 18.56KB (19005) data
h2load run 6
traffic: 22.74KB (23286) total, 323B (323) headers (space savings 85.94%), 22.27KB (22806) data
h2load run 7
traffic: 26.48KB (27120) total, 338B (338) headers (space savings 87.39%), 25.98KB (26607) data
h2load run 8
traffic: 30.23KB (30954) total, 353B (353) headers (space savings 88.48%), 29.70KB (30408) data
Click to expand...
Code (Text):
url=https://domain.com
for i in $(seq 1 8); do echo "h2load run $i"; h2load $url -n $i | tail -6 | head -1; done
h2load run 1
traffic: 4.02KB (4116) total, 248B (248) headers (space savings 35.25%), 3.71KB (3801) data
h2load run 2
traffic: 7.76KB (7950) total, 263B (263) headers (space savings 65.67%), 7.42KB (7602) data
h2load run 3
traffic: 11.51KB (11784) total, 278B (278) headers (space savings 75.81%), 11.14KB (11403) data
h2load run 4
traffic: 15.25KB (15618) total, 293B (293) headers (space savings 80.87%), 14.85KB (15204) data
h2load run 5
traffic: 18.00KB (19452) total, 308B (308) headers (space savings 83.92%), 18.56KB (19005) data
h2load run 6
traffic: 22.74KB (23286) total, 323B (323) headers (space savings 85.94%), 22.27KB (22806) data
h2load run 7
traffic: 26.48KB (27120) total, 338B (338) headers (space savings 87.39%), 25.98KB (26607) data
h2load run 8
traffic: 30.23KB (30954) total, 353B (353) headers (space savings 88.48%), 29.70KB (30408) data

eva2000 · Jun 24, 2017

testing centminmod.com site with updated patch up to 92% savings for header compression !

Code (Text):

url=https://centminmod.com

for i in $(seq 1 25); do echo "h2load run $i"; h2load $url -n $i | tail -6 | head -1; done
h2load run 1
traffic: 78.10KB (79977) total, 309B (309) headers (space savings 25.00%), 77.66KB (79520) data
h2load run 2
traffic: 155.87KB (159613) total, 326B (326) headers (space savings 60.44%), 155.31KB (159040) data
h2load run 3
traffic: 233.64KB (239249) total, 343B (343) headers (space savings 72.25%), 232.97KB (238560) data
h2load run 4
traffic: 311.41KB (318885) total, 360B (360) headers (space savings 78.16%), 310.63KB (318080) data
h2load run 5
traffic: 389.18KB (398521) total, 377B (377) headers (space savings 81.70%), 388.28KB (397600) data
h2load run 6
traffic: 535.92KB (548783) total, 401B (401) headers (space savings 83.80%), 534.81KB (547649) data
h2load run 7
traffic: 544.72KB (557793) total, 411B (411) headers (space savings 85.75%), 543.59KB (556640) data
h2load run 8
traffic: 622.51KB (637452) total, 451B (451) headers (space savings 86.32%), 621.25KB (636160) data
h2load run 9
traffic: 700.26KB (717065) total, 445B (445) headers (space savings 88.00%), 698.91KB (715680) data
h2load run 10
traffic: 778.03KB (796701) total, 462B (462) headers (space savings 88.79%), 776.56KB (795200) data
h2load run 11
traffic: 855.80KB (876337) total, 479B (479) headers (space savings 89.43%), 854.22KB (874720) data
h2load run 12
traffic: 933.57KB (955973) total, 496B (496) headers (space savings 89.97%), 931.88KB (954240) data
h2load run 13
traffic: 1.06MB (1106259) total, 544B (544) headers (space savings 89.85%), 1.05MB (1104289) data
h2load run 14
traffic: 1.13MB (1185903) total, 560B (560) headers (space savings 90.30%), 1.13MB (1183809) data
h2load run 15
traffic: 1.14MB (1194881) total, 547B (547) headers (space savings 91.15%), 1.14MB (1192800) data
h2load run 16
traffic: 1.22MB (1274540) total, 587B (587) headers (space savings 91.10%), 1.21MB (1272320) data
h2load run 17
traffic: 1.36MB (1424811) total, 611B (611) headers (space savings 91.28%), 1.36MB (1422369) data
h2load run 18
traffic: 1.43MB (1504415) total, 605B (605) headers (space savings 91.85%), 1.43MB (1501889) data
h2load run 19
traffic: 1.44MB (1513465) total, 646B (646) headers (space savings 91.75%), 1.44MB (1510880) data
h2load run 20
traffic: 1.59MB (1663719) total, 662B (662) headers (space savings 91.97%), 1.58MB (1660929) data
h2load run 21
traffic: 1.60MB (1672720) total, 672B (672) headers (space savings 92.23%), 1.59MB (1669920) data
h2load run 22
traffic: 1.67MB (1752333) total, 666B (666) headers (space savings 92.65%), 1.67MB (1749440) data
h2load run 23
traffic: 1.75MB (1831969) total, 683B (683) headers (space savings 92.79%), 1.74MB (1828960) data
h2load run 24
traffic: 1.82MB (1911621) total, 707B (707) headers (space savings 92.85%), 1.82MB (1908480) data
h2load run 25
traffic: 1.97MB (2061891) total, 748B (748) headers (space savings 92.74%), 1.96MB (2058529) data

with http compression for reduced data requests

Code (Text):

for i in $(seq 1 25); do echo "h2load run $i"; h2load $url -n $i -H 'accept-encoding: gzip, deflate, sdch, br' | tail -6 | head -1; done
h2load run 1
traffic: 12.90KB (13208) total, 301B (301) headers (space savings 26.94%), 12.52KB (12822) data
h2load run 2
traffic: 25.45KB (26064) total, 317B (317) headers (space savings 61.44%), 25.04KB (25644) data
h2load run 3
traffic: 38.02KB (38930) total, 334B (334) headers (space savings 72.91%), 37.56KB (38466) data
h2load run 4
traffic: 50.58KB (51796) total, 351B (351) headers (space savings 78.65%), 50.09KB (51288) data
h2load run 5
traffic: 63.15KB (64662) total, 368B (368) headers (space savings 82.09%), 62.61KB (64110) data
h2load run 6
traffic: 92.48KB (94695) total, 392B (392) headers (space savings 84.14%), 91.85KB (94056) data
h2load run 7
traffic: 88.30KB (90417) total, 425B (425) headers (space savings 85.23%), 87.65KB (89754) data
h2load run 8
traffic: 117.59KB (120417) total, 426B (426) headers (space savings 87.06%), 116.89KB (119699) data
h2load run 9
traffic: 113.40KB (116126) total, 436B (436) headers (space savings 88.21%), 112.69KB (115398) data
h2load run 10
traffic: 125.97KB (128992) total, 453B (453) headers (space savings 88.98%), 125.21KB (128220) data
h2load run 11
traffic: 155.31KB (159039) total, 501B (501) headers (space savings 88.93%), 154.46KB (158165) data
h2load run 12
traffic: 151.10KB (154724) total, 487B (487) headers (space savings 90.13%), 150.26KB (153864) data
h2load run 13
traffic: 163.66KB (167590) total, 504B (504) headers (space savings 90.57%), 162.78KB (166686) data
h2load run 14
traffic: 176.23KB (180456) total, 521B (521) headers (space savings 90.95%), 175.30KB (179508) data
h2load run 15
traffic: 188.83KB (193362) total, 569B (569) headers (space savings 90.77%), 187.82KB (192330) data
h2load run 16
traffic: 218.12KB (223354) total, 562B (562) headers (space savings 91.46%), 217.07KB (222275) data
h2load run 17
traffic: 213.94KB (219070) total, 579B (579) headers (space savings 91.72%), 212.87KB (217974) data
h2load run 18
traffic: 226.51KB (231943) total, 612B (612) headers (space savings 91.73%), 225.39KB (230796) data
h2load run 19
traffic: 239.05KB (244786) total, 606B (606) headers (space savings 92.24%), 237.91KB (243618) data
h2load run 20
traffic: 251.63KB (257668) total, 630B (630) headers (space savings 92.34%), 250.43KB (256440) data
h2load run 21
traffic: 297.71KB (304857) total, 670B (670) headers (space savings 92.25%), 296.39KB (303508) data
h2load run 22
traffic: 276.76KB (283400) total, 664B (664) headers (space savings 92.66%), 275.47KB (282084) data
h2load run 23
traffic: 289.34KB (296289) total, 704B (704) headers (space savings 92.55%), 287.99KB (294906) data
h2load run 24
traffic: 335.39KB (343435) total, 698B (698) headers (space savings 92.93%), 333.96KB (341977) data
h2load run 25
traffic: 314.44KB (321982) total, 708B (708) headers (space savings 93.11%), 313.04KB (320550) data

eva2000 · Jun 24, 2017

For those folks wondering what the benefit of this is might want to read HPACK: HTTP/2’s Hidden Gem - GlobalDots Blog as well

You may ask yourself to why do we even need header compression? As pointed out on Github’s HTTP/2 FAQ page, assuming a page has 80 assets and each request results in 1400 bytes of header, it takes at least 7-8 roundtrips to deliver all the headers to the client. This is largely due to TCP’s Slow Startimplementation which sets up new connections based on the number of confirmed packets, thus resulting in a limited amount of packets for the first few roundtrips. Even a low level of compression on headers greatly reduces roundtrips and payload, and can help deliver the request within just one roundtrip, or even within a single packet.

Using HPACK that’s exactly what happens. Headers can now be compressed using Huffman encoding which results in an average 30% reduction of header size. As KeyCDN pointed out in their blogpost, the 3 main benefits of HPACK are:

Resilient against compression based attacks (CRIME)

Fixed Huffman codes allow encoding of large headers

Frequently used headers can be encoded as variable length integer, opposed to re-sending the whole header every time

Essentially, the Huffman encoding algorithm operates by assigning binary codes to most commonly used characters. When a HPACK header is received, it gets looked up in the list of binary codes. This way the header can access frequently used characters compressed in smaller packets which in turn results in less bits required to complete the transmission. Previously, an accept string encoded with 7 bits per character would require 6 bytes while the HPACK version only takes 2 bytes. It means less data gets transferred which ultimately results in improved web performance.
Click to expand...

So if my calculations are correct on 1 million page requests with 100 assets per page and 400 bytes header size means:

1000000 x 100 x 400 bytes = 38,146 MegaBytes or 37.25 Gigabytes of data normally. So with up to 90% HPACK encoded header savings with HPACK full encoding, it means you save around 33.525 Gigabytes per 1 million page requests and on average lower the number of roundtrips for transferring the data per page load.

buik · Jun 24, 2017

Next to Nginx 1.13.1
Working fine on Nginx 1.12.0

traffic: 1.67MB (1746990) total, 2.29KB (2341) headers (space savings 97.98%), 1.66MB (1741000) data
Click to expand...

eva2000 · Jun 24, 2017

nice did you use 1.11 or 1.13 patch for 1.12 ?

buik · Jun 24, 2017

eva2000 said: ↑

nice did you use 1.11 or 1.13 patch for 1.12 ?
Click to expand...

There is only a git patch and 1.13.1 yet?

eva2000 · Jun 24, 2017

they renamed them there's 2 versions of hpack patch

nginx_http2_hpack.patch

nginx_1.13.1_http2_hpack.patch

buik · Jun 24, 2017

eva2000 said: ↑

they renamed them there's 2 versions of hpack patch

nginx_http2_hpack.patch

nginx_1.13.1_http2_hpack.patch

Click to expand...

That patch is based on the current HEAD version.
Click to expand...

sslconfig/nginx_1.13.1_http2_hpack.patch at hpack_1.13.1 · cloudflare/sslconfig · GitHub?

I didn't test it very well though.

Thanks.
Click to expand...

Jup the current Nginx git and 1.13.1 version as written by Cloudflare's
vkrasnov. Using the Nginx 1.13.1 patch here on Nginx 1.12.0 with a bunch of changes as it conflicts with other patches otherwise.

eva2000 · Jun 24, 2017

Good to know. I am only enabling in Centmin Mod on Nginx 1.13.1+ and higher right now though.

buik · Jun 24, 2017

eva2000 said: ↑

Good to know. I am only enabling in Centmin Mod on Nginx 1.13.1+ and higher right now though.
Click to expand...

Why?
Looking at the code, there is nothing really changed at the HTTP/2 part between 1.12 and 1.13.1.

Log in / Register

Forums

Learn about Centmin Mod LEMP Stack today

Beta Branch Nginx HTTP/2 & OpenSSL 1.1.0 patch updates

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

.

Log in / Register

Useful Searches

Learn about Centmin Mod LEMP Stack today

Beta Branch Nginx HTTP/2 & OpenSSL 1.1.0 patch updates

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

.