Nginx Optimizing TLS over TCP to reduce latency: TLS dynamic record sizing

Revenge · Jun 11, 2016

eva2000 said: ↑

no no, LibreSSL or OpenSSL will work for this with CentminMod 123.09beta01 as it's an nginx patch not openssl patch
Click to expand...

You said this on the second post, so my doubt came from there

would only work on OpenSSL 1.0.2 but Centmin Mod supports LibreSSL 2.3 and OpenSSL 1.0.2+. However, LibreSSL is default unless you set in persistent config file /etc/centminmod/custom_config.inc
Click to expand...

eva2000 · Jun 11, 2016

Revenge said: ↑

You said this on the second post, so my doubt came from there
Click to expand...

But edited that post to correct my assumption

Revenge · Jun 11, 2016

Ok, i just made a test on my site. Before and after the patch. Im using a non centminmod, with the default 16k ssl_buffer_size.

Using webpagetest.org, with 9 tests, i did not see any diference before and after the patch regard ttfb and load page.

eva2000 · Jun 11, 2016

Revenge said: ↑

Ok, i just made a test on my site. Before and after the patch. Im using a non centminmod, with the default 16k ssl_buffer_size.

Using webpagetest.org, with 9 tests, i did not see any diference before and after the patch regard ttfb and load page.
Click to expand...

yeah cloudflare blog article did say benefits will vary

Revenge · Jun 11, 2016

OK, i changed the ssl buffer size to 1369 and tested the ttfb again. Im getting results of 0,3 seconds, where it was 0,8s. Thats less than half.
With the patch, it wasn't supposed to start at 1369? The ttfb should be the same.

eva2000 · Jun 11, 2016

Revenge said: ↑

OK, i changed the ssl buffer size to 1369 and tested the ttfb again. Im getting results of 0,3 seconds, where it was 0,8s. Thats less than half.
With the patch, it wasn't supposed to start at 1369? The ttfb should be the same.
Click to expand...

think it depends on the total size of your assets and records see calculations i did at Nginx - Optimizing TLS over TCP to reduce latency: TLS dynamic record sizing | Centmin Mod Community

eva2000 said: ↑

so the patch starts with 1369 bytes sent for up to 1+40 = 41 records so first 41 x 1369 bytes = 56,129 bytes = 54.813 KB before the patch starts sending the ssl_dync_rec_size_hi 4229 bytes based records for another 40 records = 40 x 4229 = 169,160 bytes (165.19KB) before you fall back to ssl_buffer_size of 16KB for non-Centmin Mod Nginx or 1369 bytes for Centmin Mod Nginx

so 1st stage up to 54.813KB then next 165.19KB so < 220KB (54.813+165.19) before you fall back to ssl_buffer_size of 16KB for non-Centmin Mod Nginx or 1369 bytes for Centmin Mod Nginx
Click to expand...

so maybe for you you have alot more records being retransmitted above >220KB and thus hit the ssl_buffer_size fallback set size?

Revenge · Jun 11, 2016

Total size is 647 Kb but the biggest asset is 111 Kb.

eva2000 · Jun 11, 2016

Revenge said: ↑

Total size is 647 Kb but the biggest asset is 111 Kb.
Click to expand...

you'd probably have to dig into TCP traffic/wireshark to compare both tests etc

eva2000 · Jun 11, 2016

bug fix for nginx dynamic tls patch change Logic: (4229,ssl_dyn_rec_size_lo) to (4229,ssl_dyn_rec_size_hi) by 0x7E · Pull Request #40 · cloudflare/sslconfig · GitHub ?

doh it's just documentation text typo fix heh

eva2000 · Jun 11, 2016

Much older article but it's the reason Nginx eventually had an ssl_buffer_size value in the first place Optimizing TLS Record Size & Buffering Latency - igvita.com Good background reading

guess we should be testing benefits over mobile speed connections and not cable and higher

TLS record size can have significant impact on the page load time performance of your application. In fact, in the worst case, which unfortunately is also the most common case on the web today, it can delay processing of received data by up to several roundtrips! On mobile networks, this can translate to hundreds of milliseconds of unnecessary latency.

The good news is, this is a relatively simple thing to fix if your TLS server supports it. However, first let's take a step back and understand the problem: what is a TLS record, why do we need it, and why does record size affect latency of our applications?
Click to expand...

Head-of-line blocking, TLS records, and latency
TLS runs over TCP, and TCP promises in order delivery of all transferred packets. As a result, TCP suffers from head-of-line (HOL) blocking where a lost packet may hold all other received packets in the buffer until it is successfully retransmitted - otherwise, the packets would be delivered out of order! This is a well-known tradeoff for any protocol that runs on top of a reliable and in-order transport like TCP.

However, in case of TLS, we have an extra layer of buffering due to the integrity checks! Once TCP delivers the packets to the TLS layer above it, we must first accumulate the entire record, then verify its MAC checksum, and only when that passes, can we release the data to the application above it. As a result, if the server emits data in 16KB record chunks, then the receiver must also read data 16KB at a time.

In other words, even if the receiver has 15KB of the record in the buffer and is waiting for the last packet to complete the 16KB record, the application can't read it until the entire record is received and the MAC is verified - therein lies our latency problem! If any packet is lost, we will incur a minimum of an additional RTT to retransmit it. Similarly, if the record happens to exceed the current TCP congestion window, then it will take a minimum of two RTTs before the application can process sent data.
Click to expand...

this next section is why ssl_buffer_size option came into being and why Centmin Mod Nginx defaulted to ssl_buffer_size = 1400 bytes (and now 1369 bytes default in 123.09beta01)

Eliminating TLS latency
The larger the TLS record size, the higher the likelihood that we may incur an additional roundtrip due to a TCP retransmission or "overflow" of the congestion window. That said, the fix is also relatively simple: send smaller records. In fact, to eliminate this problem entirely, configure your TLS record size to fit into a single TCP segment.

If the TCP congestion window is small (i.e. during slow-start), or if we're sending interactive data that should be processed as soon as possible (in other words, most HTTP traffic), then small record size helps mitigate costly latency overhead of yet another layer of buffering.
Click to expand...

buik · Jun 11, 2016

Next stop for Cloudflare, making there Nginx Http/2 server push patch open source.
Which they obviously not going to do.
Or about a year or so if it's already in Nginx.

eva2000 · Jun 18, 2016

rikske said: ↑

Next stop for Cloudflare, making there Nginx Http/2 server push patch open source.
Which they obviously not going to do.
Or about a year or so if it's already in Nginx.
Click to expand...

yeah would be great to get hold of Nginx HTTP/2 server push patches. Though from testing Cloudflare's HTTP/2 server push implementation so far it ain't working that well for alot of folks. So don't mind them taking time to iron out the bugs first

buik · Jun 18, 2016

eva2000 said: ↑

yeah would be great to get hold of Nginx HTTP/2 server push patches. Though from testing Cloudflare's HTTP/2 server push implementation so far it ain't working that well for alot of folks. So don't mind them taking time to iron out the bugs first
Click to expand...

Hhmwha for something as server push, it is always important to look at each case specific.

There are 100 thousands and more of different websites and, therefore, enough potential problems.
So that it does not work for everyone seems logical to me.

They can release the patch, however.
eva2000 get everything to work.

buik · Jun 28, 2016

Found an interesting comment if you want to optimize the TLS patch even further.
Optimizing TLS over TCP to reduce latency

eva2000 · Jun 29, 2016

rikske said: ↑

Found an interesting comment if you want to optimize the TLS patch even further.
Optimizing TLS over TCP to reduce latency
Click to expand...

nice

2 suggestions how the code could be improved:

1) The spdy_chunk_size/http2_chunk_size could be changed automatically whenever the ssl_dyn_rec_size changes. So that a SPDY or HTTP2 chunk fits into a TLS record, whether the record is at its minimum size or already has been expanded.

2) The ssl_dyn_rec_size_lo/ssl_dyn_rec_size_hi values could be automatically adjusted based on the SSL cipher being used. Different ciphers have different TLS max record sizes. Chacha e.g. only has 29 bytes overhead (which is only ~ half of the 61 bytes from the calculation above).
The nginx configuration could look like:
ssl_dyn_rec_size_lo auto;
ssl_dyn_rec_size_hi auto;
ssl_dyn_rec_size_ip ipv4; #to calculate based on ipv4 header size
Click to expand...

Module ngx_http_v2_module

Syntax: http2_chunk_size size;
Default:
http2_chunk_size 8k;
Context: http, server, location
Sets the maximum size of chunks into which the response body is sliced. A too low value results in higher overhead. A too high value impairs prioritization due to HOL blocking.
Click to expand...

Matt · Feb 24, 2018

Just added this as well one a site, and quite a dramatic result:

Red arrow was the test 30 minutes prior to adding this.

eva2000 · Feb 24, 2018

Very nice .. always good to see real evidence of improvements and not just theory

pamamolf · Feb 24, 2018

Is this the TLS Patch that i have to add to the custom config so i can test it?

eva2000 · Feb 24, 2018

pamamolf said: ↑

Is tgis the TLS Patch that i have to add to the custom config so i can test it?
Click to expand...

Beta Branch - add Nginx Dynamic TLS Cloudflare Patch support 123.09beta01

eva2000 · Feb 24, 2018

Just re-quoting myself to bring to forefont what this TLS Dynamic record sizing patch does

eva2000 said: ↑

default threshold is 40 records

What we do now:
We use a static record size of 4K. This gives a good balance of latency and
throughput.

Optimize latency:
By initialy sending small (1 TCP segment) sized records, we are able to avoid
HoL blocking of the first byte. This means TTFB is sometime lower by a whole
RTT.

Optimizing throughput:
By sending increasingly larger records later in the connection, when HoL is not
a problem, we reduce the overhead of TLS record (29 bytes per record with
GCM/CHACHA-POLY).

Logic:
Start each connection with small records (1369 byte default, change with
ssl_dyn_rec_size_lo). After a given number of records (40, change with
ssl_dyn_rec_threshold) start sending larger records (4229, ssl_dyn_rec_size_lo).
Eventually after the same number of records, start sending the largest records
(ssl_buffer_size).

In case the connection idles for a given amount of time (1s,
ssl_dyn_rec_timeout), the process repeats itself (i.e. begin sending small
records again).
Click to expand...

so the patch starts with 1369 bytes sent for up to 1+40 = 41 records so first 41 x 1369 bytes = 56,129 bytes = 54.813 KB before the patch starts sending the ssl_dync_rec_size_hi 4229 bytes based records for another 40 records = 40 x 4229 = 169,160 bytes (165.19KB) before you fall back to ssl_buffer_size of 16KB for non-Centmin Mod Nginx or 1369 bytes for Centmin Mod Nginx

so 1st stage up to 54.813KB then next 165.19KB so < 220KB (54.813+165.19) before you fall back to ssl_buffer_size of 16KB for non-Centmin Mod Nginx or 1369 bytes for Centmin Mod Nginx
Click to expand...

and from Cloudflare blog Optimizing TLS over TCP to reduce latency

Dynamic TLS Records in NGINX
We modified NGINX to add support for dynamic TLS record sizes and are open sourcing our patch. You can find it here. The patch adds parameters to the NGINX ssl module.

ssl_dyn_rec_size_lo: the TLS record size to start with. Defaults to 1369 bytes (designed to fit the entire record in a single TCP segment: 1369 = 1500 - 40 (IPv6) - 20 (TCP) - 10 (Time) - 61 (Max TLS overhead))

ssl_dyn_rec_size_hi: the TLS record size to grow to. Defaults to 4229 bytes (designed to fit the entire record in 3 TCP segments)

ssl_dyn_rec_threshold: the number of records to send before changing the record size.

Each connection starts with records of the size ssl_dyn_rec_size_lo. After sending ssl_dyn_rec_threshold records the record size is increased to ssl_dyn_rec_size_hi. After sending an additional ssl_dyn_rec_threshold records with size ssl_dyn_rec_size_hi the record size is increased to ssl_buffer_size.

ssl_dyn_rec_timeout: if the connection idles for longer than this time (in seconds) that the TLS record size is reduced to ssl_dyn_rec_size_lo and the logic above is repeated. If this value is set to 0 then dynamic TLS record sizes are disabled and the fixed ssl_buffer_size will be used instead.
Click to expand...

Log in / Register

Forums

Get the most out of your Centmin Mod LEMP stack

Nginx Optimizing TLS over TCP to reduce latency: TLS dynamic record sizing

Revenge Active Member

eva2000 Administrator Staff Member

Revenge Active Member

eva2000 Administrator Staff Member

Revenge Active Member

eva2000 Administrator Staff Member

Revenge Active Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

buik “The best traveler is one without a camera.”

eva2000 Administrator Staff Member

Matt Well-Known Member

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

.

Log in / Register

Useful Searches

Get the most out of your Centmin Mod LEMP stack

Nginx Optimizing TLS over TCP to reduce latency: TLS dynamic record sizing

Revenge Active Member

eva2000 Administrator Staff Member

Revenge Active Member

eva2000 Administrator Staff Member

Revenge Active Member

eva2000 Administrator Staff Member

Revenge Active Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

buik “The best traveler is one without a camera.”

eva2000 Administrator Staff Member

Matt Well-Known Member

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

.