Welcome to Centmin Mod Community
Become a Member

Nginx Optimizing TLS over TCP to reduce latency: TLS dynamic record sizing

Discussion in 'Nginx and PHP-FPM news & discussions' started by rdan, Jun 10, 2016.

  1. rdan

    rdan Well-Known Member

    5,444
    1,408
    113
    May 25, 2014
    Ratings:
    +2,201
    Local Time:
    2:58 AM
    Mainline
    10.2
  2. eva2000

    eva2000 Administrator Staff Member

    54,365
    12,198
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,763
    Local Time:
    4:58 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    would only work on OpenSSL 1.0.2 but Centmin Mod supports LibreSSL 2.3 and OpenSSL 1.0.2+. However, LibreSSL is default unless you set in persistent config file /etc/centminmod/custom_config.inc
    Code:
    LIBRESSL_SWITCH='n'
    then run centmin.sh menu option 4 to recompile Nginx to use OpenSSL 1.0.2h and higher instead of LibreSSL 2.3 branch.


    edit: okay typed this before reading - it's a nginx patch not openssl patch doh
     
    Last edited: Jun 11, 2016
  3. buik

    buik “The best traveler is one without a camera.”

    2,026
    524
    113
    Apr 29, 2016
    Flanders
    Ratings:
    +1,674
    Local Time:
    7:58 PM
  4. eva2000

    eva2000 Administrator Staff Member

    54,365
    12,198
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,763
    Local Time:
    4:58 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    patch being added as optional option in 123.09beta01 as I type :)
     
  5. eva2000

    eva2000 Administrator Staff Member

    54,365
    12,198
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,763
    Local Time:
    4:58 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    centmin mod latest 123.09beta01 update :)
    Code (Text):
    *************************************************
    Nginx Patch Time - 1 seconds delay
    to allow you to patch files
    *************************************************
                 ____  _                    _   __  _               
                / ___|| |  ___   _   _   __| | / _|| |  __ _  _ __  ___
               | |    | | / _ \ | | | | / _` || |_ | | / _` || '__|/ _ \
               | |___ | || (_) || |_| || (_| ||  _|| || (_| || |  |  __/
                \____||_| \___/  \__,_| \__,_||_|  |_| \__,_||_|   \___|
                                                                    
    _   _         _                ____                                   _   
    | \ | |  __ _ (_) _ __  __  __ |  _ \  _   _  _ __    __ _  _ __ ___  (_)  ___
    |  \| | / _` || || '_ \ \ \/ / | | | || | | || '_ \  / _` || '_ ` _ \ | | / __|
    | |\  || (_| || || | | | >  <  | |_| || |_| || | | || (_| || | | | | || || (__
    |_| \_| \__, ||_||_| |_|/_/\_\ |____/  \__, ||_| |_| \__,_||_| |_| |_||_| \___|
            |___/                          |___/                               
                   _____  _      ____    ____         _         _ 
                  |_   _|| |    / ___|  |  _ \  __ _ | |_  ___ | |__
                    | |  | |    \___ \  | |_) |/ _` || __|/ __|| '_ \
                    | |  | |___  ___) | |  __/| (_| || |_| (__ | | | |
                    |_|  |_____||____/  |_|    \__,_| \__|\___||_| |_|
                                                                  
    ######################################################################
    Patching Nginx for Dynamic TLS Size Support
    ######################################################################
    Cloudflare Nginx Dynamic TLS patch
    https://github.com/cloudflare/sslconfig/raw/master/patches/nginx__dynamic_tls_records.patch
    ######################################################################
    2016-06-10 15:41:07 URL:https://raw.githubusercontent.com/cloudflare/sslconfig/master/patches/nginx__dynamic_tls_records.patch [9696/9696] -> "nginx__dynamic_tls_records.patch" [1]
    patching file src/event/ngx_event_openssl.c
    Hunk #1 succeeded at 1109 (offset 72 lines).
    Hunk #2 succeeded at 1648 (offset 72 lines).
    Hunk #3 succeeded at 1784 (offset 72 lines).
    patching file src/event/ngx_event_openssl.h
    Hunk #1 succeeded at 55 (offset 17 lines).
    Hunk #2 succeeded at 89 (offset 17 lines).
    Hunk #3 succeeded at 102 (offset 17 lines).
    patching file src/http/modules/ngx_http_ssl_module.c
    Hunk #2 succeeded at 568 (offset 1 line).
    Hunk #3 succeeded at 638 (offset 2 lines).
    Hunk #4 succeeded at 844 (offset 17 lines).
    patching file src/http/modules/ngx_http_ssl_module.h
    Hunk #1 succeeded at 57 (offset 1 line).
    


    description from patch itself
    Code (Text):
    +        /* Dynamic record resizing:
    +           We want the initial records to fit into one TCP segment
    +           so we don't get TCP HoL blocking due to TCP Slow Start.
    +           A connection always starts with small records, but after
    +           a given amount of records sent, we make the records larger
    +           to reduce header overhead.
    +           After a connection has idled for a given timeout, begin
    +           the process from the start. The actual parameters are
    +           configurable. If dyn_rec_timeout is 0, we assume dyn_rec is off. */

    Cloudflare blog
     
    Last edited: Jun 11, 2016
  6. eva2000

    eva2000 Administrator Staff Member

    54,365
    12,198
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,763
    Local Time:
    4:58 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    hmm. not sure if there's much benefit of this patch on Centmin Mod Nginx as I have set ssl_buffer_size at 1400 bytes (1.4KB) instead of nginx 16KB default. And now updated Centmin Mod Nginx to default to 1369 bytes like Cloudflare's ssl_dyn_rec_size_lo value. Trial and error testing will determine if there's any benefits for this Nginx Dynamic TLS size patch :)

    edit: actually i was just thinking of the ssl certificate chain transfer and not actual assets like images etc which for large files would benefit from Nginx Dynamic TLS size patch :D
     
    Last edited: Jun 11, 2016
  7. eva2000

    eva2000 Administrator Staff Member

    54,365
    12,198
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,763
    Local Time:
    4:58 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    updated my HTTPS version of my site to latest Centmin Mod 123.09beta01 with Cloudflare Nginx Dynamic TLS Size patch at https://centminmod.com :)
     
  8. buik

    buik “The best traveler is one without a camera.”

    2,026
    524
    113
    Apr 29, 2016
    Flanders
    Ratings:
    +1,674
    Local Time:
    7:58 PM
    Because the configuration is already significantly optimized.
    Profit rate seems minimal to me.

    But as always applies, every little bit helps the total.
    I'm very curious about your test results.
     
  9. eva2000

    eva2000 Administrator Staff Member

    54,365
    12,198
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,763
    Local Time:
    4:58 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Did webpagetest.org benchmarks for page speed for my latest Centmin Mod 123.09beta01 with Cloudflare Nginx Dynamic TLS Size patch at https://centminmod.com front index page with 7 runs of each location test for Dulles, San Jose, London and Japan for 5Mbps Cable Speeds.

    Every single test showed a page speed and faster initiate perceived page rendering with Cloudflare's Nginx Dynamic TLS Size patch enabled :cool: And my site is very minimal for graphics and mainly text. So I'd imagine for sites with alot of images, you'd have even much more of a benefit :D

    A big hugs and kisses to Cloudflare for sharing their patches :D

    Speedindex chart is the one I am mainly concerned with for perceived initial page render times and seems Time To First Byte and Time to Start Render also show the same pattern of benefits after Cloudflare Nginx Dynamic TLS patch is applied.

    FYI, for before and after - Centmin Mod 123.09beta01 default ssl_buffer_size is set at 1369 bytes

    cmm-com-wpt-filmstrip-cable-00.png
    cmm-com-wpt-filmstrip-cable-01.png
    cmm-com-wpt-filmstrip-cable-02.png
    cmm-com-wpt-filmstrip-cable-04.png
    cmm-com-wpt-filmstrip-cable-05.png
    cmm-com-wpt-filmstrip-cable-07.png
    cmm-com-wpt-filmstrip-cable-08.png
    cmm-com-wpt-filmstrip-cable-09.png
     
    Last edited: Jun 11, 2016
  10. buik

    buik “The best traveler is one without a camera.”

    2,026
    524
    113
    Apr 29, 2016
    Flanders
    Ratings:
    +1,674
    Local Time:
    7:58 PM
    @eva2000 Did you change the ssl_buffer_size value while testing?
     
  11. eva2000

    eva2000 Administrator Staff Member

    54,365
    12,198
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,763
    Local Time:
    4:58 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    i set it to new centmin mod 123.09beta01 ssl_buffer_size default of 1369 bytes for those tests
     
  12. buik

    buik “The best traveler is one without a camera.”

    2,026
    524
    113
    Apr 29, 2016
    Flanders
    Ratings:
    +1,674
    Local Time:
    7:58 PM
    Performed the same test without the adaptation to ssl_buffer_size ? If so What are the results? and changes?
     
  13. eva2000

    eva2000 Administrator Staff Member

    54,365
    12,198
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,763
    Local Time:
    4:58 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    the above are the results of before vs after nginx dynamic tls size patch both with ssl_buffer_size = 1369 bytes. Didn't do tests without ssl_buffer_size specifically set to regular 16384 bytes which non-Centmin Mod Nginx sets to which is never optimal to begin with (without nginx dynamic tls patch)

    default threshold is 40 records
    so the patch starts with 1369 bytes sent for up to 1+40 = 41 records so first 41 x 1369 bytes = 56,129 bytes = 54.813 KB before the patch starts sending the ssl_dync_rec_size_hi 4229 bytes based records for another 40 records = 40 x 4229 = 169,160 bytes (165.19KB) before you fall back to ssl_buffer_size of 16KB for non-Centmin Mod Nginx or 1369 bytes for Centmin Mod Nginx

    so 1st stage up to 54.813KB then next 165.19KB so < 220KB (54.813+165.19) before you fall back to ssl_buffer_size of 16KB for non-Centmin Mod Nginx or 1369 bytes for Centmin Mod Nginx
     
    Last edited: Jun 11, 2016
  14. buik

    buik “The best traveler is one without a camera.”

    2,026
    524
    113
    Apr 29, 2016
    Flanders
    Ratings:
    +1,674
    Local Time:
    7:58 PM
    Are you sure? Why else your question below at the blog of Cloudflare.

     
  15. eva2000

    eva2000 Administrator Staff Member

    54,365
    12,198
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,763
    Local Time:
    4:58 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    qualified my statement as never optimal without nginx dynamic tls patch hence why i asked what happens after nginx dynamic tls patch on their blog :)

    updated previous post too
     
  16. eva2000

    eva2000 Administrator Staff Member

    54,365
    12,198
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,763
    Local Time:
    4:58 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    from above calculations you won't hit ssl_buffer_size fallback until an asset is >220KB in size and since none of my site's page assets are >220KB, testing ssl_buffer_size 16KB instead of 1369 bytes wouldn't make any difference ? or under HTTP/2 single TCP stream it wouldn't be per asset ?

    upload_2016-6-11_3-46-13.png

    but data served from centminmod.com only accounts for 279,967 bytes total anyway

    upload_2016-6-11_3-49-28.png
     
  17. eva2000

    eva2000 Administrator Staff Member

    54,365
    12,198
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,763
    Local Time:
    4:58 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Additional tests with ssl_buffer_size = 16384 bytes / 16k default versus 1369 bytes.

    latest Centmin Mod 123.09beta01 with Cloudflare Nginx Dynamic TLS Size patch athttps://centminmod.com front index page with 7 runs of each location test for Dulles only

    for my site at least ssl_buffer_size = 1369 bytes with Nginx Dynamic TLS patch is best for Dulles Webpagetest.org tests at least

    but there's enough variance between test runs due to other factors too due to load on webpagetest server end too

    cmm-com-wpt-filmstrip-cable-1369-vs-16384-00.png cmm-com-wpt-filmstrip-cable-1369-vs-16384-01.png
     
  18. Revenge

    Revenge Active Member

    469
    93
    28
    Feb 21, 2016
    Portugal
    Ratings:
    +354
    Local Time:
    6:58 PM
    1.9.x
    10.1.x
    So, for this patch to work we need to use openssl instead of libressl?
     
  19. pamamolf

    pamamolf Premium Member Premium Member

    4,077
    427
    83
    May 31, 2014
    Ratings:
    +833
    Local Time:
    8:58 PM
    Nginx-1.25.x
    MariaDB 10.3.x
    But as default is Libressl and not Openssl how this patch works?
     
  20. eva2000

    eva2000 Administrator Staff Member

    54,365
    12,198
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,763
    Local Time:
    4:58 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    no no, LibreSSL or OpenSSL will work for this with CentminMod 123.09beta01 as it's an nginx patch not openssl patch