Welcome to Centmin Mod Community
Register Now

Nginx Optimizing TLS over TCP to reduce latency: TLS dynamic record sizing

Discussion in 'Nginx and PHP-FPM news & discussions' started by RoldanLT, Jun 10, 2016.

  1. RoldanLT

    RoldanLT Well-Known Member

    3,829
    929
    113
    May 25, 2014
    Phillipines
    Ratings:
    +1,258
    Local Time:
    7:43 PM
    1.11
    10.2
    • Like Like x 1
  2. eva2000

    eva2000 Administrator Staff Member

    29,000
    6,580
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,770
    Local Time:
    9:43 PM
    Nginx 1.13.x
    MariaDB 5.5
    would only work on OpenSSL 1.0.2 but Centmin Mod supports LibreSSL 2.3 and OpenSSL 1.0.2+. However, LibreSSL is default unless you set in persistent config file /etc/centminmod/custom_config.inc
    Code:
    LIBRESSL_SWITCH='n'
    then run centmin.sh menu option 4 to recompile Nginx to use OpenSSL 1.0.2h and higher instead of LibreSSL 2.3 branch.


    edit: okay typed this before reading - it's a nginx patch not openssl patch doh
     
    Last edited: Jun 11, 2016
    • Like Like x 1
  3. bassie

    bassie Active Member

    494
    104
    43
    Apr 29, 2016
    Ratings:
    +312
    Local Time:
    1:43 PM
  4. eva2000

    eva2000 Administrator Staff Member

    29,000
    6,580
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,770
    Local Time:
    9:43 PM
    Nginx 1.13.x
    MariaDB 5.5
    patch being added as optional option in 123.09beta01 as I type :)
     
    • Like Like x 1
  5. eva2000

    eva2000 Administrator Staff Member

    29,000
    6,580
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,770
    Local Time:
    9:43 PM
    Nginx 1.13.x
    MariaDB 5.5
    centmin mod latest 123.09beta01 update :)
    Code (Text):
    *************************************************
    Nginx Patch Time - 1 seconds delay
    to allow you to patch files
    *************************************************
                 ____  _                    _   __  _               
                / ___|| |  ___   _   _   __| | / _|| |  __ _  _ __  ___
               | |    | | / _ \ | | | | / _` || |_ | | / _` || '__|/ _ \
               | |___ | || (_) || |_| || (_| ||  _|| || (_| || |  |  __/
                \____||_| \___/  \__,_| \__,_||_|  |_| \__,_||_|   \___|
                                                                    
    _   _         _                ____                                   _   
    | \ | |  __ _ (_) _ __  __  __ |  _ \  _   _  _ __    __ _  _ __ ___  (_)  ___
    |  \| | / _` || || '_ \ \ \/ / | | | || | | || '_ \  / _` || '_ ` _ \ | | / __|
    | |\  || (_| || || | | | >  <  | |_| || |_| || | | || (_| || | | | | || || (__
    |_| \_| \__, ||_||_| |_|/_/\_\ |____/  \__, ||_| |_| \__,_||_| |_| |_||_| \___|
            |___/                          |___/                               
                   _____  _      ____    ____         _         _ 
                  |_   _|| |    / ___|  |  _ \  __ _ | |_  ___ | |__
                    | |  | |    \___ \  | |_) |/ _` || __|/ __|| '_ \
                    | |  | |___  ___) | |  __/| (_| || |_| (__ | | | |
                    |_|  |_____||____/  |_|    \__,_| \__|\___||_| |_|
                                                                  
    ######################################################################
    Patching Nginx for Dynamic TLS Size Support
    ######################################################################
    Cloudflare Nginx Dynamic TLS patch
    https://github.com/cloudflare/sslconfig/raw/master/patches/nginx__dynamic_tls_records.patch
    ######################################################################
    2016-06-10 15:41:07 URL:https://raw.githubusercontent.com/cloudflare/sslconfig/master/patches/nginx__dynamic_tls_records.patch [9696/9696] -> "nginx__dynamic_tls_records.patch" [1]
    patching file src/event/ngx_event_openssl.c
    Hunk #1 succeeded at 1109 (offset 72 lines).
    Hunk #2 succeeded at 1648 (offset 72 lines).
    Hunk #3 succeeded at 1784 (offset 72 lines).
    patching file src/event/ngx_event_openssl.h
    Hunk #1 succeeded at 55 (offset 17 lines).
    Hunk #2 succeeded at 89 (offset 17 lines).
    Hunk #3 succeeded at 102 (offset 17 lines).
    patching file src/http/modules/ngx_http_ssl_module.c
    Hunk #2 succeeded at 568 (offset 1 line).
    Hunk #3 succeeded at 638 (offset 2 lines).
    Hunk #4 succeeded at 844 (offset 17 lines).
    patching file src/http/modules/ngx_http_ssl_module.h
    Hunk #1 succeeded at 57 (offset 1 line).
    


    description from patch itself
    Code (Text):
    +        /* Dynamic record resizing:
    +           We want the initial records to fit into one TCP segment
    +           so we don't get TCP HoL blocking due to TCP Slow Start.
    +           A connection always starts with small records, but after
    +           a given amount of records sent, we make the records larger
    +           to reduce header overhead.
    +           After a connection has idled for a given timeout, begin
    +           the process from the start. The actual parameters are
    +           configurable. If dyn_rec_timeout is 0, we assume dyn_rec is off. */

    Cloudflare blog
     
    Last edited: Jun 11, 2016
    • Informative Informative x 1
  6. eva2000

    eva2000 Administrator Staff Member

    29,000
    6,580
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,770
    Local Time:
    9:43 PM
    Nginx 1.13.x
    MariaDB 5.5
    hmm. not sure if there's much benefit of this patch on Centmin Mod Nginx as I have set ssl_buffer_size at 1400 bytes (1.4KB) instead of nginx 16KB default. And now updated Centmin Mod Nginx to default to 1369 bytes like Cloudflare's ssl_dyn_rec_size_lo value. Trial and error testing will determine if there's any benefits for this Nginx Dynamic TLS size patch :)

    edit: actually i was just thinking of the ssl certificate chain transfer and not actual assets like images etc which for large files would benefit from Nginx Dynamic TLS size patch :D
     
    Last edited: Jun 11, 2016
    • Like Like x 1
  7. eva2000

    eva2000 Administrator Staff Member

    29,000
    6,580
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,770
    Local Time:
    9:43 PM
    Nginx 1.13.x
    MariaDB 5.5
    updated my HTTPS version of my site to latest Centmin Mod 123.09beta01 with Cloudflare Nginx Dynamic TLS Size patch at https://centminmod.com :)
     
  8. bassie

    bassie Active Member

    494
    104
    43
    Apr 29, 2016
    Ratings:
    +312
    Local Time:
    1:43 PM
    Because the configuration is already significantly optimized.
    Profit rate seems minimal to me.

    But as always applies, every little bit helps the total.
    I'm very curious about your test results.
     
  9. eva2000

    eva2000 Administrator Staff Member

    29,000
    6,580
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,770
    Local Time:
    9:43 PM
    Nginx 1.13.x
    MariaDB 5.5
    Did webpagetest.org benchmarks for page speed for my latest Centmin Mod 123.09beta01 with Cloudflare Nginx Dynamic TLS Size patch at https://centminmod.com front index page with 7 runs of each location test for Dulles, San Jose, London and Japan for 5Mbps Cable Speeds.

    Every single test showed a page speed and faster initiate perceived page rendering with Cloudflare's Nginx Dynamic TLS Size patch enabled :cool: And my site is very minimal for graphics and mainly text. So I'd imagine for sites with alot of images, you'd have even much more of a benefit :D

    A big hugs and kisses to Cloudflare for sharing their patches :D

    Speedindex chart is the one I am mainly concerned with for perceived initial page render times and seems Time To First Byte and Time to Start Render also show the same pattern of benefits after Cloudflare Nginx Dynamic TLS patch is applied.

    FYI, for before and after - Centmin Mod 123.09beta01 default ssl_buffer_size is set at 1369 bytes

    cmm-com-wpt-filmstrip-cable-00.png
    cmm-com-wpt-filmstrip-cable-01.png
    cmm-com-wpt-filmstrip-cable-02.png
    cmm-com-wpt-filmstrip-cable-04.png
    cmm-com-wpt-filmstrip-cable-05.png
    cmm-com-wpt-filmstrip-cable-07.png
    cmm-com-wpt-filmstrip-cable-08.png
    cmm-com-wpt-filmstrip-cable-09.png
     
    Last edited: Jun 11, 2016
    • Informative Informative x 2
  10. bassie

    bassie Active Member

    494
    104
    43
    Apr 29, 2016
    Ratings:
    +312
    Local Time:
    1:43 PM
    @eva2000 Did you change the ssl_buffer_size value while testing?
     
  11. eva2000

    eva2000 Administrator Staff Member

    29,000
    6,580
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,770
    Local Time:
    9:43 PM
    Nginx 1.13.x
    MariaDB 5.5
    i set it to new centmin mod 123.09beta01 ssl_buffer_size default of 1369 bytes for those tests
     
  12. bassie

    bassie Active Member

    494
    104
    43
    Apr 29, 2016
    Ratings:
    +312
    Local Time:
    1:43 PM
    Performed the same test without the adaptation to ssl_buffer_size ? If so What are the results? and changes?
     
  13. eva2000

    eva2000 Administrator Staff Member

    29,000
    6,580
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,770
    Local Time:
    9:43 PM
    Nginx 1.13.x
    MariaDB 5.5
    the above are the results of before vs after nginx dynamic tls size patch both with ssl_buffer_size = 1369 bytes. Didn't do tests without ssl_buffer_size specifically set to regular 16384 bytes which non-Centmin Mod Nginx sets to which is never optimal to begin with (without nginx dynamic tls patch)

    default threshold is 40 records
    so the patch starts with 1369 bytes sent for up to 1+40 = 41 records so first 41 x 1369 bytes = 56,129 bytes = 54.813 KB before the patch starts sending the ssl_dync_rec_size_hi 4229 bytes based records for another 40 records = 40 x 4229 = 169,160 bytes (165.19KB) before you fall back to ssl_buffer_size of 16KB for non-Centmin Mod Nginx or 1369 bytes for Centmin Mod Nginx

    so 1st stage up to 54.813KB then next 165.19KB so < 220KB (54.813+165.19) before you fall back to ssl_buffer_size of 16KB for non-Centmin Mod Nginx or 1369 bytes for Centmin Mod Nginx
     
    Last edited: Jun 11, 2016
  14. bassie

    bassie Active Member

    494
    104
    43
    Apr 29, 2016
    Ratings:
    +312
    Local Time:
    1:43 PM
    Are you sure? Why else your question below at the blog of Cloudflare.

     
  15. eva2000

    eva2000 Administrator Staff Member

    29,000
    6,580
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,770
    Local Time:
    9:43 PM
    Nginx 1.13.x
    MariaDB 5.5
    qualified my statement as never optimal without nginx dynamic tls patch hence why i asked what happens after nginx dynamic tls patch on their blog :)

    updated previous post too
     
  16. eva2000

    eva2000 Administrator Staff Member

    29,000
    6,580
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,770
    Local Time:
    9:43 PM
    Nginx 1.13.x
    MariaDB 5.5
    from above calculations you won't hit ssl_buffer_size fallback until an asset is >220KB in size and since none of my site's page assets are >220KB, testing ssl_buffer_size 16KB instead of 1369 bytes wouldn't make any difference ? or under HTTP/2 single TCP stream it wouldn't be per asset ?

    upload_2016-6-11_3-46-13.png

    but data served from centminmod.com only accounts for 279,967 bytes total anyway

    upload_2016-6-11_3-49-28.png
     
  17. eva2000

    eva2000 Administrator Staff Member

    29,000
    6,580
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,770
    Local Time:
    9:43 PM
    Nginx 1.13.x
    MariaDB 5.5
    Additional tests with ssl_buffer_size = 16384 bytes / 16k default versus 1369 bytes.

    latest Centmin Mod 123.09beta01 with Cloudflare Nginx Dynamic TLS Size patch athttps://centminmod.com front index page with 7 runs of each location test for Dulles only

    for my site at least ssl_buffer_size = 1369 bytes with Nginx Dynamic TLS patch is best for Dulles Webpagetest.org tests at least

    but there's enough variance between test runs due to other factors too due to load on webpagetest server end too

    cmm-com-wpt-filmstrip-cable-1369-vs-16384-00.png cmm-com-wpt-filmstrip-cable-1369-vs-16384-01.png
     
  18. Revenge

    Revenge Active Member

    287
    64
    28
    Feb 21, 2016
    Portugal
    Ratings:
    +227
    Local Time:
    12:43 PM
    1.9.x
    10.1.x
    So, for this patch to work we need to use openssl instead of libressl?
     
  19. pamamolf

    pamamolf Well-Known Member

    2,508
    230
    63
    May 31, 2014
    Ratings:
    +391
    Local Time:
    2:43 PM
    Nginx-1.13.x
    MariaDB 10.1.x
    But as default is Libressl and not Openssl how this patch works?
     
  20. eva2000

    eva2000 Administrator Staff Member

    29,000
    6,580
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,770
    Local Time:
    9:43 PM
    Nginx 1.13.x
    MariaDB 5.5
    no no, LibreSSL or OpenSSL will work for this with CentminMod 123.09beta01 as it's an nginx patch not openssl patch