Learn about Centmin Mod LEMP Stack today
Register Now

Featured Nginx How to use Brotli compression for Centmin Mod Nginx web servers

Discussion in 'Centmin Mod User Tutorials & Guides' started by eva2000, Mar 6, 2017.

  1. eva2000

    eva2000 Administrator Staff Member

    50,460
    11,661
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,082
    Local Time:
    3:39 AM
    Nginx 1.25.x
    MariaDB 10.x
    Centmin Mod is provide as is, so short of scripted related bugs or issues, any further optimisation to the web stack components - nginx, php-fpm, mariadb mysql, csf firewall etc or web app level specific configurations such as wordpress are left to the Centmin Mod user to deal with. So I do not provide any free support for such.

    However, Centmin Mod users are free to help each other out and ask questions or give answers on this community forum. My hopes are that this community forum evolves so that more veteran long time Centmin Mod users help new Centmin Mod users out :)

    But whether gzip or brotli is used doesn't matter that much at Cloudflare level as you're performance will be good either way. So I wouldn't worry too much about it.

     
  2. Nitin

    Nitin New Member

    24
    4
    3
    Apr 30, 2018
    Ratings:
    +4
    Local Time:
    11:09 PM
    ok. I understand. Thanks for the help. Even its not working but i have learned many new thing (y). Salute to your efforts for helping me. You are awesome. I am not going back to shared hosting again..:p
     
  3. Nitin

    Nitin New Member

    24
    4
    3
    Apr 30, 2018
    Ratings:
    +4
    Local Time:
    11:09 PM
    New Update: Cloudflare says "It is currently disabled pending a patch, but should be back online shortly." :LOL:
     
  4. eva2000

    eva2000 Administrator Staff Member

    50,460
    11,661
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,082
    Local Time:
    3:39 AM
    Nginx 1.25.x
    MariaDB 10.x
    You're welcome and yes once you go VPS/dedicated very hard to go back to shared hosting - so much control :D

    :LOL::banghead: so Cloudflare had disabled it on their end !
     
  5. Kuro

    Kuro Member

    95
    12
    8
    Feb 8, 2018
    Ratings:
    +23
    Local Time:
    12:39 AM
    1.15.10
    10.3
    Hi, i'm using CF pro plan, THIS IS NECESSARY or should i use both?
    now i installed both (CF and custom Config) but when i check my homepage. It is Gzip :D
    and when i on TLS 1.3 on my website:
    This site can't provide a secure connection (i must back TLS 1.2)

    [​IMG]

    My custom_config.inc
    [​IMG]


    [​IMG]
     
    Last edited: May 16, 2018
  6. eva2000

    eva2000 Administrator Staff Member

    50,460
    11,661
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,082
    Local Time:
    3:39 AM
    Nginx 1.25.x
    MariaDB 10.x
  7. eva2000

    eva2000 Administrator Staff Member

    50,460
    11,661
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,082
    Local Time:
    3:39 AM
    Nginx 1.25.x
    MariaDB 10.x
  8. rdan

    rdan Well-Known Member

    5,419
    1,386
    113
    May 25, 2014
    Ratings:
    +2,167
    Local Time:
    1:39 AM
    Mainline
    10.2
    How can we check if this thing is being used by Cloudflare or just useless with CF? :unsure:
     
  9. eva2000

    eva2000 Administrator Staff Member

    50,460
    11,661
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,082
    Local Time:
    3:39 AM
    Nginx 1.25.x
    MariaDB 10.x
    Brotli usage where at Cloudflare to Visitor stage or you mean between Cloudflare and Centmin Mod Nginx ? Seems Cloudflare may have turned off Brotli temporary on their servers Nginx - How to use Brotli compression for Centmin Mod Nginx web servers ?

    On CentOS 7 you can use curl resolve command to bypass Cloudflare to check headers for Centmin Mod Nginx backend origin to see if Brotli is supported.

    Inspecting output for commands. For posting code or output from commands to keep the formatting, you might want to use CODE tags for code How to use forum BBCODE code tags
    Code (Text):
    curl -sI -H "Accept-Encoding: gzip,br" https://yourdomain.com
    

    Code (Text):
    curl -sI -H "Accept-Encoding: gzip,br" http://yourdomain.com
    

    checking the Content-Encoding: line to see if it shows gzip or br

    Cloudflare Users & Brotli



    You can verify if cloudflare or centmin mod nginx is where the gzip compression is coming from by running curl command which bypasses cloudflare to connect with centmin mod nginx backend replacing YOURREALSERVER_IP with your real server IP address below and yourdomain.com with your domain name. You may want to mask your real ip when you post the output on this forum to protect your real IP address
    Code (Text):
    curl -sI -H "Accept-Encoding: gzip,br" --resolve 'yourdomain.com:443:YOURREALSERVER_IP' https://yourdomain.com
    

    compare it to header check with cloudflare in front
    Code (Text):
    curl -sI -H "Accept-Encoding: gzip,br" https://yourdomain.com
    

    Inspecting whether Content-Encoding shows br or gzip. If bypass curl header check shows br but cloudflare curl header check shows gzip, then something at web app level i.e. wordpress is precompressing static assets as gzip.

    Cloudflare Origin Authentication Pull And Brotli



    If you have Cloudflare origin authentication setup then curl header bypass command will result in HTTP/1.1 400 Bad Request error as regular HTTPS requests to https backend nginx site are only allowed via Cloudflare, then you need to temporarily disable Cloudflare origin authentication by commenting out the line in your nginx vhost domain config file

    changing line from
    Code (Text):
    ssl_client_certificate /usr/local/nginx/conf/ssl/cloudflare/domain.com/origin.crt;
    ssl_verify_client on;
    

    to comment out and disable them with hash in front
    Code (Text):
    #ssl_client_certificate /usr/local/nginx/conf/ssl/cloudflare/domain.com/origin.crt;
    #ssl_verify_client on;
    

    restart Nginx and PHP-FPM services
    Code (Text):
    nprestart
    

    Then disable Original Authentication Pull in Cloudflare dashboard.

    Then re-run single command line for curl header check that bypasses cloudflare
    Code (Text):
    curl -sI -H"Accept-Encoding: gzip,br" --resolve yourdomain.com:443:YOURREALSERVER_IP' https://yourdomain.com
    

    After you got output, re-enable Cloudflare origin authentication by removing comment hashes from the 2 lines and then restarting nginx and php-fpm services and then re-enable Original Authentication Pull in Cloudflare dashboard.

    For further checks I suppose you can do a custom Nginx log format which includes the content-encoding header being logged to nginx access log similar to Cloudflare logging done for CF-RAY header at Cloudflare - Cloudflare custom Nginx logging
     
  10. buik

    buik “The best traveler is one without a camera.”

    1,860
    492
    83
    Apr 29, 2016
    Flanders
    Ratings:
    +1,567
    Local Time:
    7:39 PM
    Why using Brotli anyway?:)
    Gzip is supported by all for all.

    With Cloudflare Gzip (Zlib), you get the maximum out of it.
    If you use both, you must test both: test updates, compile updates, extra layer for debugging problems over and over again.
     
  11. eva2000

    eva2000 Administrator Staff Member

    50,460
    11,661
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,082
    Local Time:
    3:39 AM
    Nginx 1.25.x
    MariaDB 10.x
    Actually doesn't matter if Brotli isn't used it falls back to Gzip on either Cloudflare or Centmin Mod Nginx layer. So there's no troubleshooting involved - let server decide which to server visitors with - either Gzip or Brotli compression when available.
     
  12. buik

    buik “The best traveler is one without a camera.”

    1,860
    492
    83
    Apr 29, 2016
    Flanders
    Ratings:
    +1,567
    Local Time:
    7:39 PM
    I limit myself to Centminmod. That is what this topic is about.

    True giving the fact that a nice fall back is realized by Nginx Brotli if Brotli isn't supported.
    If all is settled and done.

    Problem is that the Nginx Brotli plugin isn't maintained since 2016.
    There is a fork. But the pre-existence is very uncertain. Given this quote:
     
  13. eva2000

    eva2000 Administrator Staff Member

    50,460
    11,661
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,082
    Local Time:
    3:39 AM
    Nginx 1.25.x
    MariaDB 10.x
    Yeah Centmin Mod Nginx used more updated ngx_brotli fork at eustas/ngx_brotli though the maintainer is also the maintainer for official Google Brotli library too so there would be some updating I suppose :)
     
  14. buik

    buik “The best traveler is one without a camera.”

    1,860
    492
    83
    Apr 29, 2016
    Flanders
    Ratings:
    +1,567
    Local Time:
    7:39 PM
    Did you bench Coudflare Zlib vs Brotli? curious about the results
     
  15. eva2000

    eva2000 Administrator Staff Member

    50,460
    11,661
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,082
    Local Time:
    3:39 AM
    Nginx 1.25.x
    MariaDB 10.x
  16. eva2000

    eva2000 Administrator Staff Member

    50,460
    11,661
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,082
    Local Time:
    3:39 AM
    Nginx 1.25.x
    MariaDB 10.x
    @Nitin @Kuro @RoldanLT looks like Cloudflare has now re-enabled Brotli on their end

    upload_2018-5-18_14-16-25.png

    Code (Text):
    curl -I -H "Accept-Encoding: gzip,br" https://community.centminmod.com/
    HTTP/2 200 
    date: Fri, 18 May 2018 04:18:38 GMT
    content-type: text/html; charset=UTF-8
    set-cookie: __cfduid=dea2e1bcf37bd11ca3ffd795c8b3ec2461526617118; expires=Sat, 18-May-19 04:18:38 GMT; path=/; domain=.centminmod.com; HttpOnly
    vary: Accept-Encoding
    expires: Thu, 19 Nov 1981 08:52:00 GMT
    cache-control: private, max-age=0
    set-cookie: xfcmi_session=20f5c86be55b597512003266b97262a8; path=/; secure; HttpOnly
    x-frame-options: SAMEORIGIN
    x-xss-protection: 1
    x-powered-by: centminmod
    x-content-type-options: nosniff
    referrer-policy: strict-origin-when-cross-origin
    strict-transport-security: max-age=31536000; includeSubdomains
    link: </styles/xenbase/font-awesome/css/font-awesome.min.css>; rel="preload" as="style"
    link: </js/jquery/jquery-1.11.0.min.js>; rel="preload" as="script"
    link: </js/xenforo/xenforo.js>; rel="preload" as="script"
    link: </OneSignalSDK.js>; rel="preload" as="script"
    link: </styles/xenbase/font-awesome/fonts/fontawesome-webfont.woff2?v=4.7.0>; rel="preload" as="font" crossorigin
    expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
    server: cloudflare
    cf-ray: 41cb855e6be41ead-SJC
    content-encoding: br
    
     
  17. rdan

    rdan Well-Known Member

    5,419
    1,386
    113
    May 25, 2014
    Ratings:
    +2,167
    Local Time:
    1:39 AM
    Mainline
    10.2
    I run this on my Ubuntu Desktop and doesn't output anything.
    Code:
    curl -sI -H "Accept-Encoding: gzip,br" --resolve 'yourdomain.com:443:YOURREALSERVER_IP' https://yourdomain.com
    
    This works fine:
    Code:
    curl -sI -H "Accept-Encoding: gzip,br" https://yourdomain.com
    My CURL installed:
    Code:
    curl -V
    curl 7.58.0 (x86_64-pc-linux-gnu) libcurl/7.58.0 OpenSSL/1.1.0g zlib/1.2.11 libidn2/2.0.4 libpsl/0.19.1 (+libidn2/2.0.4) nghttp2/1.30.0 librtmp/2.3
    Release-Date: 2018-01-24
    Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp
    Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL 
     
  18. rdan

    rdan Well-Known Member

    5,419
    1,386
    113
    May 25, 2014
    Ratings:
    +2,167
    Local Time:
    1:39 AM
    Mainline
    10.2
    Yes, between Cloudflare Edge server to/from our CMM Server.
     
  19. rdan

    rdan Well-Known Member

    5,419
    1,386
    113
    May 25, 2014
    Ratings:
    +2,167
    Local Time:
    1:39 AM
    Mainline
    10.2
    Aha :D
    Seems cert error because I use Cloudflare ECC own cert.
     
  20. rdan

    rdan Well-Known Member

    5,419
    1,386
    113
    May 25, 2014
    Ratings:
    +2,167
    Local Time:
    1:39 AM
    Mainline
    10.2
    Works now:
    Code:
    curl -sI -H "Accept-Encoding: gzip,br" -k --resolve 'yourdomain.com:443:YOURREALSERVER_IP' https://yourdomain.com