Featured Nginx How to use Brotli compression for Centmin Mod Nginx web servers

Brotli compression for web servers is an alternative to Gzip compression for static files. It is one of the performance configuration tips I listed in my guide at Insight Guide - How to boost Centmin Mod LEMP stack performance. April 18, 2020: ngx_brotli module has an update to fix a 100% cpu load issue. See info on how to update at Nginx Brotli Module Update To Fix 100% CPU Load Issues

You can read about Brotli compression in web server context over on Cloudflare's blog and the current state of web browser adoption at http://caniuse.com/#search=brotli. Google Play Store on a daily basis saves 1.5 petabytes (1.5 million gigabytes) of bandwidth each day using Brotli !

Notes:

• Centmin Mod Nginx Brotli is used for page load speed - better compression means smaller compressed file sizes. But it comes at expense of scalability in terms of handling larger concurrency traffic loads according to benchmarks here. However, this only applies to Nginx Brotli implementation. If you use Cloudflare in front of your server, using Cloudflare's Brotli implementation is faster than Cloudflare zlib/gzip in terms of page load speed + scales just as well so Cloudflare Brotli is a good to have enabled. So for Centmin Mod Nginx you may want to disable ngx_brotli (the default state already) if you have alot of concurrent user traffic and you're hitting server resource limits. Though for page load speed ngx_brotli is better so if you have the server resources to handle the concurrent traffic load and not running against server resource limits, ngx_brotli is still viable option to enable.

How To Use Nginx Brotli Compression

To properly utilise Brotli compression, your web server needs to be able to serve Brotli Content-Encoding headers to only web browsers that support it and your web site needs to serve the web assets, css and js etc over HTTPS with SSL certificate.

For Nginx there is an ngx_brotli module which can be compiled into Nginx to support such. Centmin Mod Nginx uses a more up to date fork of original Google ngx_brotli module maintained and developed by one of the original Google Brotli authors, Eugene Kliuchnikov at eustas/ngx_brotli. However, Eugene recently moved development back to original Google Nginx Brotli Github repo at https://github.com/google/ngx_brotli/ so Centmin Mod Nginx has moved back as well.

Since October 2015, Centmin Mod latest 123.09beta01 branch's Nginx server has built in support for ngx_brotli module that can optionally be enabled or disabled via persistent config file /etc/centminmod/custom_config.inc set variables below:

Code (Text):

NGXDYNAMIC_BROTLI='y'
NGINX_LIBBROTLI='y'

Once variables set, you recompile Centmin Mod Nginx via centmin.sh menu option 4 and will have ngx_brotli support enabled. Centmin Mod Nginx dynamic modules are loaded from /usr/local/nginx/modules directory via an include file /usr/local/nginx/conf/dynamic-modules.conf in /usr/local/nginx/conf/nginx.conf.

centmin.sh menu option 4

Code (Text):

--------------------------------------------------------
     Centmin Mod Menu 123.09beta01 centminmod.com
--------------------------------------------------------
1).  Centmin Install
2).  Add Nginx vhost domain
3).  NSD setup domain name DNS
4).  Nginx Upgrade / Downgrade
5).  PHP Upgrade / Downgrade
6).  XCache Re-install
7).  APC Cache Re-install
8).  XCache Install
9).  APC Cache Install
10). Memcached Server Re-install
11). MariaDB MySQL Upgrade & Management
12). Zend OpCache Install/Re-install
13). Install/Reinstall Redis PHP Extension
14). SELinux disable
15). Install/Reinstall ImagicK PHP Extension
16). Change SSHD Port Number
17). Multi-thread compression: pigz,pbzip2,lbzip2...
18). Suhosin PHP Extension install
19). Install FFMPEG and FFMPEG PHP Extension
20). NSD Install/Re-Install
21). Update - Nginx + PHP-FPM + Siege
22). Add Wordpress Nginx vhost + Cache Plugin
23). Update Centmin Mod Code Base
24). Exit
--------------------------------------------------------
Enter option [ 1 - 24 ]

You can check Centmin Mod Nginx has ngx_brotli enabled via command below to show Nginx's compiled and enabled modules

Code (Text):

nginx -V

Look for the line --add-dynamic-module=../ngx_brotli

With the added update I made, the Nginx version command (nginx -V) will output a build tag which also displays ngx_brotli module's latest commit hash so you can check if you have latest ngx_brotli module updates built for Centmin Mod Nginx binary

Example below where (190420-104720-centos7-5cd76ab-br-bcceaab) is the build tag which now shows data stamp 190420 for April 19, 2020 build AT 10:47:20 AM from Centmin Mod commit 5cd76ab from commit history at centminmod/centminmod and because NGINX_LIBBROTLI='y' was enabled as per instructions, also shows bcceaab which at time of writing is latest commit from ngx_brotli history at google/ngx_brotli. This makes troubleshooting easier as you know the state of the Nginx version built.

In /usr/local/nginx/conf/nginx.conf contains include file to load Nginx dynamic modules

Code (Text):

include /usr/local/nginx/conf/dynamic-modules.conf;

And also contains an include file with actual Nginx Brotli settings

Code (Text):

include /usr/local/nginx/conf/brotli_inc.conf;

Contents of /usr/local/nginx/conf/dynamic-modules.conf will contain Nginx Brotli's dynamic and static modules

Code (Text):

load_module "modules/ngx_http_brotli_filter_module.so";
load_module "modules/ngx_http_brotli_static_module.so";

Contents of /usr/local/nginx/conf/brotli_inc.conf with ngx_brotli settings

Code (Text):

/usr/local/nginx/conf/brotli_inc.conf
brotli on;
brotli_static on;
brotli_min_length 1000;
brotli_buffers 32 8k;
brotli_comp_level 5;
brotli_types text/plain text/css text/xml application/javascript application/x-javascript application/xml application/xml+rss application/ecmascript application/json image/svg+xml;

With Centmin Mod Nginx's Brotli module enabled, this will configure both Brotli on the fly compression as well as support Brotli static file serving if a *.br extension file is detected. To test, you can use curl command with appropate Accept-Encoding directives for gzip and br.

Curl content encoding gzip,br check for Centmin Mod Nginx based server with ngx_brotli enabled. Check for Content-Encoding: br to confirm that Nginx is serving Brotli compressed version of the css or js file.

Code (Text):

curl -sI /dev/null -H"Accept-Encoding: gzip,br" https://domain.com/brotlitest2/bootstrap.min.css
HTTP/1.1 200 OK
Date: Sun, 05 Mar 2017 16:39:04 GMT
Content-Type: text/css
Content-Length: 16149
Last-Modified: Mon, 25 Jul 2016 16:08:01 GMT
Connection: keep-alive
Vary: Accept-Encoding
ETag: "57963961-3f15"
Content-Encoding: br
Server: nginx centminmod
X-Powered-By: centminmod
Expires: Tue, 04 Apr 2017 16:39:04 GMT
Cache-Control: max-age=2592000
Access-Control-Allow-Origin: *
Cache-Control: public, must-revalidate, proxy-revalidate

Curl content encoding gzip check for Centmin Mod Nginx based server with ngx_brotli enabled. Check for Content-Encoding: gzip to confirm that Nginx is serving Gzip compressed version of the css or js file.

Code (Text):

curl -sI /dev/null -H"Accept-Encoding: gzip" https://domain.com/brotlitest2/bootstrap.min.css
HTTP/1.1 200 OK
Date: Sun, 05 Mar 2017 16:40:19 GMT
Content-Type: text/css
Content-Length: 18322
Last-Modified: Mon, 25 Jul 2016 16:08:01 GMT
Connection: keep-alive
Vary: Accept-Encoding
ETag: "57963961-4792"
Content-Encoding: gzip
Server: nginx centminmod
X-Powered-By: centminmod
Expires: Tue, 04 Apr 2017 16:40:19 GMT
Cache-Control: max-age=2592000
Access-Control-Allow-Origin: *
Cache-Control: public, must-revalidate, proxy-revalidate

---

Other Notes

• For Xenforo you may need to add to your library/config.php the setting
  Code (Text):

  $config['enableGzip'] = false;

 

Also posted on Github a brotli.sh tool to generate static brotli (*.br) and gzip (*.gz) assets for css and js files you can play with at https://centminmod.com/brotlistatic

• brotli.sh tool to auto compress css and js files by specifying path on CentOS based Centmin Mod LEMP web stack servers.
• Appending clean flag on end of directory path will remove any *.br or *.gz compressed files. I usually place my tools at /root/tools but you can place brotli.sh whereever you want.
• Appending display flag will find and display all *.br and *.gz compressed css and js files.

Code (Text):

Usage

/root/tools/brotli.sh /path/to/parent/directory
/root/tools/brotli.sh /path/to/parent/directory clean
/root/tools/brotli.sh /path/to/parent/directory display

gzip_static and brotli_static options in Nginx are enabled by default. So when Nginx sees a html, css, js file it wants to serve to visitors, it can either serve and compress on the fly at visitor request time the static file using gzip or brotli (if supported) or with gzip_static and brotli_static enabled look for *.gz or *.br precompressed versions of the static file and serve that instead.

Precompressed static assets can be compressed using higher compression levels to further reduce the size of the static assets and improve page loading performance without the expensive cpu overhead of on the fly compression. You can see dramatic difference in performance if you can safely serve pre-compressed gzip and/or brotli static file assets compared to on the fly gzip or brotli compression in benchmarks posted here and below results table.

config	compressed size	req/s	avg latency	max latency
Centmin Mod Nginx zlib level 5 pre-compress static (cf fork)	3.79KB	72443	3.27ms	32.38ms
Centmin Mod Nginx zlib level 6 pre-compress static (cf fork)	3.79KB	71905	3.36ms	50.43ms
Centmin Mod Nginx brotli level 5 pre-compress static	3.38KB	84643	2.76ms	39.79ms
Centmin Mod Nginx brotli level 6 pre-compress static	3.38KB	84975	2.96ms	87.54ms
Centmin Mod Nginx zlib level 5 dynamic (cf fork)	4.12KB	21906	9.07ms	74.32ms
Centmin Mod Nginx zlib level 6 dynamic (cf fork)	4.00KB	16997	11.67ms	49.06ms
Centmin Mod Nginx brotli level 5 dynamic	3.93KB	5060	38.61ms	167.43ms
Centmin Mod Nginx brotli level 6 dynamic	3.66KB	4875	40.12ms	119.51ms

 

You can also use your Chrome or web browser's developer tools network tab to see Content-Encoding of site's assets i.e. html file index for centminmod.com has Content-Encoding = br for brotli compression. Page is served over HTTP/2 based HTTPS (Protocol = h2).

 

Don't have a HTTP/2 based HTTPS site yet for Brotli ? Centmin Mod Nginx has 2 guides for setting up free Letsencrypt SSL certificates and Centmin Mod Nginx HTTP/2 based HTTPS below:

• Centmin Mod guide for fresh Centmin Mod Nginx HTTP/2 based HTTPS vhost site creation with free Letsencrypt SSL certificates Using Centmin Mod acmetool.sh addon for Nginx HTTP/2 based HTTPS with free Letsencrypt SSL certificates.
• For existing Centmin Mod Nginx HTTP users wishing to migrate from HTTP to HTTPS at Migrating Existing Nginx Vhost From HTTP to HTTP/2 based HTTPS With Letsencrypt SSL Certificates

 

yes .br should only be served to browsers that support it. Though i have one issue related to Sucuri cloudproxy's caching of brotli files and serving to non supported browsers https://community.centminmod.com/threads/site-broken-in-internet-explorer-edge.9701/. So turned off brotli for forums when behind Sucuri cloudproxy.

Yes for on the fly compression if no precompressed .br or .gz files are detected. Otherwise if precompressed .br or .gz files detected, they get served and can get out of date and need recompressing if you precompress files i.e. using brotli.sh from centminmod.com/brotlistatic

Works for me on centminmod.com cloudflare mirror at centmin.sh though not sure why files are served gzip and not brotli there.

 

If you use wordpress plugins that cache and/or pre-gzip compress files, then you'll already have pre-gzipped compressed versions that Centmin Mod Nginx serves via gzip_static directive. Not sure but IIRC pre-gzipped files take priority over on the fly brotli compression. But I believe if both pre-gzipped and pre-brotli compressed files exist, pre-brotli compressed files take priority if browser supports brotli.

Code (Text):

curl -sI /dev/null -H"Accept-Encoding: gzip,br" https://domain.com/brotlitest2/bootstrap.min.css
HTTP/1.1 200 OK
Date: Sun, 05 Mar 2017 16:39:04 GMT
Content-Type: text/css
Content-Length: 16149
Last-Modified: Mon, 25 Jul 2016 16:08:01 GMT
Connection: keep-alive
Vary: Accept-Encoding
ETag: "57963961-3f15"
Content-Encoding: br
Server: nginx centminmod
X-Powered-By: centminmod
Expires: Tue, 04 Apr 2017 16:39:04 GMT
Cache-Control: max-age=2592000
Access-Control-Allow-Origin: *
Cache-Control: public, must-revalidate, proxy-revalidate

 

i see the same for my mirror under Cloudflare

@RB1 and @BamaStangGuy so probably doesn't work with Cloudflare - might need to ask tech support at CF

 

Guess for cloudflare you could turn off gzip and brotli on the backend then if you want