Beta Branch add optional Cloudflare zlib performance fork support in 123.09beta01

add optional Cloudflare zlib performance fork support in 123.09beta01

- for nginx compiles when you set in persistent config file /etc/centminmod/custom_config.inc variable CLOUDFLARE_ZLIB='y' (disabled by default) + recompile nginx via centmin.sh menu option 4, you can optionally choose using Cloudflare zlib performance fork which is reversioned from 1.2.8 to 1.3.0 in centmin mod as opposed to standard stock zlib 1.2.11 version GitHub - cloudflare/zlib: Cloudflare fork of zlib with massive performance improvements zlib-cloudflare
- Fighting Cancer: The Unexpected Benefit Of Open Sourcing Our Code
- benchmarks Zlib Benchmarks
- Nginx - [PATCH] Fix error when building a Nginx source RPM with custom Zlib - Nginx 1.13.7
- only test on test vps servers and not live servers for now as the long term issues of using Cloudflare zlib fork is unknow for Centmin Mod Nginx server environments.

---

Continue reading...

123.09beta01 branch

• Branch: GitHub - centminmod/centminmod at 123.09beta01
• Commit History: Commits · centminmod/centminmod · GitHub

 

CentOS 7.4 64bit examples with default Centmin Mod 123.09beta01's Nginx standard zlib 1.2.11 compilation

CentOS 7.4 64bit examples with default Centmin Mod 123.09beta01's Nginx with Cloudflare zlib performance fork

 

Just heads up DO NOT try this option on live server. It may have an issue breaking the server Nginx - [PATCH] Fix error when building a Nginx source RPM with custom Zlib - Nginx 1.13.7.

Until I find the cause/solution. Just don't enable this optional option yet as it's disabled by default.

Updated Solution

edit 2: patch no longer required as solution zlib workaround committed to 123.09beta01 branch code now

edit: seems solution is to not install custom zlib as a system shared library but just allow nginx binary to use custom zlib version and revert/switch back all other system services from custom zlib version to system zlib provided version. This means for next one off nginx recompile via centmin.sh menu option 4, an automated restart of affected services is required - which would include MariaDB mysql server, php-fpm, memcached etc (only for services that are running of course). I haven't pushed this update to 123.09beta01 github branch yet as requires more testing. For folks who want to help test on their own test servers, you can apply the changes via a patch located here.

To apply patch, use the following commands in SSH

Code (Text):

cd /usr/local/src/centminmod
git stash
git pull
wget https://gist.github.com/centminmod/500c8cbc468000f5b2d968d9ebd61e0a/raw/removezlib-patch.diff
patch -p1 < removezlib-patch.diff
./centmin.sh

then run centmin.sh menu option 4 to recompile nginx specifying nginx version you want i.e. 1.13.7

a once off message will show mentioning the need to restart services

Code (Text):

--------------------------------------------------------
     Centmin Mod Menu 123.09beta01 centminmod.com    
--------------------------------------------------------
1).  Centmin Install
2).  Add Nginx vhost domain
3).  NSD setup domain name DNS
4).  Nginx Upgrade / Downgrade
5).  PHP Upgrade / Downgrade
6).  XCache Re-install
7).  APC Cache Re-install
8).  XCache Install
9).  APC Cache Install
10). Memcached Server Re-install
11). MariaDB MySQL Upgrade & Management
12). Zend OpCache Install/Re-install
13). Install/Reinstall Redis PHP Extension
14). SELinux disable
15). Install/Reinstall ImagicK PHP Extension
16). Change SSHD Port Number
17). Multi-thread compression: pigz,pbzip2,lbzip2...
18). Suhosin PHP Extension install
19). Install FFMPEG and FFMPEG PHP Extension
20). NSD Install/Re-Install
21). Update - Nginx + PHP-FPM + Siege
22). Add Wordpress Nginx vhost + Cache Plugin
23). Update Centmin Mod Code Base
24). Exit
--------------------------------------------------------
Enter option [ 1 - 24 ] 4
--------------------------------------------------------

Code (Text):

Do you want to run YUM install checks ?  [y/n]

This will increase your upgrade duration time wise.
Check the change log centminmod.com/changelog.html
to see if any Nginx or PHP related new additions
which require checking YUM prequisites are met.
If no new additions made, you can skip the
YUM install check to speed up upgrade time.

 [y/n]: n
**********************************************************************
* Nginx Update script - Included in Centmin Extras
* Version: 1.2.3-eva2000.09.009 - Date: 31/12/2017 - Copyright 2011-2017 CentminMod.com
**********************************************************************

This software comes with no warranty of any kind. You are free to use
it for both personal and commercial use as licensed under the GPL.

Code (Text):

Upgrade Note:
This nginx recompile involves zlib changes which have one time
service restarts including MariaDB MySQL restart to switch
from custom zlib shared library back to system zlib library
for services other than Nginx. Nginx will use custom zlib
by itself while all other services will revert back to system
zlib shared library

Nginx Upgrade - Would you like to continue? [y/n] y

Install which version of Nginx? (version i.e. type 1.13.7): 1.13.7

if successful, the lsof command below should report the system shared zlib 1.2.7 version for nearly all services - some might not report 1.2.7 and may require a server reboot

Code (Text):

lsof | grep libz.so

/usr/lib64/libz.so.1.2.7 is yum system provided zlib 1.2.7 on CentOS 7. On CentOS 6 zlib system provided version is zlib 1.2.3. Nginx custom zlib is compiled into /usr/local/sbin/nginx binary itself so won't show here.

Code (Text):

lsof | grep libz.so                                    
systemd       1           root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
lvmetad     599           root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
polkitd     724        polkitd  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
gmain       724  734   polkitd  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
gdbus       724  736   polkitd  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
JS          724  738   polkitd  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
JS          724  740   polkitd  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
polkitd     724  745   polkitd  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
rsyslogd    743           root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
in:imjour   743  752      root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
rs:main     743  753      root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
chronyd     767         chrony  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1028           root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1029           root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1031           root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
sshd       1032           root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1036           root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
tuned      1037           root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
gmain      1037 1299      root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
tuned      1037 1307      root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
tuned      1037 1309      root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
tuned      1037 1315      root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
nginx      1139           root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
nginx      1140          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
nginx      1141          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
nginx      1142          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
nginx      1144          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469          mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1624     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1625     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1653     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1654     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1655     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1656     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1657     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1658     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1659     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1660     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1661     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1662     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1663     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1664     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1665     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1666     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1710     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1711     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1712     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1713     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1714     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1715     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1716     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1717     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1718     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1719     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1720     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1721     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1722     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1723     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1724     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
mysqld     1469 1725     mysql  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
master     1511           root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
qmgr       1514        postfix  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1520           root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1553          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1554          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1555          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1556          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1558          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
memcached  1560      memcached  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
memcached  1560 1561 memcached  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
memcached  1560 1562 memcached  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
memcached  1560 1563 memcached  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
memcached  1560 1564 memcached  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
memcached  1560 1567 memcached  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
memcached  1560 1569 memcached  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
memcached  1560 1570 memcached  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
memcached  1560 1571 memcached  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
memcached  1560 1572 memcached  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1587          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1588          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1590          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1591          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1592          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1593          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1594          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1595          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1596          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1597          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1602          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1604          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1605          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1607          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
php-fpm    1608          nginx  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
pure-ftpd  5540           root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
systemd-u  8877           root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
pickup     8997        postfix  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7
sshd      10167           root  mem       REG                9,2      90664   12587754 /usr/lib64/libz.so.1.2.7

 

FYI, i will be pushing the zlib workaround update mentioned in previous post to 123.09beta01 branch within next 12-24hrs.

 

• nginx = zlib 1.2.11 or cloudflare zlib 1.2.8
• rest of system services = /usr/lib64/libz.so.1.2.7 is yum system provided zlib 1.2.7 on CentOS 7. On CentOS 6 zlib system provided version is zlib 1.2.3

 

FYI, workaround zlib code outlined in post #2 above has been committed to 123.09beta01 branch code - you can see the commits dated Decemeber 15th & 16th at Commits · centminmod/centminmod · GitHub

 

Full details for Nginx zlib routine update at 123.09beta01 Updated Nginx zlib routine with optional Cloudflare zlib support

 

You using default clang 3.4.2 on centos 6 ? ~20hrs ago I updated 123.09beta01 for a workaround fix for when clang 3.4.2 is used with CLOUDFLARE_ZLIB='y' at Beta Branch - update inc/zlib.inc. You using latest 123.09beta01 code ? try running update via command

Code (Text):

cmupdate

to see if there's any updates to 123.09beta01 code at /usr/local/src/centminmod

 

what have you set in persistent config file at /etc/centminmod/custom_config.inc ?

Code (Text):

cat /etc/centminmod/custom_config.inc

To troubleshoot, you need to check the the nginx upgrade log at /root/centminlogs and instructions under Sharing logs and errors heading for using Pastebin.com or Gists to share a sanitised version of the contents of the nginx_upgrade.log log. You can see full details at How to troubleshoot Centmin Mod initial install issues

also what's output of

Code (Text):

nginx -t

when you run centmin.sh menu option 4 there's a nginx upgrade log timestamped at /root/centminlogs

if you type this command it lists all logs in date ascending order so latest log at bottom

Code (Text):

ls -lArt /root/centminlogs

so copy the entire contents of latest nginx_upgrade log to gist.github.com or pastebin.com

you can use grep to filter the logs, i.e. look for nginx_upgrade in log name

Code (Text):

ls -lahrt /root/centminlogs/ | grep nginx_upgrade
-rw-r--r--  1 root root 3.2M Oct 11 15:55 centminmod_1.2.3-eva2000.09.001_111016-155345_nginx_upgrade.log
-rw-r--r--  1 root root 672K Oct 11 22:06 centminmod_1.2.3-eva2000.09.001_111016-220515_nginx_upgrade.log

So the last nginx upgrade log was named centminmod_1.2.3-eva2000.09.001_111016-220515_nginx_upgrade.log and located at /root/centminlogs/centminmod_1.2.3-eva2000.09.001_111016-220515_nginx_upgrade.log

then use cat command to output the contents of that log

clear your ssh window buffer/screen and type

Code (Text):

cat /root/centminlogs/centminmod_1.2.3-eva2000.09.001_111016-220515_nginx_upgrade.log

then select and copy and paste output to pastebin.com or gist.github.com file to share. If your SSH client's scroll buffer isn't large enough using cat might not output the entire log file contents, so you may need to download the log and use local text editor to open and copy and paste.

so only need content of one specific log, in this case most recent nginx_upgrade.log log

also be sure your centos 6 server was previously addressed for zlib workaround issue. I ran into it on my test centos 6 virtualbox server which I hadn't done the nginx recompile + ccache recompile workarounds for - see errors i encountered for failed nginx compiles here.

 

@pamamolf got to the bottom your issue with CentOS 6 and CLOUDFLARE_ZLIB='y'. I set the workaround for clang 3.4.2 and CLOUDFLARE_ZLIB='y' to fall back to CLANG_FOUR='y' for clang 4.0.1 if CLANG='y' is set and if CLANG='n' fall back to GCC. But Clang 4 is only available for CentOS 7 systems right now via devtoolset-llvm-7 yum packages. So the fall back didn't work when CLANG='y' set. So Now I will fix it so that on CentOS 6 fallback to GCC if CLOUDFLARE_ZLIB='y' and on CentOS 7 just use CLANG_FOUR='y'.

 

for php zlib best to discuss in it's thread at Beta Branch - add CLOUDFLARE_ZLIBPHP variable to use Cloudflare zlib fork for PHP compiles

anyway i added a few more fixes for php cloudflare zlib.

 

Quick benchmarks done for Centmin Mod Nginx HTTPS based target site for Cloudflare zlib compressed requests versus Google ngx_brotli compressed requests Nginx - Nginx with Cloudflare zlib fork VS nxg_brotli compression level tests

 

yup Cloudflare coded the patch as such