Sysadmin Round 2: Compression Comparison Benchmarks: zstd vs brotli vs pigz vs bzip2 vs xz etc

This is round 2 comparison compression & decompression test benchmarks. You can read up on round 1 benchmarks here and also tar gzip vs tar zstd benchmarks here. In round 2, updated the following versions:

• updated zstd from 1.3.3 to 1.3.5 facebook/zstd
• updated brotli from 1.0.4 to 1.05 google/brotli

Compression Algorithms Tested

• zstd & pzstd v1.3.5 - Facebook developed realtime compression algorithm.here.
• brotli v1.0.5 - Google developed Brotli compression algorithm
• zip v3.0
• gzip v1.5
• bzip2 v1.06
• pigz v2.4 - multi-threaded version of gzip
• pbzip2 v1.1.12 – multi-threaded version of bzip2
• lbzip2 v2.5– multi-threaded version of bzip2
• lzip v1.19 – based on LZMA compression algorithm
• plzip v1.6 – multi-threaded version of lzip
• xz v5.2.2
• pxz v5.2.2 - multi-threaded version of xz

Test Data Files

The test data set was taken from Silesia Compression Corpus zip file here turned into a tar archive for compression tests.

Code (Text):

mkdir -p /home/gziptest/silesia
cd /home/gziptest/silesia
wget http://sun.aei.polsl.pl/~sdeor/corpus/silesia.zip
unzip silesia.zip
cd /home/gziptest
rm -rf /home/gziptest/silesia/silesia.zip
tar -cvf silesia.tar /home/gziptest/silesia

~203 MB tar archive

Code (Text):

ls -la /home/gziptest/silesia.tar
-rw-r--r-- 1 root root 211957760 Sep  3 00:41 /home/gziptest/silesia.tar

Test System Configuration

System:

• OVH MC-32 Intel Core i7 4790K
• 32GB Memory
• 2x240GB SSD
• 250Mbit Network Bandwidth
• CentOS 7.5 64bit
• Centmin Mod 123.09beta01 LEMP stack - Nginx 1.15.1, MariaDB 10.1.34 MySQL, + CSF Firewall
• BHS, Canada

Compression Comparison Results

Below are the comparison results for compression tests with links to the raw data as well.

• 1-compress-test1.md pigz, lz4, zstd, lbzip2 etc 08/07/18

Compression speed = MB/s and compression time = seconds



Charted comparison of compression level speed to compression ratio for pigz vs lbzip2 vs pxz vs zstd vx plzip

---

Compression ratios only