Learn about Centmin Mod LEMP Stack today
Register Now

Sysadmin Round 2: Compression Comparison Benchmarks: zstd vs brotli vs pigz vs bzip2 vs xz etc

Discussion in 'System Administration' started by eva2000, Jul 8, 2018.

Thread Status:
Not open for further replies.
  1. eva2000

    eva2000 Administrator Staff Member

    42,268
    9,550
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +14,708
    Local Time:
    10:54 AM
    Nginx 1.17.x
    MariaDB 5.5/10.x
    This is round 2 comparison compression & decompression test benchmarks. You can read up on round 1 benchmarks here and also tar gzip vs tar zstd benchmarks here. In round 2, updated the following versions:

    Compression Algorithms Tested


    Test Data Files



    The test data set was taken from Silesia Compression Corpus zip file here turned into a tar archive for compression tests.
    Code (Text):
    mkdir -p /home/gziptest/silesia
    cd /home/gziptest/silesia
    wget http://sun.aei.polsl.pl/~sdeor/corpus/silesia.zip
    unzip silesia.zip
    cd /home/gziptest
    rm -rf /home/gziptest/silesia/silesia.zip
    tar -cvf silesia.tar /home/gziptest/silesia
    

    ~203 MB tar archive
    Code (Text):
    ls -la /home/gziptest/silesia.tar
    -rw-r--r-- 1 root root 211957760 Sep  3 00:41 /home/gziptest/silesia.tar
    


    Test System Configuration



    System:
    • OVH MC-32 Intel Core i7 4790K
    • 32GB Memory
    • 2x240GB SSD
    • 250Mbit Network Bandwidth
    • CentOS 7.5 64bit
    • Centmin Mod 123.09beta01 LEMP stack - Nginx 1.15.1, MariaDB 10.1.34 MySQL, + CSF Firewall
    • BHS, Canada

    Compression Comparison Results



    Below are the comparison results for compression tests with links to the raw data as well.
    Compression speed = MB/s and compression time = seconds

    compress-test-080718-01.png compress-test-080718-02.png

    Charted comparison of compression level speed to compression ratio for pigz vs lbzip2 vs pxz vs zstd vx plzip

    compress-test-080718-leveltest-chart-01.png

    Compression ratios only

    compress-test-080718-leveltest-chart-02.png
     
    Last edited: Jul 8, 2018
Thread Status:
Not open for further replies.