Want more timely Centmin Mod News Updates?
Become a Member

Zstd -Zstandard v1.4.9

Discussion in 'Other Centmin Mod Installed software' started by pamamolf, Mar 12, 2021.

  1. pamamolf

    pamamolf Premium Member Premium Member

    3,932
    396
    83
    May 31, 2014
    Ratings:
    +768
    Local Time:
    12:37 AM
    Nginx-1.17.x
    MariaDB 10.3.x
    This is an incremental release which includes various improvements and bug-fixes.

    >2x Faster Long Distance Mode
    Long Distance Mode (LDM) --long just got a whole lot faster thanks to optimizations by @mpu in #2483! These optimizations preserve the compression ratio but drastically speed up compression. It is especially noticeable in multithreaded mode, because the long distance match finder is not parallelized. Benchmarking with zstd -T0 -1 --long=31 on an Intel I9-9900K at 3.2 GHz we see:


    File
    v1.4.8 MB/s v1.4.9 MB/s Improvement
    silesia.tar 308 692 125%
    linux-versions* 312 667 114%
    enwik9 294 747 154%
    * linux-versions is a concatenation of the linux 4.0, 5.0, and 5.10 git archives.

    New Experimental Decompression Feature: ZSTD_d_refMultipleDDicts
    If the advanced parameter ZSTD_d_refMultipleDDicts is enabled, then multiple calls to ZSTD_refDDict() will be honored in the corresponding DCtx. Example usage:

    ZSTD_DCtx* dctx = ZSTD_createDCtx();
    ZSTD_DCtx_setParameter(dctx, ZSTD_d_refMultipleDDicts, ZSTD_rmd_refMultipleDDicts);
    ZSTD_DCtx_refDDict(dctx, ddict1);
    ZSTD_DCtx_refDDict(dctx, ddict2);
    ZSTD_DCtx_refDDict(dctx, ddict3);
    ...
    ZSTD_decompress...

    Decompression of multiple frames, each with their own dictID, is now possible with a single ZSTD_decompress call. As long as the dictID from each frame header references one of the dictIDs within the DCtx, then the corresponding dictionary will be used to decompress that particular frame. Note that this feature is disabled with a statically-allocated DCtx.

    Changelog

    It will be good to have this version on Centminmod menu option 17 :)

    Thanks
     
  2. eva2000

    eva2000 Administrator Staff Member

    46,426
    10,551
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +16,371
    Local Time:
    7:37 AM
    Nginx 1.19.x
    MariaDB 5.5/10.x
    That is the plan :D
     
  3. Sanmayce

    Sanmayce New Member

    17
    7
    3
    Jun 9, 2018
    Sofia
    Ratings:
    +22
    Local Time:
    11:37 PM
    Guys, my texttoy is no match to the awesome Zstd, yet it is faster when decompressing 100++MB textual data:

    Textual Decompression Showdown - Zstd v1.4.9 vs Nakamichi vs bzip2 v1.0.8

    Quick stats:
    Nakamichi is 2380% faster than BZIP2 = (5.408-0.218)/0.218*100%
    Zstd is 73% stronger than Nakamichi = (32,595,292-18,813,175)/18,813,175*100%
    Zstd is 44% slower than Nakamichi = (0.315-0.218)/0.218*100%

    Testmachine: laptop 'Compressionette' i5-7200U max turbo 3.1GHz, 36GB DDR4 2133MHz, Fedora 33
    Testfile: glibc-2.32.tar (226,877,440 bytes)

    Code:
    +----------------------------+---------------+-------------------------+----------------+----------------+
    | Decompressor, MAX settings |          Size | Decompression Wall Time |   instructions | insn per cycle |
    +----------------------------+---------------+-------------------------+----------------+----------------+
    | Nakamichi 'Double-Deuce'   |    32,595,292 |               0.218 sec |    366,715,085 |           0.94 |
    | Zstd 1.4.9                 |    18,813,175 |               0.315 sec |  1,355,290,139 |           2.09 |
    | bzip2 1.0.8                |    23,982,240 |               5.408 sec | 17,918,477,508 |           1.09 |
    +----------------------------+---------------+-------------------------+----------------+----------------+
    
    Compression lines:
    /usr/bin/time -v zstd --single-thread --ultra -22 --long=31 --zstd=wlog=31,clog=30,hlog=30,slog=26 "$1" -o "$1.L22W2GB.zst"
    /usr/bin/time -v ./Nakamichi_DD-192.elf "$1" "$1.Nakamichi" 26 123000 i
    /usr/bin/time -v bzip2 -9 -k "$1"
    
    Decompression lines:
    perf stat -d zstd -d --long=31 "$1.L22W2GB.zst" -o /dev/null
    perf stat -d ./Nakamichi_DD-192.elf "$1.Nakamichi" > /dev/null
    cat "$1.bz2" | perf stat -d bzip2 -d -k > /dev/null
    
    +----------------+----------------+----------------+
    | Zstd 1.4.9     | Nakamichi 'DD' | bzip2 1.0.8    |
    +----------------+----------------+----------------+-------------------------+
    |          0.990 |          0.995 |          1.000 | CPUs utilized           |
    |           2.02 |           0.94 |           1.09 | insn per cycle          |
    |         312.01 |         217.46 |       5,407.13 | msec task-clock:u       |
    |  1,368,619,089 |    366,715,085 | 17,918,477,508 | instructions:u          |
    |    155,243,093 |     54,632,574 |  3,526,925,660 | branches:u              |
    |      4,887,072 |      6,490,361 |    127,177,075 | branch-misses:u         |
    |    380,228,246 |     35,730,747 |  4,344,466,576 | L1-dcache-loads:u       |
    |     19,137,410 |      7,682,673 |    187,276,863 | L1-dcache-load-misses:u |
    |      1,628,108 |      3,365,779 |     79,885,744 | LLC-loads:u             |
    |        286,959 |        829,414 |      9,377,915 | LLC-load-misses:u       |
    +----------------+----------------+----------------+-------------------------+
    
     
    Last edited: Mar 14, 2021