Join the community today
Register Now

[NEW PATCH] Add Facebook's Zstd compression support to Tar

Discussion in 'CentOS, Redhat & Oracle Linux News' started by bassie, Dec 23, 2018.

Tags:
  1. bassie

    bassie Well-Known Member

    1,045
    255
    83
    Apr 29, 2016
    Ratings:
    +748
    Local Time:
    11:08 AM
    The Zstd compression feature from Facebook's developers team is
    available for Tar since 1.30.90 (Git*) alpha.

    As i don't like to use alpha software on enterprise hard- and software.
    I made a simple backport-patch based on the git code.
    This allows you to use Red Hat Enterprise Linux 7's (RHEL) Tar, along with Zstd from
    Extra Packages for Enterprise Linux (EPEL).

    Ps it is always recommended to process custom APPS as RHEL Software Collections (SCL), next to the existing, not adapted APPS.

    Add Facebook's Zstd compression support to Tar - patch
     
    • Informative Informative x 2
  2. eva2000

    eva2000 Administrator Staff Member

    37,710
    8,254
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,710
    Local Time:
    8:08 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
  3. bassie

    bassie Well-Known Member

    1,045
    255
    83
    Apr 29, 2016
    Ratings:
    +748
    Local Time:
    11:08 AM
    Red Hat Enterprise Linux 7 maintained code.

    I have backported various of code lines among other things, as the testing suite is changed in Tar 1.30/alpha.

    In favor of Tar 1.26 plus patches which is used at Red Hat.

    Tested it with TB, TB and TB's of data on a medium size test farm and it works perfectly.
     
    • Like Like x 1
  4. eva2000

    eva2000 Administrator Staff Member

    37,710
    8,254
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,710
    Local Time:
    8:08 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    I see that's for sharing/clarification :)
     
  5. bassie

    bassie Well-Known Member

    1,045
    255
    83
    Apr 29, 2016
    Ratings:
    +748
    Local Time:
    11:08 AM
    Yes. As some may known. I prefer as little code changes as possible.
    Therefore this Zstd patch based on the proven Red Hat EL7, Tar 1.26.

    I only use edgie APPS with notable improvements, if I can't do it otherwise with EL7.
    Just like PHP 7+, TLS 1.3, server push, 0-RTT etc.

    Although the biggest stumbling block remains the consumer connection.
    The most degradation is stuck there, if you look at the entire chain.

    A modern site, without unreasonable concessions (to force speed above customer-experience), does not go faster than 0.3 seconds.
     
  6. eva2000

    eva2000 Administrator Staff Member

    37,710
    8,254
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,710
    Local Time:
    8:08 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    Well looks like tar 1.31 stable was released with zstd support so rebuilt my custom tar RPMs for CentOS 6/7 Beta Branch - CentOS 7.x - Custom Tar Archiver RPM Build With Facebook Zstd Compression Support :)

    Code (Text):
    tar --version
    tar (GNU tar) 1.31
    Copyright (C) 2019 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.
    
    Written by John Gilmore and Jay Fenlason.
    

    Code (Text):
    yum -q info tar-zstd-gcc7
    Installed Packages
    Name        : tar-zstd-gcc7
    Arch        : x86_64
    Version     : 1.31
    Release     : 1.el7
    Size        : 3.5 M
    Repo        : installed
    Summary     : tar 1.31 zstd supported for centminmod.com LEMP stack
    URL         : https://centminmod.com
    License     : unknown
    Description : tar 1.31 zstd supported for centminmod.com LEMP stack
    
     
..