Previews addons/borgbackups.sh Premium Members' Preview

While working on cPanel/WHM to Centmin Mod backup/importer migration tool for a paying client, I also started an addons/borgbackups.sh script to use incremental, encrypted and deduplicated backups via borg. So thought I'd share first look preview with Centmin Mod Premium members first. Centmin Mod Premium members will find the addons/borgbackups.sh setup and configuration instructions in the private Centmin Mod Premium members only forum. So make sure to upgrade to Centmin Mod Premium membership if you want access and support

---

Support Options:

• As such any and all support for addons/borgbackups.sh (script itself) will only be available in the private Centmin Mod Premium members only forum started user threads and no support is provided on the public forum.
• Support for using borg command line itself will be up to end user yourself to read the documentation outlined at Borg Documentation — Borg - Deduplicating Archiver 1.1.10 documentation

Notes

• Only files are backed up, MySQL databases are not backed up, for that use dbbackup.sh which backups up MySQL databases into /home/nginx/domains/yourdomain.com/backup/mysql/ directories. addons/borgbackups.sh by default excludes the backup directory from backups via BORG_EXCLUDE_DIRS variable.
  
  Code (Text):

  # exclude directories from borg backup
  # space separated list
  # https://borgbackup.readthedocs.io/en/stable/usage/create.html
  BORG_EXCLUDE_DIRS='/home/nginx/domains/*/log /home/nginx/domains/*/backup'
  

  
  you can remove that from addons/borgbackups.sh so that MySQL sql backups are included in your borgbackups.sh runs.
  
  Code (Text):

  BORG_EXCLUDE_DIRS='/home/nginx/domains/*/log'

  
  
• Currently addons/borgsbackups.sh has only been tested on CentOS 7 64bit OS systems. No CentOS 6 system testing has been done yet.
• addons/borgbackups.sh install routine will setup an passphrase encrypted borg repo at /home/borgbackups and insert the BORG_PASSPHRASE and BORG_REPO environmental variables into your /root/.bashrc file so when you SSH login you can manage borg commands without passing borg passphrase on command line.
• addons/borgbackups.sh will reserve an additional 4GB of free disk space as safety net for out of disk situations, so expect disk usage to be at minimum 2x times more than data and files being backed up + 4GB. Borg deduplication feature will reduce disk usage though due to only changes being stored in subsequent backup runs.
• By default non site directories are backed up for Centmin Mod LEMP stack server's environment directories that directly Centmin Mod as defined by space separated list of directories defined in BORG_BACKUPTARGETS variable. addons/borgbackups.sh uses zstd compression level 6 by default for creating borg backup repos.
  
  Code (Text):

  BORG_BACKUPTARGETS='/root/.ssh /root/tools /root/centminlogs /usr/local/nginx/conf /usr/local/nginx/conf/conf.d /etc/centminmod'
  

  
  You could add /home/nginx/domains to that list if you want to backup Nginx vhost domain sites too.
  
  Code (Text):

  BORG_BACKUPTARGETS='/home/nginx/domains /root/.ssh /root/tools /root/centminlogs /usr/local/nginx/conf /usr/local/nginx/conf/conf.d /etc/centminmod'
  

Example of borg info output right after installation with empty borg repos.

Code (Text):

borg info     
Repository ID: 97a25e149c89a17ca4d5fb50b10820461b63812eceb77f34b0d3499a1cf5ecec
Location: /home/borgbackups
Encrypted: Yes (repokey BLAKE2b)
Cache: /root/.cache/borg/97a25e149c89a17ca4d5fb50b10820461b63812eceb77f34b0d3499a1cf5ecec
Security dir: /root/.config/borg/security/97a25e149c89a17ca4d5fb50b10820461b63812eceb77f34b0d3499a1cf5ecec
------------------------------------------------------------------------------
                      Original size      Compressed size    Deduplicated size
All archives:                    0 B                  0 B                  0 B

                      Unique chunks         Total chunks
Chunk index:                       0                    0

Backup Usage

Example backup run using wrapper script for borg create commands.

Code (Text):

./borgbackups.sh backup

Code (Text):

./borgbackups.sh backup         
borgbackup for .ssh
------------------------------------------------------------------------------
Archive name: .ssh-080719-131709
Archive fingerprint: 8c03640c4d14b354639c3bd4778d53918ac6d54c05c8beb073fc8c75583ac401
Time (start): Mon, 2019-07-08 13:17:10
Time (end):   Mon, 2019-07-08 13:17:10
Duration: 0.02 seconds
Number of files: 5
Utilization of max. archive size: 0%
------------------------------------------------------------------------------
                      Original size      Compressed size    Deduplicated size
This archive:                3.74 kB              2.60 kB              2.60 kB
All archives:                3.74 kB              2.60 kB              2.60 kB

                      Unique chunks         Total chunks
Chunk index:                       7                    7
------------------------------------------------------------------------------
------------------------------------------------------------------------------
                      Original size      Compressed size    Deduplicated size
Deleted data:                    0 B                  0 B                  0 B
All archives:                3.74 kB              2.60 kB              2.60 kB

                      Unique chunks         Total chunks
Chunk index:                       7                    7
------------------------------------------------------------------------------
borgbackup for tools
------------------------------------------------------------------------------
Archive name: tools-080719-131711
Archive fingerprint: 854837240e494ec06871cda3df6ed3a514da7b0abccd785121abeae31567d86e
Time (start): Mon, 2019-07-08 13:17:11
Time (end):   Mon, 2019-07-08 13:17:40
Duration: 28.47 seconds
Number of files: 1975
Utilization of max. archive size: 0%
------------------------------------------------------------------------------
                      Original size      Compressed size    Deduplicated size
This archive:              569.34 MB            209.25 MB            207.61 MB
All archives:              569.35 MB            209.26 MB            207.61 MB

                      Unique chunks         Total chunks
Chunk index:                    1763                 2129
------------------------------------------------------------------------------
------------------------------------------------------------------------------
                      Original size      Compressed size    Deduplicated size
Deleted data:                    0 B                  0 B                  0 B
All archives:              569.35 MB            209.26 MB            207.61 MB

                      Unique chunks         Total chunks
Chunk index:                    1763                 2129
------------------------------------------------------------------------------
borgbackup for centminlogs
------------------------------------------------------------------------------
Archive name: centminlogs-080719-131741
Archive fingerprint: 51407a0630126fa66e04022020bf7e122478dec717fb965d41393956d467cf2d
Time (start): Mon, 2019-07-08 13:17:42
Time (end):   Mon, 2019-07-08 13:17:43
Duration: 1.42 seconds
Number of files: 1028
Utilization of max. archive size: 0%
------------------------------------------------------------------------------
                      Original size      Compressed size    Deduplicated size
This archive:               94.98 MB              9.01 MB            133.65 kB
All archives:              664.33 MB            218.27 MB            207.75 MB

                      Unique chunks         Total chunks
Chunk index:                    1832                 3119
------------------------------------------------------------------------------
------------------------------------------------------------------------------
                      Original size      Compressed size    Deduplicated size
Deleted data:                    0 B                  0 B                  0 B
All archives:              664.33 MB            218.27 MB            207.75 MB

                      Unique chunks         Total chunks
Chunk index:                    1832                 3119
------------------------------------------------------------------------------
borgbackup for conf
------------------------------------------------------------------------------
Archive name: conf-080719-131744
Archive fingerprint: 7cdcbd951dc099c2a4483f3c64163427c0c1288a0c93b8401d29441a74594801
Time (start): Mon, 2019-07-08 13:17:45
Time (end):   Mon, 2019-07-08 13:17:45
Duration: 0.26 seconds
Number of files: 239
Utilization of max. archive size: 0%
------------------------------------------------------------------------------
                      Original size      Compressed size    Deduplicated size
This archive:              678.10 kB            223.70 kB            147.82 kB
All archives:              665.01 MB            218.49 MB            207.89 MB

                      Unique chunks         Total chunks
Chunk index:                    1976                 3334
------------------------------------------------------------------------------
------------------------------------------------------------------------------
                      Original size      Compressed size    Deduplicated size
Deleted data:                    0 B                  0 B                  0 B
All archives:              665.01 MB            218.49 MB            207.89 MB

                      Unique chunks         Total chunks
Chunk index:                    1976                 3334
------------------------------------------------------------------------------
borgbackup for conf.d
------------------------------------------------------------------------------
Archive name: conf.d-080719-131746
Archive fingerprint: 9d9bbd04f2514265b7317808a2104610da1d3f1f4fb9a1c1277fec9a11412f2e
Time (start): Mon, 2019-07-08 13:17:46
Time (end):   Mon, 2019-07-08 13:17:46
Duration: 0.04 seconds
Number of files: 13
Utilization of max. archive size: 0%
------------------------------------------------------------------------------
                      Original size      Compressed size    Deduplicated size
This archive:               49.70 kB             18.12 kB              1.81 kB
All archives:              665.06 MB            218.51 MB            207.90 MB

                      Unique chunks         Total chunks
Chunk index:                    1978                 3349
------------------------------------------------------------------------------
------------------------------------------------------------------------------
                      Original size      Compressed size    Deduplicated size
Deleted data:                    0 B                  0 B                  0 B
All archives:              665.06 MB            218.51 MB            207.90 MB

                      Unique chunks         Total chunks
Chunk index:                    1978                 3349
------------------------------------------------------------------------------
borgbackup for centminmod
------------------------------------------------------------------------------
Archive name: centminmod-080719-131747
Archive fingerprint: fe7b9ede09237873f7fd90bd516017b75345d55539dc70e2e963981ef836664f
Time (start): Mon, 2019-07-08 13:17:48
Time (end):   Mon, 2019-07-08 13:17:48
Duration: 0.11 seconds
Number of files: 74
Utilization of max. archive size: 0%
------------------------------------------------------------------------------
                      Original size      Compressed size    Deduplicated size
This archive:               44.45 kB             24.81 kB              8.05 kB
All archives:              665.10 MB            218.53 MB            207.90 MB

                      Unique chunks         Total chunks
Chunk index:                    2002                 3425
------------------------------------------------------------------------------
------------------------------------------------------------------------------
                      Original size      Compressed size    Deduplicated size
Deleted data:                    0 B                  0 B                  0 B
All archives:              665.10 MB            218.53 MB            207.90 MB

                      Unique chunks         Total chunks
Chunk index:                    2002                 3425
------------------------------------------------------------------------------
.ssh-080719-131709                   Mon, 2019-07-08 13:17:10 [8c03640c4d14b354639c3bd4778d53918ac6d54c05c8beb073fc8c75583ac401]
tools-080719-131711                  Mon, 2019-07-08 13:17:11 [854837240e494ec06871cda3df6ed3a514da7b0abccd785121abeae31567d86e]
centminlogs-080719-131741            Mon, 2019-07-08 13:17:42 [51407a0630126fa66e04022020bf7e122478dec717fb965d41393956d467cf2d]
conf-080719-131744                   Mon, 2019-07-08 13:17:45 [7cdcbd951dc099c2a4483f3c64163427c0c1288a0c93b8401d29441a74594801]
conf.d-080719-131746                 Mon, 2019-07-08 13:17:46 [9d9bbd04f2514265b7317808a2104610da1d3f1f4fb9a1c1277fec9a11412f2e]
centminmod-080719-131747             Mon, 2019-07-08 13:17:48 [fe7b9ede09237873f7fd90bd516017b75345d55539dc70e2e963981ef836664f]

Ending output lists the borg repo names created

Code (Text):

.ssh-080719-131709                   Mon, 2019-07-08 13:17:10 [8c03640c4d14b354639c3bd4778d53918ac6d54c05c8beb073fc8c75583ac401]
tools-080719-131711                  Mon, 2019-07-08 13:17:11 [854837240e494ec06871cda3df6ed3a514da7b0abccd785121abeae31567d86e]
centminlogs-080719-131741            Mon, 2019-07-08 13:17:42 [51407a0630126fa66e04022020bf7e122478dec717fb965d41393956d467cf2d]
conf-080719-131744                   Mon, 2019-07-08 13:17:45 [7cdcbd951dc099c2a4483f3c64163427c0c1288a0c93b8401d29441a74594801]
conf.d-080719-131746                 Mon, 2019-07-08 13:17:46 [9d9bbd04f2514265b7317808a2104610da1d3f1f4fb9a1c1277fec9a11412f2e]
centminmod-080719-131747             Mon, 2019-07-08 13:17:48 [fe7b9ede09237873f7fd90bd516017b75345d55539dc70e2e963981ef836664f]

Get borg repo info for repo name = .ssh-080719-131709

Code (Text):

borg info ::.ssh-080719-131709

Code (Text):

borg info ::.ssh-080719-131709
Archive name: .ssh-080719-131709
Archive fingerprint: 8c03640c4d14b354639c3bd4778d53918ac6d54c05c8beb073fc8c75583ac401
Comment: .ssh
Hostname: test
Username: root
Time (start): Mon, 2019-07-08 13:17:10
Time (end): Mon, 2019-07-08 13:17:10
Duration: 0.02 seconds
Number of files: 5
Command line: /usr/bin/borg create -e /home/nginx/domains/demodomain.com/log -e /home/nginx/domains/demodomain.com/backup --stats --comment .ssh --compression auto,zstd,6 ::.ssh-080719-131709 /root/.ssh
Utilization of maximum supported archive size: 0%
------------------------------------------------------------------------------
                      Original size      Compressed size    Deduplicated size
This archive:                1.51 kB              1.38 kB              2.60 kB
All archives:              665.10 MB            218.53 MB            207.90 MB

                      Unique chunks         Total chunks
Chunk index:                    2002                 3425

borgbackups.sh retention

borgbackups.sh is written to prune older borg backups according to the following retention schedule

Code (Text):

# borg prune how long to keep backups before removal
BORG_KEEP_HOURLY='24'
BORG_KEEP_DAILY='7'
BORG_KEEP_WEEKLY='4'
BORG_KEEP_MONTHLY='12'

So it will keep

• the last 24 hourly backups if you setup a hourly cronjob schedule
• the last 7 days backups if you setup a daily cronjob schedule
• the last 4 weeks of backups
• the last 12 monthly backups

 

borg extract

Extracting borg repo backups is outlined at borg extract — Borg - Deduplicating Archiver 1.1.10 documentation

borg mount/unmount

You can get direct access to borg repo backup contents using borg mount & borg unmount commands which uses FUSE filesystem which non-OpenVZ systems should be fine with. Alot of OpenVZ VPS though might not work properly with FUSE filesystem support so borg mount may not work on OpenVZ VPSes.

Code (Text):

borg list
.ssh-080719-131709                   Mon, 2019-07-08 13:17:10 [8c03640c4d14b354639c3bd4778d53918ac6d54c05c8beb073fc8c75583ac401]
tools-080719-131711                  Mon, 2019-07-08 13:17:11 [854837240e494ec06871cda3df6ed3a514da7b0abccd785121abeae31567d86e]
centminlogs-080719-131741            Mon, 2019-07-08 13:17:42 [51407a0630126fa66e04022020bf7e122478dec717fb965d41393956d467cf2d]
conf-080719-131744                   Mon, 2019-07-08 13:17:45 [7cdcbd951dc099c2a4483f3c64163427c0c1288a0c93b8401d29441a74594801]
conf.d-080719-131746                 Mon, 2019-07-08 13:17:46 [9d9bbd04f2514265b7317808a2104610da1d3f1f4fb9a1c1277fec9a11412f2e]
centminmod-080719-131747             Mon, 2019-07-08 13:17:48 [fe7b9ede09237873f7fd90bd516017b75345d55539dc70e2e963981ef836664f]

temp mount for borg repo back up for /root/.ssh named = .ssh-080719-131709

Code (Text):

mkdir -p /home/temp-sshd
borg mount ::.ssh-080719-131709 /home/temp-sshd

will return to prompt when done

now you can access .ssh-080719-131709 repo's contents at /home/temp-sshd for /root/.ssh backed up files

Code (Text):

ls -lah /home/temp-sshd/root/.ssh/
total 3.0K
drwx------ 1 root root   0 Jan 18 05:28 .
drwxr-xr-x 1 root root   0 Jul  8 16:43 ..
-rw------- 1 root root 178 Jan 18 05:56 authorized_keys
-rw-r--r-- 1 root root  90 Jan 18 05:24 config
-rw-r--r-- 1 root root 757 Mar 24 19:36 known_hosts
-rw------- 1 root root 399 Jan 18 05:22 my1.key
-rw-r--r-- 1 root root  89 Jan 18 05:22 my1.key.pub

To unmount use borg umount command

Code (Text):

borg umount /home/temp-sshd

no contents to access

Code (Text):

ls -lah /home/temp-sshd/
total 8.0K
drwxr-xr-x   2 root root 4.0K Jul  8 16:42 .
drwxr-xr-x. 20 root root 4.0K Jul  8 16:42 ..