Nginx Nginx 1.23.1 Upgrade issue

looks like broken incomplete openssl 1.1.1q download

Code (Text):

tar (child): /svr-setup/openssl-1.1.1q.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
Assembler messages:
Fatal error: can't create help-dummy.o: No such file or directory
/usr/lib64/libatomic_ops.a found
tar (child): /svr-setup/pcre-8.45.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now

actually more downloads are broken

Code (Text):

tar (child): /svr-setup/srcache-nginx-module_v0.32.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now

Code (Text):

tar (child): /svr-setup/pcre-8.45.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now

check your server's connectivity

Also 124.00stable version is a few commits behind so make sure to run cmupdate first Commits · centminmod/centminmod

 

nginx version wouldn't have affected this but I did notice your /svr-setup directory is ramdisk mounted as tmpfs. This is only temp setup at initial Centmin Mod install time if it detects if you have enough memory and at end of initial install should remove ramdisk /svr-setup version and copy files to disk based /svr-setup version via a unsetramdisk function. Looks like it didn't do this in your case.

Code (Text):

Filesystem     Type      Size  Used Avail Use% Mounted on
devtmpfs       devtmpfs   32G     0   32G   0% /dev
tmpfs          tmpfs      32G     0   32G   0% /dev/shm
tmpfs          tmpfs      32G  2.4G   29G   8% /run
tmpfs          tmpfs      32G     0   32G   0% /sys/fs/cgroup
/dev/md2       xfs       411G  320G   91G  78% /
tmpfs          tmpfs      32G  100K   32G   1% /tmp
/dev/nvme0n1p1 vfat      510M  3.6M  507M   1% /boot/efi
tmpfs          tmpfs     6.3G     0  6.3G   0% /run/user/0
tmpfs          tmpfs     2.2G     0  2.2G   0% /svr-setup

noticed from nginx upgrade log complaining of /svr-setup device busy should that could be related

Code (Text):

tar (child): /svr-setup/srcache-nginx-module_v0.32.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
rm: cannot remove ‘/svr-setup/’: Device or resource busy

You can manually fix this using these commands

Code (Text):

DIR_TMP=/svr-setup
mkdir -p "${DIR_TMP}_disk"
\cp -R ${DIR_TMP}/* "${DIR_TMP}_disk"
umount -l "${DIR_TMP}"
mkdir -p "${DIR_TMP}"
\cp -R ${DIR_TMP}_disk/* "${DIR_TMP}"
rm -rf "${DIR_TMP}_disk"

verify with

Code (Text):

df -hT

 

interesting so the fix I suggested worked right?

 

whoops missing a step so updated above post but for now just

Code (Text):

DIR_TMP=/svr-setup
mkdir -p "${DIR_TMP}"

 

Yeah typo fixed

 

huh? what the?

re-create the /svr-setup directory and then run centmin.sh debug mode via cmifno debug-menuexit command

Code (Text):

cminfo debug-menuexit

should save a debug verbose log of a centmin.sh run and 24 exit at /root/centminlogs/debug-menuexit.txt

Code (Text):

cminfo debug-menuexit

------------------------------------------------------------------
Debugging centmin.sh menu option 24 exit routine starting
Please wait until complete...
------------------------------------------------------------------
Full debug log saved at /root/centminlogs/debug-menuexit.txt

Inspect Yum Check Times In debug-menuexit.txt log

2022-09-01 19:49:48 + echo ' checking for YUM updates... please wait...'
2022-09-01 19:49:48  checking for YUM updates... please wait...
2022-09-01 19:49:48 +++ awk '/Updated: / {print $1,$2,$3}' /var/log/yum.log
2022-09-01 19:49:48 +++ tail -1
2022-09-01 19:49:48 ++ date -d 'Aug 31 19:47:30' +%s
2022-09-01 19:49:48 + LASTYUMCHECK_EPOCHDATE=1661975250
2022-09-01 19:49:48 ++ date +%s

Yum Check Duration: 6.00 seconds

------------------------------------------------------------------
Debugging centmin.sh menu option 24 exit routine completed
------------------------------------------------------------------

sanitize and remove sensitive info from /root/centminlogs/debug-menuexit.txt and then share that log here or via PM

 

As far as I can tell nothing running centmin.sh will delete /svr-setup from egrep filer

Code (Text):

cat /root/centminlogs/debug-menuexit.txt | egrep -C1 'svr-setup|DIR_TMP'
2022-09-01 19:54:25 + HN=hostname
2022-09-01 19:54:25 + DIR_TMP=/svr-setup
2022-09-01 19:54:25 + CENTMINLOGDIR=/root/centminlogs
--
2022-09-01 19:54:25 + TMPFSLIMIT=4000000
2022-09-01 19:54:25 + '[' '!' -d /svr-setup ']'
2022-09-01 19:54:25 ++ cat /etc/resolv.conf
--
2022-09-01 19:54:25 + [[ -f /usr/local/bin/ccache ]]
2022-09-01 19:54:25 + [[ ! -f /svr-setup/ccache-4.5.1.tar.gz ]]
2022-09-01 19:54:25 + [[ -f /usr/local/bin/ccache ]]
2022-09-01 19:54:25 + [[ -d /svr-setup/ccache-4.5.1 ]]
2022-09-01 19:54:25 ++ ccache -V
--
2022-09-01 19:54:26 + [[ -f /usr/local/bin/axel ]]
2022-09-01 19:54:26 + [[ ! -d /svr-setup/axel-2.16.1 ]]
2022-09-01 19:54:26 + fileperm_fixes
--
2022-09-01 19:54:26 + ngxmodule_cleanups
2022-09-01 19:54:26 + '[' -f /svr-setup/nginx-develkit_v0.2.19.tar.gz ']'
2022-09-01 19:54:26 + '[' -d /svr-setup/ngx_devel_kit-0.2.19 ']'
2022-09-01 19:54:26 + wpcompfix

 

There's nothing in the log that shows why /svr-setup is being deleted.

You can install auditd via tools/auditd.sh https://community.centminmod.com/th...td-support-added-in-latest-123-09beta01.9071/

Then setup an auditd rule for /svr-setup directory like https://community.centminmod.com/th...added-in-latest-123-09beta01.9071/#post-37761

create and add file /etc/audit/rules.d/svr-setup.rules file with the rule below added setting auditd key = svrsetup

Code (Text):

-w /svr-setup -p wa -k svrsetup

then run

Code (Text):

cd /usr/local/src/centminmod/tools
./auditd.sh updaterules

then re-create /svr-setup

Code (Text):

mkdir -p /svr-setup

then run centmin.sh menu and exit or your centmin.sh menu option 4 and exit

then check auditd rule key svrsetup with command

Code (Text):

ausearch -k svrsetup

and separate saved output for command

Code (Text):

ausearch -k svrsetup | aureport -f

and then post output to a pastebin link as it may contain a lot of entries

 

Good to hear it's resolved. Though was a strange issue!