Sysadmin AlmaLinux Very strange issue running mariabackup restoration

I've been using mariabackup to do database backups on our larger sites. Think 27GB databases.

The first time I tried to use this tool I ran into a lot of issues... I was moving from a CentOS 7 machine to Alma Linux 9. The mariadb daemon was failing because upon restoration there was a version mismatch and most of the instructions didn't mention that I needed to run mariadb-upgrade. After I figured that out, restoration and daemon restart proceeded.

However, Xenforo wouldn't connect because it would give me the error that it was unable to connect to the database. I could connect from the command line, but Xenforo refused to connect.

I did the nprestart... mysql restart. I even changed the config.php to use root credentials and still the basic can't connect error.

I rebooted the machine and it worked fine. Wow... weird, one off.

Now this has happened on multiple machines after mariadb restoration. I have the process down, everything works... but I literally have to reboot the machine before it works sometimes. I've never seen anything like it in my life.

Hoping someone can do some testing on this. Do a mariabackup on another machine, do a mairabackup --prepare. Copy it to a new machine, shut down mariadb, delete the mariadb, do a mariabackup --copy-back, change the permissions, restart the daemon, restart nginx/phpfpm.... and boom, error.

Try everything to fix it, nothing. Reboot. Boom, works perfect.

This has happened multiple times. This is on VPS and also bare metal. I've never had to reboot a linux box to fix a problem like this before, so it's very strange.

---

 

Yup, there are zero errors. It's like PHP can't connect but doing it from command line with same credentials works fine. Very strange. Even after a php and nginx restart.

Then a reboot, without changing a single thing and it's fixed.