MariaDB Unable To Start during installation

BamaStangGuy · Sep 14, 2017

Just did a completely fresh install and received errors during the installation that mariadb could not start.

Once installation is done MariaDB would still not start

Code:

[root@li40-144 ~]# systemctl -l status mariadb.service
● mariadb.service - MariaDB database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/mariadb.service.d
           └─migrated-from-my.cnf-settings.conf, openfileslimit.conf, protecthome.conf
   Active: failed (Result: exit-code) since Thu 2017-09-14 06:44:06 UTC; 1min 28s ago
  Process: 5626 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=exited, status=1/FAILURE)
  Process: 5389 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= ||   VAR=`/usr/bin/galera_recovery`; [ $? -eq 0 ]   && systemctl set-environment _WSREP_START_POSITION=$VAR || exit 1 (code=exited, status=0/SUCCESS)
  Process: 5387 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
 Main PID: 5626 (code=exited, status=1/FAILURE)
   Status: "MariaDB server is down"

Sep 14 06:44:05 hostname systemd[1]: Starting MariaDB database server...
Sep 14 06:44:06 hostname mysqld[5626]: 2017-09-14  6:44:06 140142445205760 [Note] /usr/sbin/mysqld (mysqld 10.1.26-MariaDB) starting as process 5626 ...
Sep 14 06:44:06 hostname systemd[1]: mariadb.service: main process exited, code=exited, status=1/FAILURE
Sep 14 06:44:06 hostname systemd[1]: Failed to start MariaDB database server.
Sep 14 06:44:06 hostname systemd[1]: Unit mariadb.service entered failed state.
Sep 14 06:44:06 hostname systemd[1]: mariadb.service failed.

BamaStangGuy · Sep 14, 2017

Code:

2017-09-14  6:42:02 140519981660416 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2017-09-14  6:42:02 140519981660416 [Note] InnoDB: The InnoDB memory heap is disabled
2017-09-14  6:42:02 140519981660416 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2017-09-14  6:42:02 140519981660416 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2017-09-14  6:42:02 140519981660416 [Note] InnoDB: Compressed tables use zlib 1.2.7
2017-09-14  6:42:02 140519981660416 [Note] InnoDB: Using Linux native AIO
2017-09-14  6:42:02 140519981660416 [Note] InnoDB: Using SSE crc32 instructions
2017-09-14  6:42:02 140519981660416 [ERROR] mysqld: Can't create/write to file '/home/mysqltmp/ibZV7lGI' (Errcode: 30 "Read-only file system")
2017-09-14 06:42:02 7fcd5b973900  InnoDB: Error: unable to create temporary file; errno: 30
2017-09-14  6:42:02 140519981660416 [ERROR] Plugin 'InnoDB' init function returned error.
2017-09-14  6:42:02 140519981660416 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2017-09-14  6:42:02 140519981660416 [Note] Plugin 'FEEDBACK' is disabled.
2017-09-14  6:42:02 140519981660416 [ERROR] Unknown/unsupported storage engine: InnoDB
2017-09-14  6:42:02 140519981660416 [ERROR] Aborting

2017-09-14  6:44:06 140142445205760 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2017-09-14  6:44:06 140142445205760 [Note] InnoDB: The InnoDB memory heap is disabled
2017-09-14  6:44:06 140142445205760 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2017-09-14  6:44:06 140142445205760 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2017-09-14  6:44:06 140142445205760 [Note] InnoDB: Compressed tables use zlib 1.2.7
2017-09-14  6:44:06 140142445205760 [Note] InnoDB: Using Linux native AIO
2017-09-14  6:44:06 140142445205760 [Note] InnoDB: Using SSE crc32 instructions
2017-09-14  6:44:06 140142445205760 [ERROR] mysqld: Can't create/write to file '/home/mysqltmp/ibKwtXsk' (Errcode: 30 "Read-only file system")
2017-09-14 06:44:06 7f7574aa1900  InnoDB: Error: unable to create temporary file; errno: 30
2017-09-14  6:44:06 140142445205760 [ERROR] Plugin 'InnoDB' init function returned error.
2017-09-14  6:44:06 140142445205760 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2017-09-14  6:44:06 140142445205760 [Note] Plugin 'FEEDBACK' is disabled.
2017-09-14  6:44:06 140142445205760 [ERROR] Unknown/unsupported storage engine: InnoDB
2017-09-14  6:44:06 140142445205760 [ERROR] Aborting

More info:

eva2000 · Sep 14, 2017

What does your /etc/my.cnf look like ? You using MariaDB Galera Cluster ?

Your systemctl status looks different from a working system with a few active processes i.e.

Code (Text):

   Active: failed (Result: exit-code) since Thu 2017-09-14 06:44:06 UTC; 1min 28s ago
  Process: 5626 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=exited, status=1/FAILURE)
  Process: 5389 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= ||   VAR=`/usr/bin/galera_recovery`; [ $? -eq 0 ]   && systemctl set-environment _WSREP_START_POSITION=$VAR || exit 1 (code=exited, status=0/SUCCESS)
  Process: 5387 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)

on a working system

Code (Text):

systemctl status mariadb | sed -e "s|$(hostname)|hostname|"
* mariadb.service - MariaDB database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/mariadb.service.d
           `-migrated-from-my.cnf-settings.conf, openfileslimit.conf, protecthome.conf
   Active: active (running) since Tue 2017-09-12 11:10:23 UTC; 1 day 23h ago
 Main PID: 15000 (mysqld)
   Status: "Taking your SQL requests now..."
   CGroup: /system.slice/mariadb.service
           `-15000 /usr/sbin/mysqld

Sep 12 11:10:23 hostname systemd[1]: Starting MariaDB database server...
Sep 12 11:10:23 hostname mysqld[15000]: 2017-09-12 11:10:23 140616390711552 [Note] /usr/sbin/mysqld (mysqld 10.1.26-MariaDB) starting as process 15000 ...
Sep 12 11:10:23 hostname systemd[1]: Started MariaDB database server.

what's output for

Code (Text):

ls -lah /home | grep mysqltmp

example

Code (Text):

ls -lah /home | grep mysqltmp    
drwxrwxrwt  2 mysql mysql 4.0K Sep 12 11:10 mysqltmp

and output for command

Code (Text):

journalctl -u mariadb --no-pager

contents of /usr/lib/systemd/system/mariadb.service ?

Code (Text):

cat /usr/lib/systemd/system/mariadb.service

contents of /etc/systemd/system/mariadb.service.d/protecthome.conf

Code (Text):

cat /etc/systemd/system/mariadb.service.d/protecthome.conf

example on working system

Code (Text):

cat /etc/systemd/system/mariadb.service.d/protecthome.conf
[Service]
ProtectHome=false

BamaStangGuy · Sep 14, 2017

All I did was create a brand new Linode instance with Centos 7 and then copy the installer line from
yum -y update; curl -O https://centminmod.com/betainstaller.sh && chmod 0700 betainstaller.sh && bash betainstaller.sh

It simply fails during installation (installer tries to connect to database and can not over and over again.) and then fails to start after installation is done.

I can repeat this on different sized Linodes though the $40 one is the one I need it working on.

I will start up a new one shortly and run some commands before I start the install.

eva2000 · Sep 14, 2017

wonder if it's due to CentOS 7.4 release changes or something centmin mod isn't picking up with CentOS 7.4

after yum update to CentOS 7.4

Code (Text):

cat /etc/redhat-release
CentOS Linux release 7.4.1708 (Core)

looks like it is

Code (Text):

service mariadb restart
Redirecting to /bin/systemctl restart mariadb.service
Job for mariadb.service failed because the control process exited with error code. See "systemctl status mariadb.service" and "journalctl -xe" for details.

Code (Text):

journalctl -u mariadb --no-pager | sed -e "s|$(hostname)|hostname|"
-- Logs begin at Tue 2017-09-12 10:50:18 UTC, end at Thu 2017-09-14 11:35:52 UTC. --
Sep 12 11:00:48 hostname systemd[1]: Starting MariaDB database server...
Sep 12 11:00:49 hostname mysqld[19208]: 2017-09-12 11:00:49 140266854897920 [Note] /usr/sbin/mysqld (mysqld 10.1.26-MariaDB) starting as process 19208 ...
Sep 12 11:00:49 hostname systemd[1]: Started MariaDB database server.
Sep 12 11:01:04 hostname systemd[1]: Stopping MariaDB database server...
Sep 12 11:01:06 hostname systemd[1]: Starting MariaDB database server...
Sep 12 11:01:07 hostname mysqld[20074]: 2017-09-12 11:01:07 139630060120320 [Note] /usr/sbin/mysqld (mysqld 10.1.26-MariaDB) starting as process 20074 ...
Sep 12 11:01:07 hostname systemd[1]: Started MariaDB database server.
Sep 12 11:01:07 hostname systemd[1]: Stopping MariaDB database server...
Sep 12 11:01:10 hostname systemd[1]: Stopped MariaDB database server.
Sep 12 11:01:11 hostname systemd[1]: Starting MariaDB database server...
Sep 12 11:01:12 hostname mysqld[20342]: 2017-09-12 11:01:12 139828499077376 [Note] /usr/sbin/mysqld (mysqld 10.1.26-MariaDB) starting as process 20342 ...
Sep 12 11:01:13 hostname systemd[1]: Started MariaDB database server.
Sep 12 11:01:22 hostname systemd[1]: Stopping MariaDB database server...
Sep 12 11:01:24 hostname systemd[1]: Stopped MariaDB database server.
Sep 12 11:10:23 hostname systemd[1]: Starting MariaDB database server...
Sep 12 11:10:23 hostname mysqld[15000]: 2017-09-12 11:10:23 140616390711552 [Note] /usr/sbin/mysqld (mysqld 10.1.26-MariaDB) starting as process 15000 ...
Sep 12 11:10:23 hostname systemd[1]: Started MariaDB database server.
Sep 14 11:28:10 hostname systemd[1]: Stopping MariaDB database server...
Sep 14 11:28:12 hostname systemd[1]: Starting MariaDB database server...
Sep 14 11:28:13 hostname mysqld[31011]: 2017-09-14 11:28:13 140206596241664 [Note] /usr/sbin/mysqld (mysqld 10.1.26-MariaDB) starting as process 31011 ...
Sep 14 11:28:13 hostname systemd[1]: Started MariaDB database server.
Sep 14 11:35:49 hostname systemd[1]: Stopping MariaDB database server...
Sep 14 11:35:51 hostname systemd[1]: Starting MariaDB database server...
Sep 14 11:35:52 hostname mysqld[12235]: 2017-09-14 11:35:52 140588877760768 [Note] /usr/sbin/mysqld (mysqld 10.1.26-MariaDB) starting as process 12235 ...
Sep 14 11:35:52 hostname mysqld[12235]: 2017-09-14 11:35:52 140588877760768 [Note] InnoDB: Using mutexes to ref count buffer pool pages
Sep 14 11:35:52 hostname mysqld[12235]: 2017-09-14 11:35:52 140588877760768 [Note] InnoDB: The InnoDB memory heap is disabled
Sep 14 11:35:52 hostname mysqld[12235]: 2017-09-14 11:35:52 140588877760768 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
Sep 14 11:35:52 hostname mysqld[12235]: 2017-09-14 11:35:52 140588877760768 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
Sep 14 11:35:52 hostname mysqld[12235]: 2017-09-14 11:35:52 140588877760768 [Note] InnoDB: Compressed tables use zlib 1.2.7
Sep 14 11:35:52 hostname mysqld[12235]: 2017-09-14 11:35:52 140588877760768 [Note] InnoDB: Using Linux native AIO
Sep 14 11:35:52 hostname mysqld[12235]: 2017-09-14 11:35:52 140588877760768 [Note] InnoDB: Using SSE crc32 instructions
Sep 14 11:35:52 hostname mysqld[12235]: 2017-09-14 11:35:52 140588877760768 [ERROR] mysqld: Can't create/write to file '/home/mysqltmp/ibfl1r4q' (Errcode: 30 "Read-only file system")
Sep 14 11:35:52 hostname mysqld[12235]: 2017-09-14 11:35:52 7fdd661e4900  InnoDB: Error: unable to create temporary file; errno: 30
Sep 14 11:35:52 hostname mysqld[12235]: 2017-09-14 11:35:52 140588877760768 [ERROR] Plugin 'InnoDB' init function returned error.
Sep 14 11:35:52 hostname mysqld[12235]: 2017-09-14 11:35:52 140588877760768 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
Sep 14 11:35:52 hostname mysqld[12235]: 2017-09-14 11:35:52 140588877760768 [Note] Plugin 'FEEDBACK' is disabled.
Sep 14 11:35:52 hostname mysqld[12235]: 2017-09-14 11:35:52 140588877760768 [ERROR] Unknown/unsupported storage engine: InnoDB
Sep 14 11:35:52 hostname mysqld[12235]: 2017-09-14 11:35:52 140588877760768 [ERROR] Aborting
Sep 14 11:35:52 hostname systemd[1]: mariadb.service: main process exited, code=exited, status=1/FAILURE
Sep 14 11:35:52 hostname systemd[1]: Failed to start MariaDB database server.
Sep 14 11:35:52 hostname systemd[1]: Unit mariadb.service entered failed state.
Sep 14 11:35:52 hostname systemd[1]: mariadb.service failed.

For now quick fix is comment out tmpdir line with hash # in front in /etc/my.cnf and restart MariaDB MySQL server

Code (Text):

#tmpdir=/home/mysqltmp

Updated fix MariaDB MySQL Not Restarting After CentOS 7.4 Update Fix

eva2000 · Sep 14, 2017

MariaDB MySQL Not Restarting After CentOS 7.4 Update Fix

BamaStangGuy · Sep 14, 2017

Is there anyway we can do this before installing?

eva2000 · Sep 14, 2017

Working on solution now could disable custom tmpdir but that defeats the purpose of it's inclusion as some servers have low /tmp disk free space and won't be able to handle mysql database maintenance routines properly i.e. repairing corrupt database tables etc if /tmp doesn't have the disk free space to accommodate up to 2-3x times the largest mysql database table size of your databases. It's problem you won't experience normally, until you actually need it most i.e. corrupt mysql tables needing repair. That's why i placed MySQL's tmpdir = /home/mysqltmp and setup /etc/systemd/system/mariadb.service.d/protecthome.conf to set ProtectHome=false to disable that systemd security restriction, but seems this is being ignored after CentOS 7.4 update

eva2000 · Sep 14, 2017

updated fix MariaDB MySQL Not Restarting After CentOS 7.4 Update Fix

buik · Sep 14, 2017

@BamaStangGuy Please remove li40-144.members.linod....... from your logs as it resolves to your server. (Bruteforce, SPAM etc prevention)

BamaStangGuy · Sep 14, 2017

bassie said: ↑

@BamaStangGuy Please remove li40-144.members.linod....... from your logs as it resolves to your server. (Bruteforce, SPAM etc prevention)
Click to expand...

It isn't a working Linode anymore (at least not mine as I destroyed it)

eva2000 · Sep 14, 2017

oh i edited your post anyway @BamaStangGuy

eva2000 · Sep 14, 2017

Updated both Centmin Mod 123.08stable and 123.09beta01 builds with permanent workaround fix for MariaDB MySQL server not restarting after CentOS 7.4 updates. The issue is CentOS 7.4 only issue and MariaDB 10.1 I believe as systemd protections didn't get added until MariaDB 10.1. So CentOS 6 and MariaDB 10.0.x users (123.08stable) would probably not experience the issue. Details here MariaDB MySQL Not Restarting After CentOS 7.4 Update Fix

To update your Centmin Mod builds follow instructions at centminmod.com/upgrade.html and respective version threads below:

Centmin Mod 1.2.3-eva2000.08 stable install & update thread

Centmin Mod 123.09beta thread

Example of centmin.sh menu option 23 git updating code.

How to upgrade Centmin Mod + backing up customisations

After updating Centmin Mod, run centmin.sh one more time for fix to auto populate to /etc/systemd/system/mariadb.service.d/protecthome.conf which should now contain
Code (Text):
[Service]
ProtectSystem=false
ProtectHome=false
When you re-run centmin.sh on updated code, you will get a one time message like below:
Code (Text):
./centmin.sh 

Update MariaDB 10 setting for ProtectSystem=false addition

Log in / Register

Forums

Welcome to Centmin Mod Community

MariaDB Unable To Start during installation

BamaStangGuy Active Member

BamaStangGuy Active Member

eva2000 Administrator Staff Member

BamaStangGuy Active Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

BamaStangGuy Active Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

BamaStangGuy Active Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

.

Log in / Register

Useful Searches

Welcome to Centmin Mod Community

MariaDB Unable To Start during installation

BamaStangGuy Active Member

BamaStangGuy Active Member

eva2000 Administrator Staff Member

BamaStangGuy Active Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

BamaStangGuy Active Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

buik “The best traveler is one without a camera.”

BamaStangGuy Active Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

.