Join the community today
Become a Member

CentOS 7.x Latest upgrade locked me out

Discussion in 'Beta release code' started by Tracy Perry, Dec 13, 2016.

  1. Tracy Perry

    Tracy Perry Active Member

    188
    80
    28
    Aug 24, 2014
    Texas
    Ratings:
    +129
    Local Time:
    6:10 PM
    1.11.5
    MariaDB 10.0.28
    Anyone else have an issue with the latest alert that CentMin gave and then applying the updates locking you out of your server? My keys no longer work and my passwords are no longer accepted, so now I'm totally locked out of my server.
     
  2. Tracy Perry

    Tracy Perry Active Member

    188
    80
    28
    Aug 24, 2014
    Texas
    Ratings:
    +129
    Local Time:
    6:10 PM
    1.11.5
    MariaDB 10.0.28
    In fact, nothing appears to be running - my sites are even down.
     
  3. Tracy Perry

    Tracy Perry Active Member

    188
    80
    28
    Aug 24, 2014
    Texas
    Ratings:
    +129
    Local Time:
    6:10 PM
    1.11.5
    MariaDB 10.0.28
    I've attempted to enter single user mode and using
    Code:
    rw init=/sysroot/bin/sh
    in place of the RO segment, but it returns Failed to start Switch Root. So, looks like I get to reinstall my server - but can't do it today as I have to take my wife to have a heart cath done. :mad:

    Never had this issue EVER with a Debian upgrade.
     
  4. Sunka

    Sunka Active Member

    888
    230
    43
    Oct 31, 2015
    Rijeka, Croatia
    Ratings:
    +376
    Local Time:
    1:10 AM
    Nginx 1.13.3
    MariaDB 10.1.24
    Maybe same problem as here?
     
  5. Tracy Perry

    Tracy Perry Active Member

    188
    80
    28
    Aug 24, 2014
    Texas
    Ratings:
    +129
    Local Time:
    6:10 PM
    1.11.5
    MariaDB 10.0.28
    Don't think so as I can boot into the older kernel - after going into single user mode under one of the older kernels - and updated my password and can log in via IPMI now.

    EDIT:
    As well as my keys being recognized and log in via SSH from my Mac, so it has to do with booting under the new kernel.
     
  6. Tracy Perry

    Tracy Perry Active Member

    188
    80
    28
    Aug 24, 2014
    Texas
    Ratings:
    +129
    Local Time:
    6:10 PM
    1.11.5
    MariaDB 10.0.28
    OK, disable CSF and can log in... so now time to trouble shoot.

    When I look at my yum history info ## I notice this at the bottom

    Code:
    Scriptlet output:
       1 warning: /etc/shadow created as /etc/shadow.rpmnew
       2 warning: /etc/nsswitch.conf created as /etc/nsswitch.conf.rpmnew
       3 warning: /etc/dovecot/conf.d/10-mail.conf created as /etc/dovecot/conf.d/10-mail.conf.rpmnew
       4 warning: /etc/sysctl.conf created as /etc/sysctl.conf.rpmnew
       5 warning: /etc/named.conf created as /etc/named.conf.rpmnew
       6 warning: /etc/selinux/targeted/seusers created as /etc/selinux/targeted/seusers.rpmnew
       7 ‘/etc/selinux/targeted/modules/active/seusers’ -> ‘/etc/selinux/targeted/active/seusers.local’
       8 warning: /etc/sudoers created as /etc/sudoers.rpmnew
       9 warning: /etc/selinux/targeted/modules/active/seusers.final saved as /etc/selinux/targeted/modules/active/seusers.final.rpmsave
      10 warning: /var/lib/logrotate.status saved as /var/lib/logrotate.status.rpmsave
     
  7. Tracy Perry

    Tracy Perry Active Member

    188
    80
    28
    Aug 24, 2014
    Texas
    Ratings:
    +129
    Local Time:
    6:10 PM
    1.11.5
    MariaDB 10.0.28
    OK, looks like I may have figured out how to fix mine. I had to disable CSF with
    Code:
    csf -x
    and then re-enabled it with
    Code:
    csf -e
    and all appears to be back to working.

    firewalld was NOT installed nor running on my setup.
     
  8. eva2000

    eva2000 Administrator Staff Member

    28,942
    6,570
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,751
    Local Time:
    9:10 AM
    Nginx 1.13.x
    MariaDB 5.5
    weird indeed maybe related to CentOS 7.3 release ?

    which updates did you apply ? command used ? was it yum update ? CentOS 7.3 came out CentOS 7.x - Official CentOS 7.3 1611 Release | Centmin Mod Community
     
  9. eva2000

    eva2000 Administrator Staff Member

    28,942
    6,570
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,751
    Local Time:
    9:10 AM
    Nginx 1.13.x
    MariaDB 5.5
    FYI, updated 5x CentOS 7.3 systems so far with Centmin Mod 123.09beta01 none have issues with CSF Firewall to date.

    IIRC, @Tracy Perry did you set custom iptables/ipset blacklists in csfpre.sh script to start before csf firewall ? maybe related ?

    This is on my OVH i7 4790K with CentOS 7.2 to 7.3 update
    Code (Text):
    yum history info 40
    Loaded plugins: fastestmirror, priorities
    Transaction ID : 40
    Begin time     : Tue Dec 13 13:19:57 2016
    Begin rpmdb    : 694:3083ebc912d6b662e7ed90763e025c40a4ab8512
    End time       :            13:20:24 2016 (27 seconds)
    End rpmdb      : 696:ab201b21b57f6032cb9631ccd0232e5f05dbafba
    User           : root <root>
    Return-Code    : Success
    Command Line   : update --disableplugin=priorities --enablerepo=remi
    Transaction performed with:
        Installed     rpm-4.11.3-21.el7.x86_64                      @cr
        Installed     yum-3.4.3-150.el7.centos.noarch               @cr
        Installed     yum-metadata-parser-1.1.4-10.el7.x86_64       @anaconda
        Installed     yum-plugin-fastestmirror-1.1.31-40.el7.noarch @cr
    Packages Altered:
        Obsoleted  ImageMagick-last-6.9.6.7-1.el7.remi.x86_64           @remi
        Obsoleted  ImageMagick-last-c++-6.9.6.7-1.el7.remi.x86_64       @remi
        Obsoleted  ImageMagick-last-c++-devel-6.9.6.7-1.el7.remi.x86_64 @remi
        Obsoleted  ImageMagick-last-devel-6.9.6.7-1.el7.remi.x86_64     @remi
        Obsoleted  ImageMagick-last-libs-6.9.6.7-1.el7.remi.x86_64      @remi
        Obsoleting ImageMagick6-6.9.6.8-1.el7.remi.x86_64               @remi
        Obsoleting ImageMagick6-c++-6.9.6.8-1.el7.remi.x86_64           @remi
        Obsoleting ImageMagick6-c++-devel-6.9.6.8-1.el7.remi.x86_64     @remi
        Obsoleting ImageMagick6-devel-6.9.6.8-1.el7.remi.x86_64         @remi
        Obsoleting ImageMagick6-libs-6.9.6.8-1.el7.remi.x86_64          @remi
        Updated    bash-4.2.46-20.el7_2.x86_64                          @updates
        Update          4.2.46-21.el7_3.x86_64                          @updates
    Loading mirror speeds from cached hostfile
     * base: centos.bhs.mirrors.ovh.net
     * epel: ftp.osuosl.org
     * extras: centos.bhs.mirrors.ovh.net
     * rpmforge: repoforge.mirror.constant.com
     * updates: centos.bhs.mirrors.ovh.net
    350 packages excluded due to repository priority protections
        Updated    centos-release-7-2.1511.el7.centos.2.10.x86_64       ?
        Update                    7-3.1611.el7.centos.x86_64            @base
        Updated    expat-2.1.0-8.el7.x86_64                             @?base
        Update           2.1.0-10.el7_3.x86_64                          @updates
        Updated    expat-devel-2.1.0-8.el7.x86_64                       @base
        Update                 2.1.0-10.el7_3.x86_64                    @updates
        Updated    glibc-2.17-157.el7.x86_64                            @cr
        Update           2.17-157.el7_3.1.x86_64                        @updates
        Updated    glibc-common-2.17-157.el7.x86_64                     @cr
        Update                  2.17-157.el7_3.1.x86_64                 @updates
        Updated    glibc-devel-2.17-157.el7.x86_64                      @cr
        Update                 2.17-157.el7_3.1.x86_64                  @updates
        Updated    glibc-headers-2.17-157.el7.x86_64                    @cr
        Update                   2.17-157.el7_3.1.x86_64                @updates
        Install    kernel-3.10.0-514.2.2.el7.x86_64                     @updates
        Install    kernel-devel-3.10.0-514.2.2.el7.x86_64               @updates
        Updated    kernel-headers-3.10.0-514.el7.x86_64                 @cr
        Update                    3.10.0-514.2.2.el7.x86_64             @updates
        Updated    kernel-tools-3.10.0-327.36.3.el7.x86_64              @updates
        Update                  3.10.0-514.2.2.el7.x86_64               @updates
        Updated    kernel-tools-libs-3.10.0-327.36.3.el7.x86_64         @updates
        Update                       3.10.0-514.2.2.el7.x86_64          @updates
        Updated    krb5-devel-1.14.1-26.el7.x86_64                      @cr
        Update                1.14.1-27.el7_3.x86_64                    @updates
        Updated    krb5-libs-1.14.1-26.el7.x86_64                       @cr
        Update               1.14.1-27.el7_3.x86_64                     @updates
        Updated    libkadm5-1.14.1-26.el7.x86_64                        @cr
        Update              1.14.1-27.el7_3.x86_64                      @updates
        Updated    libreport-filesystem-2.1.11-32.el7.centos.x86_64     ?
        Update                          2.1.11-35.el7.centos.x86_64     @base
        Updated    lz4-r131-1.el7.x86_64                                @epel
        Update         1.7.3-1.el7.x86_64                               @epel
        Updated    python-linux-procfs-0.4.6-3.el7.noarch               @base
        Update                         0.4.9-3.el7.noarch               @base
        Updated    python-perf-3.10.0-514.el7.x86_64                    @cr
        Update                 3.10.0-514.2.2.el7.x86_64                @updates
        Updated    python-schedutils-0.4-4.el7.x86_64                   @base
        Update                       0.4-6.el7.x86_64                   @base
        Updated    remi-release-7.1-3.el7.remi.noarch                   ?
        Update                  7.2-1.el7.remi.noarch                   @remi
        Updated    selinux-policy-3.13.1-102.el7_3.4.noarch             @cr
        Update                    3.13.1-102.el7_3.7.noarch             @updates
        Updated    selinux-policy-targeted-3.13.1-102.el7_3.4.noarch    @cr
        Update                             3.13.1-102.el7_3.7.noarch    @updates
        Updated    sudo-1.8.6p7-20.el7.x86_64                           @cr
        Update          1.8.6p7-21.el7_3.x86_64                         @updates
        Updated    tuna-0.11.1-12.el7_2.noarch                          @updates
        Update          0.13-5.el7.noarch                               @base
        Updated    tuned-2.7.1-3.el7.noarch                             @cr
        Update           2.7.1-3.el7_3.1.noarch                         @updates
        Updated    tzdata-2016i-1.el7.noarch                            @cr
        Update            2016j-1.el7.noarch                            @updates
    Scriptlet output:
       1 warning: /etc/nsswitch.conf created as /etc/nsswitch.conf.rpmnew
       2 warning: /etc/ImageMagick-last/ImageMagick-6/policy.xml saved as /etc/ImageMagick-last/ImageMagick-6/policy.xml.rpmsave
    history info
    
     
  10. eva2000

    eva2000 Administrator Staff Member

    28,942
    6,570
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,751
    Local Time:
    9:10 AM
    Nginx 1.13.x
    MariaDB 5.5
    just means you have custom files already so updated ones get renamed as .rpmnew to not override your existing ones :)

    do you have selinux and sudo users enabled/added ?
     
  11. Tracy Perry

    Tracy Perry Active Member

    188
    80
    28
    Aug 24, 2014
    Texas
    Ratings:
    +129
    Local Time:
    6:10 PM
    1.11.5
    MariaDB 10.0.28
    selinux disabled, sudo users enabled.
    and your YUM update looks very similar to mine.
    Yes, I'm loading some ipsets using the scripts, but that should not have caused issues. I "think" it may have been due to where CSF was loading - I noticed I kept getting an iptables error (went by to fast in IPMI to really see). Once I disabled CSF and then re-enabled it it appears to have fixed it (but haven't rebooted yet again as I had to take my wife to have a heart cath done).
     
    • Informative Informative x 1