Welcome to Centmin Mod Community
Register Now

Sysadmin CentOS 7 Python 2.7 will reach the end of its life on January 1st, 2020

Discussion in 'System Administration' started by eva2000, Dec 8, 2019.

Thread Status:
Not open for further replies.
  1. eva2000

    eva2000 Administrator Staff Member

    55,197
    12,251
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,829
    Local Time:
    7:45 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    CentOS 7 users doing manual pip install upgrades may get a deprecated warning message like:
    CentOS & Redhat's Python 2.7 usually gets backported updates to same version. So that message is just cosmetic and can be ignored. You do not overwrite and upgrade system Python 2.7 on CentOS otherwise you break CentOS and possibly YUM itself so won't be able to update via YUM and break any system dependencies on Python 2.7.


    You can install Python 3.6 side by side with Python 2.7 via Centmin Mod 123.09beta01's addons/python36_install.sh and you'd have pip3.6 and python3.6 access to 3.6 version
    Code (Text):
    python --version
    Python 2.7.5
    
    python3.6 --version
    Python 3.6.8
    

    then if your python app needs python 3.6, you can use virtualenv to setup sandbox for python 3.6 similar to outline at How to install the latest version of Python on CentOS - Daniel Eriksson under Create your first isolated Python environment header instead though of pip2.7 it's just pip for 1st step.

    virtualenv and pipenv are 2 was of creating sandboxes see Pipenv & Virtual Environments — The Hitchhiker's Guide to Python along with virtualenvwrapper

    example for virtualenv using python3.6 interpreter
    Code (Text):
    pip install virtualenv
    mkdir -p /home/python_projects
    cd /home/python_projects
    virtualenv -p /usr/bin/python3.6 venv1
    source venv1/bin/activate
    
    # pip doesn't like ccache so disable it
    export CC='gcc'
    export CXX="g++"
    
    # pip needs a tmp directory that doesn't have noexec restrictions
    mkdir -p /home/piptmp
    chmod 1777 /home/piptmp
    export TMPDIR=/home/piptmp
    
    # update pip defined by -p flag to use python3.6 binary
    pip install -U pip
    pip --version
    

    Code (Text):
    virtualenv -p /usr/bin/python3.6 venv1
    Running virtualenv with interpreter /usr/bin/python3.6
    Already using interpreter /usr/bin/python3.6
    Using base prefix '/usr'
      No LICENSE.txt / LICENSE found in source
    New python executable in /home/python_projects/venv1/bin/python3.6
    Also creating executable in /home/python_projects/venv1/bin/python
    Installing setuptools, pip, wheel...
    done.
    

    Code (Text):
    source venv1/bin/activate
    pip install -U pip
    Requirement already up-to-date: pip in ./venv1/lib/python3.6/site-packages (19.3.1)
    (venv1)
    
    pip --version
    pip 19.3.1 from /home/python_projects/venv1/lib/python3.6/site-packages/pip (python 3.6)
    

    python/pip binaries installed in virtual environment called venv1 in bin directory at /home/python_projects/venv1/bin
    Code (Text):
    ls -lah /home/python_projects/venv1/bin/
    total 84K
    drwxr-xr-x 2 root root 4.0K Dec  7 19:55 .
    drwxr-xr-x 5 root root 4.0K Dec  7 19:54 ..
    -rw-r--r-- 1 root root 2.2K Dec  7 19:54 activate
    -rw-r--r-- 1 root root 1.5K Dec  7 19:54 activate.csh
    -rw-r--r-- 1 root root 3.1K Dec  7 19:54 activate.fish
    -rw-r--r-- 1 root root 1.8K Dec  7 19:54 activate.ps1
    -rw-r--r-- 1 root root 1.5K Dec  7 19:54 activate_this.py
    -rw-r--r-- 1 root root 1.2K Dec  7 19:54 activate.xsh
    -rwxr-xr-x 1 root root  245 Dec  7 19:55 cygdb
    -rwxr-xr-x 1 root root  266 Dec  7 19:55 cython
    -rwxr-xr-x 1 root root  246 Dec  7 19:55 cythonize
    -rwxr-xr-x 1 root root  255 Dec  7 19:54 easy_install
    -rwxr-xr-x 1 root root  255 Dec  7 19:54 easy_install-3.6
    -rwxr-xr-x 1 root root  242 Dec  7 19:54 pip
    -rwxr-xr-x 1 root root  242 Dec  7 19:54 pip3
    -rwxr-xr-x 1 root root  242 Dec  7 19:54 pip3.6
    lrwxrwxrwx 1 root root    9 Dec  7 19:54 python -> python3.6
    lrwxrwxrwx 1 root root    9 Dec  7 19:54 python3 -> python3.6
    -rwxr-xr-x 1 root root  12K Dec  7 19:54 python3.6
    -rwxr-xr-x 1 root root 2.3K Dec  7 19:54 python-config
    -rwxr-xr-x 1 root root  233 Dec  7 19:54 wheel
    

    Direct access to venv1 sandbox's version of python 3.6 and pip 3.6 to install python packages will end up in /home/python_projects/venv1/ sandbox directory not to polute system python 2.7
    Code (Text):
    /home/python_projects/venv1/bin/python3.6 --version
    Python 3.6.8
    
    /home/python_projects/venv1/bin/pip3.6 --version
    pip 19.3.1 from /home/python_projects/venv1/lib/python3.6/site-packages/pip (python 3.6)
    
    /home/python_projects/venv1/bin/pip --version
    pip 19.3.1 from /home/python_projects/venv1/lib/python3.6/site-packages/pip (python 3.6)
    
     
  2. eva2000

    eva2000 Administrator Staff Member

    55,197
    12,251
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,829
    Local Time:
    7:45 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    FYI Redhat and CentOS Python 2 support after 2020 statement How is Python 2 supported in RHEL after 2020? - Red Hat Customer Portal

     
  3. eva2000

    eva2000 Administrator Staff Member

    55,197
    12,251
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,829
    Local Time:
    7:45 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Above instructions are for Python 3.6 usage. Centmin Mod users is they'd use CentOS and CentOS 7 only has native Python 2.7 and addons/python36_install.sh is highest version installed via IUS Community Yum repo for Python 3.6.

    Centmin Mod and CentOS 7 won't have the required Python 3.7+ versions for such an install. However, you can install Python 3.7 manually too. Below is copy of outline for Python 3.7 at Quick tools to perform Server Auditing

    Install Python 3.7 first.

    The compilation can take a long time to install especially on slow servers or single cpu threaded servers - could take hours on very slow servers!

    Code (Text):
    # update Centmin Mod code
    cmupdate
    
    # pip doesn't like ccache so disable it
    export CC='gcc'
    export CXX="g++"
    
    # pip needs a tmp directory that doesn't have noexec restrictions
    mkdir -p /home/piptmp
    chmod 1777 /home/piptmp
    export TMPDIR=/home/piptmp
    
    # install python 3.7.7 or 3.8.2
    yum -y install libffi-devel
    python_ver=3.7.7
    python_prefixver=$(echo $python_ver | cut -d . -f1,2)
    cd /svr-setup
    wget https://www.python.org/ftp/python/${python_ver}/Python-${python_ver}.tgz
    tar xzf Python-${python_ver}.tgz
    cd Python-${python_ver}
    make clean
    if [[ "$(nproc)" -le '2' ]]; then time ./configure --prefix=/opt/python${python_prefixver} --with-openssl=/usr; else time ./configure --enable-optimizations --prefix=/opt/python${python_prefixver} --with-openssl=/usr; fi
    time make -j$(nproc)
    time make altinstall
    ls -lah /opt/python${python_prefixver}/bin
    ln -s /opt/python${python_prefixver}/bin/python${python_prefixver} /opt/python${python_prefixver}/bin/python3
    ln -s /opt/python${python_prefixver}/bin/python${python_prefixver} /opt/python${python_prefixver}/bin/python
    ln -s /opt/python${python_prefixver}/bin/python${python_prefixver} /usr/bin/python${python_prefixver}
    ln -s /opt/python${python_prefixver}/bin/pip${python_prefixver} /opt/python${python_prefixver}/bin/pip3
    ln -s /opt/python${python_prefixver}/bin/pip${python_prefixver} /opt/python${python_prefixver}/bin/pip
    ln -s /opt/python${python_prefixver}/bin/easy_install-${python_prefixver} /opt/python${python_prefixver}/bin/easy_install
    ln -s /opt/python${python_prefixver}/bin/idle${python_prefixver} /opt/python${python_prefixver}/bin/idle
    ln -s /opt/python${python_prefixver}/bin/2to3-${python_prefixver} /opt/python${python_prefixver}/bin/2to3
    ln -s /opt/python${python_prefixver}/bin/pydoc${python_prefixver} /opt/python${python_prefixver}/bin/pydoc
    ln -s /opt/python${python_prefixver}/bin/pyvenv-${python_prefixver} /opt/python${python_prefixver}/bin/pyvenv
    rm -f /svr-setup/Python-${python_ver}.tgz
    /opt/python${python_prefixver}/bin/python${python_prefixver} --version
    /opt/python${python_prefixver}/bin/pip${python_prefixver} --version
    

    Python 3.7 will be installed at
    Code (Text):
    /opt/python3.7/bin/python --version
    Python 3.7.7
    
    /opt/python3.7/bin/pip --version
    pip 19.2.3 from /opt/python3.7/lib/python3.7/site-packages/pip (python 3.7)
    

    Code (Text):
    ls -lah /opt/python3.7/bin
    total 27M
    drwxr-xr-x 2 root root 4.0K Jan 31 20:15 .
    drwxr-xr-x 6 root root   56 Jan 31 18:58 ..
    lrwxrwxrwx 1 root root   27 Jan 31 19:02 2to3 -> /opt/python3.7/bin/2to3-3.7
    -rwxr-xr-x 1 root root  105 Jan 31 20:15 2to3-3.7
    lrwxrwxrwx 1 root root   35 Jan 31 19:02 easy_install -> /opt/python3.7/bin/easy_install-3.7
    -rwxr-xr-x 1 root root  245 Jan 31 18:58 easy_install-3.7
    lrwxrwxrwx 1 root root   26 Jan 31 19:04 idle -> /opt/python3.7/bin/idle3.7
    -rwxr-xr-x 1 root root  103 Jan 31 20:15 idle3.7
    lrwxrwxrwx 1 root root   25 Jan 31 19:02 pip -> /opt/python3.7/bin/pip3.7
    lrwxrwxrwx 1 root root   25 Jan 31 19:02 pip3 -> /opt/python3.7/bin/pip3.7
    -rwxr-xr-x 1 root root  227 Jan 31 18:58 pip3.7
    -rwxr-xr-x 1 root root  218 Jan 31 19:24 pipenv
    -rwxr-xr-x 1 root root  229 Jan 31 19:24 pipenv-resolver
    lrwxrwxrwx 1 root root   27 Jan 31 19:04 pydoc -> /opt/python3.7/bin/pydoc3.7
    -rwxr-xr-x 1 root root   88 Jan 31 20:15 pydoc3.7
    lrwxrwxrwx 1 root root   28 Jan 31 19:02 python -> /opt/python3.7/bin/python3.7
    lrwxrwxrwx 1 root root   28 Jan 31 19:02 python3 -> /opt/python3.7/bin/python3.7
    -rwxr-xr-x 2 root root  14M Jan 31 20:12 python3.7
    -rwxr-xr-x 2 root root  14M Jan 31 20:12 python3.7m
    -rwxr-xr-x 1 root root 2.9K Jan 31 20:15 python3.7m-config
    lrwxrwxrwx 1 root root   29 Jan 31 19:02 pyvenv -> /opt/python3.7/bin/pyvenv-3.7
    -rwxr-xr-x 1 root root  445 Jan 31 20:15 pyvenv-3.7
    -rwxr-xr-x 1 root root  224 Jan 31 19:24 virtualenv
    -rwxr-xr-x 1 root root  229 Jan 31 19:24 virtualenv-clone
    

    Then you'd use Python 3.7 to create virtualenv for your specific project i.e. at
    /home/python_projects/myproject or whatever directory you choose
    Code (Text):
    python_ver=3.7.7
    python_prefixver=$(echo $python_ver | cut -d . -f1,2)
    mkdir -p /home/python_projects/myproject
    /opt/python${python_prefixver}/bin/python${python_prefixver} -m venv /home/python_projects/myproject
    source /home/python_projects/myproject/bin/activate
    pip install -U pip
    

    Or outside of source activation of specific Python virtualenv, direct references
    Code (Text):
    /home/python_projects/myproject/bin/pip --version
    pip 19.2.3 from /home/python_projects/myproject/lib/python3.7/site-packages/pip (python 3.7)
    

    Code (Text):
    ls -lah /home/python_projects/myproject/bin/
    total 40K
    drwxr-xr-x 2 root root 4.0K Mar 31 04:43 .
    drwxr-xr-x 5 root root 4.0K Mar 31 04:43 ..
    -rw-r--r-- 1 root root 2.2K Mar 31 04:43 activate
    -rw-r--r-- 1 root root 1.3K Mar 31 04:43 activate.csh
    -rw-r--r-- 1 root root 2.4K Mar 31 04:43 activate.fish
    -rwxr-xr-x 1 root root  262 Mar 31 04:43 easy_install
    -rwxr-xr-x 1 root root  262 Mar 31 04:43 easy_install-3.7
    -rwxr-xr-x 1 root root  244 Mar 31 04:43 pip
    -rwxr-xr-x 1 root root  244 Mar 31 04:43 pip3
    -rwxr-xr-x 1 root root  244 Mar 31 04:43 pip3.7
    lrwxrwxrwx 1 root root    9 Mar 31 04:43 python -> python3.7
    lrwxrwxrwx 1 root root    9 Mar 31 04:43 python3 -> python3.7
    lrwxrwxrwx 1 root root   28 Mar 31 04:43 python3.7 -> /opt/python3.7/bin/python3.7
    

    Example of using the myproject Python 3.7 virtualenv pip command directly

    To update pip in virtualenv
    Code (Text):
    /home/python_projects/myproject/bin/pip install --upgrade pip
    Collecting pip
      Using cached https://files.pythonhosted.org/packages/54/0c/d01aa759fdc501a58f431eb594a17495f15b88da142ce14b5845662c13f3/pip-20.0.2-py2.py3-none-any.whl
    Installing collected packages: pip
      Found existing installation: pip 19.2.3
        Uninstalling pip-19.2.3:
          Successfully uninstalled pip-19.2.3
    Successfully installed pip-20.0.2
    

    Code (Text):
    /home/python_projects/myproject/bin/pip --version
    pip 20.0.2 from /home/python_projects/myproject/lib/python3.7/site-packages/pip (python 3.7)
    
     
    Last edited: Mar 31, 2020
  4. eva2000

    eva2000 Administrator Staff Member

    55,197
    12,251
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,829
    Local Time:
    7:45 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    pip 21.x version was released on Jan 24, 2021 and it drops support for Python 2.7 and can break pip on CentOS 7 systems if you update to that version. So for CentOS systems make sure you do not upgrade to pip 21.x and if you did, downgrade to pip 20.3.4 using below outlined method. More updates for this issue will be made to this post as they are available.

    Centmin Mod 123.09beta01 has 2 updates related to this
    1. While troubleshooting this issue, I disabled centmin.sh menu triggered tools/pip-updates.sh runs to auto update pip as at the time it updated to pip 21.x and break pip commands for python 2.7 system installed pip. Beta Branch - skip centmin.sh triggered pip updates in 123.09beta01. I may at a later time re-enable tools/pip-update.sh centmin.sh triggered routines once I get a better grasp of the issue.
    2. This update applies to new fresh Centmin Mod 123.09beta01 and higher installs to make sure initial install only upgrades pip to 20.3.4 and not 21.x for python 2.7 system pip Beta Branch - update python 2.7 pip update to 20.3.4 max in 123.09beta01. Doing this seems to allow pip upgrades to max out at 20.3.4 instead of installing 21.x version which drops support for python 2.7
      Code (Text):
      pip install --upgrade pip
      DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
      Requirement already up-to-date: pip in /usr/lib/python2.7/site-packages (20.3.4)
      
    Python 2.7 upgrade message can be ignored. See Sysadmin - CentOS 7 Python 2.7 will reach the end of its life on January 1st, 2020 and https://blog.centminmod.com/2021/01...ade-deprecation-end-of-life-january-1st-2020/

    If you incorrectly auto updated or manually updated python 2.7 system pip version to 21.x, you will get errors like below due to pip 21.x dropping support for python 2.7

    Workaround Fix For PIP 21.x Decprecation Of Python 2.7



    For CentOS 7 at least, to fix the errors, you need to reinstall python2-pip YUM package and only update pip to 20.3.4 max version using below commands. Need to set a TMPDIR which isn't noexec restricted and disable ccache compiler caching by only using gcc.
    Code (Text):
    mkdir -p /home/piptmp
    chmod 1777 /home/piptmp
    export TMPDIR=/home/piptmp
    export CC='gcc'
    
    yum -y reinstall python2-pip --disableplugin=versionlock,priorities --disableexcludes=main
    pip install --upgrade pip==20.3.4
    pip install --upgrade "pip==20.3.4" setuptools
    

    pip list command to verify installed pip and setuptools versions
    Code (Text):
    pip list
    DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
    Package                          Version
    -------------------------------- -------
    backports.ssl-match-hostname     3.5.0.1
    chardet                          2.2.1
    configobj                        4.7.2
    decorator                        3.4.0
    future                           0.18.2
    Glances                          3.1.6.1
    iniparse                         0.4
    iotop                            0.6
    ipaddress                        1.0.16
    IPy                              0.75
    kitchen                          1.1.1
    netifaces                        0.10.9
    perf                             0.1
    pip                              20.3.4
    policycoreutils-default-encoding 0.1
    psutil                           5.8.0
    py-cpuinfo                       4.0.0
    pycurl                           7.19.0
    pygobject                        3.22.0
    pygpgme                          0.3
    pyliblzma                        0.5.3
    pymdstat                         0.4.2
    pyOpenSSL                        0.13.1
    pyparsing                        1.5.6
    python-dateutil                  1.5
    python-linux-procfs              0.4.9
    pyudev                           0.15
    pyxattr                          0.5.1
    schedutils                       0.4
    seobject                         0.1
    sepolicy                         1.1
    setuptools                       44.1.1
    six                              1.9.0
    slip                             0.4.0
    slip.dbus                        0.4.0
    urlgrabber                       3.10
    urllib3                          1.26.2
    yum-metadata-parser              1.1.4
    
     
Thread Status:
Not open for further replies.