Get the most out of your Centmin Mod LEMP stack
Become a Member

Beta Branch Reworking centmin.sh menu option 23 for better efficiency & performance

Discussion in 'Beta release code' started by eva2000, Feb 11, 2016.

Thread Status:
Not open for further replies.
  1. eva2000

    eva2000 Administrator Staff Member

    54,087
    12,177
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,735
    Local Time:
    11:26 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Centmin Mod 123.08stable was first release to use the new centmin.sh menu option 23 for git based update of Centmin Mod code as outlined at centminmod.com/upgrade.html. Just thinking out aloud on possible improvements in efficiency for centmin.sh menu option 23.

    Code:
    --------------------------------------------------------
    Centmin Mod 1.2.3-eva2000.09 - http://centminmod.com
    --------------------------------------------------------
                       Centmin Mod Menu          
    --------------------------------------------------------
    1).  Centmin Install
    2).  Add Nginx vhost domain
    3).  NSD setup domain name DNS
    4).  Nginx Upgrade / Downgrade
    5).  PHP Upgrade / Downgrade
    6).  XCache Re-install
    7).  APC Cache Re-install
    8).  XCache Install
    9).  APC Cache Install
    10). Memcached Server Re-install
    11). MariaDB 5.2/5.5 & 10.x Upgrade Sub-Menu
    12). Zend OpCache Install/Re-install
    13). Install ioping.sh vbtechsupport.com/1239/
    14). SELinux disable
    15). Install/Reinstall ImagicK PHP Extension
    16). Change SSHD Port Number
    17). Multi-thread compression: pigz,pbzip2,lbzip2...
    18). Suhosin PHP Extension install
    19). Install FFMPEG and FFMPEG PHP Extension
    20). NSD Re-install
    21). Update - Nginx + PHP-FPM + Siege
    22). Add Wordpress Nginx vhost + Cache Plugin
    23). Update Centmin Mod Code Base
    24). Exit
    --------------------------------------------------------
    Enter option [ 1 - 24 ] 23
    --------------------------------------------------------
    
    Code:
    --------------------------------------------------------
            Centmin Mod Updater Sub-Menu     
    --------------------------------------------------------
    1). Setup Centmin Mod Github Environment
    2). Update Centmin Mod Current Branch
    3). Update Centmin Mod Newer Branch
    4). Back to Main menu
    --------------------------------------------------------
    Enter option [ 1 - 4 ]
    --------------------------------------------------------
    The full git clone method gets larger and larger in size due to git clone by default pulling all Centmin Mod github branches and history as well Branches · centminmod/centminmod · GitHub So I am thinking of reworking centmin.sh menu option 23 so that it only pulls the specific branch needed rather than all the branches. It means for centmin.sh menu option 23 submenu option 3 for switching branches, I can not rely on listing all branches within git command environment and need to look at other avenues.

    Original Size Of Full Git Clones



    To illustrate the size difference here is the default full git clone environment when you run centmin.sh menu option 23 submenu option 1 to create /usr/local/src/centminmod

    702MB in size
    Code:
    du -sh /usr/local/src/centminmod
    702M    /usr/local/src/centminmod
    
    size-pack = 691,401 bytes
    Code:
    git count-objects -v
    count: 0
    size: 0
    in-pack: 9595
    packs: 1
    size-pack: 691401
    prune-packable: 0
    garbage: 0
    size-garbage: 0
    
    list of all branches available
    Code:
    git branch -a
    * 123.09beta01
      master
      remotes/origin/123.06stable
      remotes/origin/123.07stable
      remotes/origin/123.08beta03
      remotes/origin/123.08beta03-el7ffmpeg-fix
      remotes/origin/123.08beta03-rtmp
      remotes/origin/123.08beta03libressl
      remotes/origin/123.08stable
      remotes/origin/123.08zerodown
      remotes/origin/123.09beta01
      remotes/origin/123.09beta01-gcc
      remotes/origin/123.09beta01-gcc2
      remotes/origin/123.09beta01le
      remotes/origin/123.09beta01le2
      remotes/origin/123.09beta01le3
      remotes/origin/123.09beta01le4
      remotes/origin/HEAD -> origin/master
      remotes/origin/master

    Proposed New Method For Git Clone Of Single Branch



    The proposed new centmin.sh menu option 23 submenu option 1 and 3 options is to pull just the single github branch. It will be much faster to clone/pull and smaller in size. But won't have all the branches available for listing.

    Single branch git clone
    Code:
    git clone -b 123.09beta01 --depth=1 https://github.com/centminmod/centminmod.git centminmod
    
    Timing how much faster it's to clone = 3.666 seconds :)
    Code:
    time git clone -b 123.09beta01 --depth=1 https://github.com/centminmod/centminmod.git centminmod
    Cloning into 'centminmod'...
    remote: Counting objects: 283, done.
    remote: Compressing objects: 100% (252/252), done.
    remote: Total 283 (delta 54), reused 98 (delta 15), pack-reused 0
    Receiving objects: 100% (283/283), 24.15 MiB | 8.22 MiB/s, done.
    Resolving deltas: 100% (54/54), done.
    
    real    0m3.666s
    user    0m0.287s
    sys     0m0.185s
    
    Much smaller size 51MB vs 702MB as it only pulls download the single 123.09beta01 branch instead of all branches and history
    Code:
    du -sh centminmod/
    51M     centminmod/
    
    size-pack = 24737 bytes
    Code:
    git count-objects -v  
    count: 0
    size: 0
    in-pack: 283
    packs: 1
    size-pack: 24737
    prune-packable: 0
    garbage: 0
    size-garbage: 0
    
    Only 123.09beta01 branch available
    Code:
    git branch -a
    * 123.09beta01
      remotes/origin/123.09beta01

    centmin.sh menu option 23 submenu option 3



    With the new proposed method as you can see you can no longer list all available branches in centmin.sh menu option 23 submenu menu option 3 to switch branches.

    Existing method pulls the branch list via git as you have access to all branch listing info.


    Code:
    --------------------------------------------------------
            Centmin Mod Updater Sub-Menu       
    --------------------------------------------------------
    1). Setup Centmin Mod Github Environment
    2). Update Centmin Mod Current Branch
    3). Update Centmin Mod Newer Branch
    4). Back to Main menu
    --------------------------------------------------------
    Enter option [ 1 - 4 ] 3
    --------------------------------------------------------
    
    fetching latest branch list...
    
    Update Centmin Mod to newer branch via git
    You need to input the name of the branch
    List of current remote branches by descending date order
    
    2016-02-11 07:22:38 +1000 7 hours ago   123.09beta01
    2016-02-11 05:54:49 +1000 8 hours ago   master
    2016-02-11 05:54:49 +1000 8 hours ago   master
    2016-02-11 05:54:39 +1000 8 hours ago   123.08stable
    2016-02-04 03:57:20 +1000 7 days ago    123.09beta01le4
    2016-01-21 08:47:28 +1000 3 weeks ago   123.09beta01-gcc2
    2016-01-17 04:47:23 +1000 4 weeks ago   123.09beta01le3
    2016-01-05 07:29:32 +1000 5 weeks ago   123.09beta01le2
    2015-12-30 02:03:08 +1000 6 weeks ago   123.09beta01le
    2015-12-23 05:39:43 +1000 7 weeks ago   123.09beta01-gcc
    2015-07-26 22:47:23 +1000 7 months ago  123.08beta03
    2015-07-16 07:46:33 +1000 7 months ago  123.08beta03-el7ffmpeg-fix
    2015-07-10 19:59:08 +1000 7 months ago  123.07stable
    2015-06-14 13:15:27 +1000 8 months ago  123.08beta03-rtmp
    2015-06-03 21:13:57 +1000 8 months ago  123.08beta03libressl
    2015-03-25 03:45:39 +1000 11 months ago 123.08zerodown
    2014-06-23 17:08:44 +1000 1 year, 8 months ago  123.06stable
            ->
    
    Enter the branch name you want to switch to i.e. 123.09beta01 :
    So with the new proposed method, I need to find a way to display this branch listing info when a user selects centmin.sh menu option 23 submenu option 3.

    Currently, I have worked on a plain curl command to grab the branch listings from github.com site itself outside of git cmd which means it's much faster.

    For github.com branch listing, they have listed on site active branches so it means more relevant listing as i don't display older unused branches. The curl command ends up with following listing:
    Code:
    curl -s https://github.com/centminmod/centminmod/branches/active | grep -P '\/centminmod\/centminmod\/tree\/.*">' | grep -Po '(?<=href=")[^"]*(?=")' | sed 's|/centminmod/centminmod/tree/||g' | sort -u                     
    123.08stable
    123.09beta01
    123.09beta01-gcc
    123.09beta01-gcc2
    123.09beta01le
    123.09beta01le2
    123.09beta01le3
    123.09beta01le4
    The gitlab back centminmod repo, doesn't list active branches separately so has a slightly different display if i order by most recently updated branches as older inactive branches are included unlike with github's active branch listing page. The curl command ends up with following listing:
    Code:
    curl -s https://gitlab.com/centminmod/centminmod/branches?sort=recently_updated | grep -P '\/centminmod\/centminmod\/tree\/.*">' | grep -Po '(?<=href=")[^"]*(?=")' | sed 's|/centminmod/centminmod/tree/||g' | sort -u
    123.06stable
    123.07stable
    123.08beta03
    123.08beta03-rtmp
    123.08stable
    123.08zerodown
    123.09beta01
    123.09beta01-gcc
    123.09beta01-gcc2
    123.09beta01le
    123.09beta01le2
    123.09beta01le3
    123.09beta01le4
    master
    
    The new proposed centmin.sh menu option 23 submenu option 3 will look like this:
    Code:
    --------------------------------------------------------
            Centmin Mod Updater Sub-Menu            
    --------------------------------------------------------
    1). Setup Centmin Mod Github Environment
    2). Update Centmin Mod Current Branch
    3). Update Centmin Mod Newer Branch
    4). Back to Main menu
    --------------------------------------------------------
    Enter option [ 1 - 4 ] 3
    --------------------------------------------------------
    
    fetching latest branch list...
    
    Update Centmin Mod to newer branch via git
    You need to input the name of the branch
    List of current remote branches by descending date order
    
    123.09beta01
    123.08stable
    123.09beta01le4
    123.09beta01le3
    123.09beta01le2
    123.09beta01le
    
    Enter the branch name you want to switch to i.e. 123.09beta01 :
     
    Last edited: Feb 11, 2016
  2. eva2000

    eva2000 Administrator Staff Member

    54,087
    12,177
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,735
    Local Time:
    11:26 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
Thread Status:
Not open for further replies.