Get the most out of your Centmin Mod LEMP stack
Become a Member

Nginx nginx upgrade

Discussion in 'Install & Upgrades or Pre-Install Questions' started by emre22, Feb 15, 2017.

  1. emre22

    emre22 Member

    42
    5
    8
    Oct 15, 2016
    Ratings:
    +5
    Local Time:
    8:26 AM
    nginx-1.10.1
    MariaDB 10
    Please fill in any relevant information that applies to you:
    • CentOS Version: CentOS 6 64bit
    • Centmin Mod Version Installed:123.09beta01
    • Nginx Version Installed: 1.11.9 to 1.11.10
    • PHP Version Installed: 5.6.29
    • MariaDB MySQL Version Installed: 10.0.28
    • When was last time updated Centmin Mod code base ? : checking daily -> 23,2

    Hello,

    I just wanted to upgrade nginx to 1.11.10

    that is what I received:

    Code:
     .....
    -o objs/addon/src/ngx_rewrite_driver_factory.o \
            ../ngx_pagespeed-1.12.34.2-beta/src/ngx_rewrite_driver_factory.cc
    gcc: internal compiler error: Killed (program cc1plus)
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See <http://bugzilla.redhat.com/bugzilla> for instructions.
    make[1]: *** [objs/addon/src/ngx_pagespeed.o] Error 4
    make[1]: *** Waiting for unfinished jobs....
    
    what should I do?
     
  2. eva2000

    eva2000 Administrator Staff Member

    30,168
    6,784
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    4:26 PM
    Nginx 1.13.x
    MariaDB 5.5
    Sounds like you ran out of memory during upgrade.

    Do you have any persistent config file options set in /etc/centminmod/custom_config.inc ? You can check via this command:
    Code (Text):
    cat /etc/centminmod/custom_config.inc
    

    Post output in CODE tags.

    To troubleshoot, you need to check the the nginx upgrade log at /root/centminlogs and instructions under Sharing logs and errors heading for using Pastebin.com or Gists to share a sanitised version of the contents of the nginx_upgrade.log log. You can see full details at How to troubleshoot Centmin Mod initial install issues

    also what's output of
    Code (Text):
    nginx -t

    when you run centmin.sh menu option 4 there's a nginx upgrade log timestamped at /root/centminlogs

    if you type this command it lists all logs in date ascending order so latest log at bottom
    Code (Text):
    ls -lArt /root/centminlogs

    so copy the entire contents of latest nginx_upgrade log to gist.github.com or pastebin.com

    you can use grep to filter the logs, i.e. look for nginx_upgrade in log name
    Code (Text):
    ls -lahrt /root/centminlogs/ | grep nginx_upgrade
    -rw-r--r--  1 root root 3.2M Oct 11 15:55 centminmod_1.2.3-eva2000.09.001_111016-155345_nginx_upgrade.log
    -rw-r--r--  1 root root 672K Oct 11 22:06 centminmod_1.2.3-eva2000.09.001_111016-220515_nginx_upgrade.log

    So the last nginx upgrade log was named centminmod_1.2.3-eva2000.09.001_111016-220515_nginx_upgrade.log and located at /root/centminlogs/centminmod_1.2.3-eva2000.09.001_111016-220515_nginx_upgrade.log

    then use cat command to output the contents of that log

    clear your ssh window buffer/screen and type
    Code (Text):
    cat /root/centminlogs/centminmod_1.2.3-eva2000.09.001_111016-220515_nginx_upgrade.log

    then select and copy and paste output to pastebin.com or gist.github.com file to share. If your SSH client's scroll buffer isn't large enough using cat might not output the entire log file contents, so you may need to download the log and use local text editor to open and copy and paste.

    so only need content of one specific log, in this case most recent nginx_upgrade.log log
     
    • Like Like x 1
  3. emre22

    emre22 Member

    42
    5
    8
    Oct 15, 2016
    Ratings:
    +5
    Local Time:
    8:26 AM
    nginx-1.10.1
    MariaDB 10
    Code:
    cat /etc/centminmod/custom_config.inc
    NGINX_PAGESPEED='y'
    NGXDYNAMIC_NGXPAGESPEED='y'
    ENABLEMOTD_CSFMSG='n'         # motd banner csf message alert disaply
    ENABLEMOTD_LINKSMSG='n'       # motd centminmod.com links bookmark alert display
    
    Code:
    # nginx -t
    nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
    nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful
    
    # ls -lArt /root/centminlogs

    centminmod_1.2.3-eva2000.09.001_140217-205123_nginx_upgrade
     
  4. eva2000

    eva2000 Administrator Staff Member

    30,168
    6,784
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    4:26 PM
    Nginx 1.13.x
    MariaDB 5.5
    looks like you ran out of memory when it came to ngx_pagespeed part of nginx install/compile

    how much memory do you have ?

    what's output for this command in CODE tags
    Code (Text):
    free -m

    what's output for sar memory usage profile for yesterday
    Code (Text):
    sar -r -f /var/log/sa/sa$(date +%d -d yesterday) 
    

    and today
    Code (Text):
    sar -r
    

    check around 14/02/17 20:51 time which is the timestamp of your nginx log
     
    • Like Like x 1
  5. emre22

    emre22 Member

    42
    5
    8
    Oct 15, 2016
    Ratings:
    +5
    Local Time:
    8:26 AM
    nginx-1.10.1
    MariaDB 10
  6. eva2000

    eva2000 Administrator Staff Member

    30,168
    6,784
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    4:26 PM
    Nginx 1.13.x
    MariaDB 5.5
    yup sar output for yesterday confirms it at ~9:20pm you had >100% committed memory so ran out of memory during nginx compile
    Code (Text):
    07:10:01 PM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit
    07:20:01 PM     81752    420248     83.71      2860    124080   1290664     83.24
    07:30:01 PM     83920    418080     83.28      3156    115940   1286676     82.98
    07:40:01 PM     93540    408460     81.37      8788    156872   1220704     78.73
    07:50:01 PM     70232    431768     86.01     11044    118792   1292944     83.38
    08:00:01 PM     86172    415828     82.83      3072    107216   1321340     85.22
    08:10:01 PM     99248    402752     80.23       380     77848   1359792     87.70
    08:20:01 PM     80668    421332     83.93      9372    131244   1282708     82.72
    08:30:02 PM     91940    410060     81.69      4788    134300   1236840     79.77
    08:40:01 PM     73244    428756     85.41      5956    125392   1317188     84.95
    08:50:01 PM     93524    408476     81.37      2724    121188   1273428     82.13
    09:00:01 PM     95696    406304     80.94     10284    145084   1212692     78.21
    09:10:01 PM     97152    404848     80.65      3372    140120   1246436     80.39
    09:20:03 PM     74020    427980     85.25      1488     41560   1887792    121.75
    09:30:03 PM     69568    432432     86.14      1680     28088   2177200    140.41
    09:40:02 PM     91056    410944     81.86       464     30820   1970648    127.09
    09:50:02 PM     88108    413892     82.45       740     24444   2031120    130.99
    


    maybe try centmin.sh menu option 4 recompile/upgrade of nginx again - could be something else eating into memory at same time - though sar output shows constant 77-85% mem used so you're running on borderline.

    Good time to switch to 1GB Linode at $5/month though https://community.centminmod.com/th...-month-digitalocean-vs-vultr-vs-linode.10437/ :D
     
    • Like Like x 1
  7. emre22

    emre22 Member

    42
    5
    8
    Oct 15, 2016
    Ratings:
    +5
    Local Time:
    8:26 AM
    nginx-1.10.1
    MariaDB 10
    yea I also saw that 5$ new linode droplets, but as far as I know there is no hd upgrade only. and I want to keep it as low as possible :D
     
  8. eva2000

    eva2000 Administrator Staff Member

    30,168
    6,784
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    4:26 PM
    Nginx 1.13.x
    MariaDB 5.5
    check benchmarks https://community.centminmod.com/th...-month-digitalocean-vs-vultr-vs-linode.10437/ including post install disk free space comparison ;)

     
    • Like Like x 1
  9. emre22

    emre22 Member

    42
    5
    8
    Oct 15, 2016
    Ratings:
    +5
    Local Time:
    8:26 AM
    nginx-1.10.1
    MariaDB 10
  10. eva2000

    eva2000 Administrator Staff Member

    30,168
    6,784
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    4:26 PM
    Nginx 1.13.x
    MariaDB 5.5
  11. eva2000

    eva2000 Administrator Staff Member

    30,168
    6,784
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    4:26 PM
    Nginx 1.13.x
    MariaDB 5.5
    are you actually using ngx_pagespeed, if not you can disable it from nginx compiles to save memory usage too

    so set /etc/centminmod/custom_config.inc to
    Code (Text):
    NGINX_PAGESPEED='n'
    NGXDYNAMIC_NGXPAGESPEED='n'
    
     
    • Like Like x 1
  12. emre22

    emre22 Member

    42
    5
    8
    Oct 15, 2016
    Ratings:
    +5
    Local Time:
    8:26 AM
    nginx-1.10.1
    MariaDB 10
    no I use it, improves my sitespeed a bit, well is it a bad idea to stick to version 1.11.9?
     
  13. eva2000

    eva2000 Administrator Staff Member

    30,168
    6,784
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    4:26 PM
    Nginx 1.13.x
    MariaDB 5.5
    Not really as 1.11.10 isn't a security release so should be fine. But generally better to keep updated.

    have you tried a 2nd attempt at nginx upgrade as is ? try it
     
    • Like Like x 1
  14. emre22

    emre22 Member

    42
    5
    8
    Oct 15, 2016
    Ratings:
    +5
    Local Time:
    8:26 AM
    nginx-1.10.1
    MariaDB 10
    okay fine, I will stick to it until the next security update.

    yes I tried it and same result:

    Code:
    strip nginx binary...
    ls: cannot access objs/nginx: No such file or directory
    strip: 'objs/nginx': No such file
    ls: cannot access objs/nginx: No such file or directory
    
    
    Wed Feb 15 12:53:01 UTC 2017
    Error: 2, Nginx make failed
    
    report errors on the forums at community.centminmod.com/forums/8/
    Total Nginx Upgrade Time: 412.408847330 seconds
    
    or what do you think about stopping php mysql nginx and trying it again?
     
  15. eva2000

    eva2000 Administrator Staff Member

    30,168
    6,784
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    4:26 PM
    Nginx 1.13.x
    MariaDB 5.5
    yeah you can do that i do that on 128MB VPS running CentOS 6 32bit with centmin mod. Stop php-fpm, mysql and memcached servers, update Nginx and restart :)
     
    • Like Like x 1
  16. emre22

    emre22 Member

    42
    5
    8
    Oct 15, 2016
    Ratings:
    +5
    Local Time:
    8:26 AM
    nginx-1.10.1
    MariaDB 10
    I will do it tonight when I have no visitors on my sites and will tell you the result :D
     
  17. eva2000

    eva2000 Administrator Staff Member

    30,168
    6,784
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    4:26 PM
    Nginx 1.13.x
    MariaDB 5.5
    to prevent errors for visitors do it like this
    Code (Text):
    npstop
    mysqlstop
    memcachedstop
    

    update nginx
    Code (Text):
    mysqlstart
    memcachedstart
    nprestart
    
     
  18. emre22

    emre22 Member

    42
    5
    8
    Oct 15, 2016
    Ratings:
    +5
    Local Time:
    8:26 AM
    nginx-1.10.1
    MariaDB 10
    sorry for delayed answer. that worked like a charm!
     
    • Like Like x 1
  19. eva2000

    eva2000 Administrator Staff Member

    30,168
    6,784
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    4:26 PM
    Nginx 1.13.x
    MariaDB 5.5
    Great to hear :D
     
    • Like Like x 1
  20. elargento

    elargento Member

    293
    16
    18
    Jan 4, 2016
    Ratings:
    +39
    Local Time:
    3:26 AM
    10
    Is there any way to check which Nginx version is installed before run the upgrade script? Would be great if it can be displayed in the upgrade script