Learn about Centmin Mod LEMP Stack today
Become a Member

Disk space is going away

Discussion in 'System Administration' started by Sunka, Dec 12, 2015.

  1. Sunka

    Sunka Well-Known Member

    1,150
    325
    83
    Oct 31, 2015
    Pula, Croatia
    Ratings:
    +525
    Local Time:
    1:47 AM
    Nginx 1.17.9
    MariaDB 10.3.22
    Hi,
    My disk space is getting smaller and smaller with every new day.
    It is about 1 GB per day.
    I'm not using that server for backup, nor I upload something every day (nor my users too) what will eat disk space 1gb per day.


    How to list biggest folders on server?
    I upgraded nginx, php etc... in past few days, so maybe there is some left junk files on server?
    Maybe logs are eating space on hard, I am using logrotate but maybe is not configured in proper way?

    Any one knows program to see what was changed in x hours/days on server to see what is consuming a lot of disk space.
    Any one have experience with NCDU app?
     
  2. eva2000

    eva2000 Administrator Staff Member

    55,225
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,831
    Local Time:
    9:47 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    try
    Code:
    df -h
    du -h --max-depth=1 /
    du -h --max-depth=1 /home
    du -h --max-depth=1 /usr
    du -h --max-depth=1 /var
     
  3. Sunka

    Sunka Well-Known Member

    1,150
    325
    83
    Oct 31, 2015
    Pula, Croatia
    Ratings:
    +525
    Local Time:
    1:47 AM
    Nginx 1.17.9
    MariaDB 10.3.22
    That list biggest folders, but not show me why every day 500-1000 mb is growing inside X folder.
    I have to find what files are that and in which folder(s) they are created
     
  4. elargento

    elargento Member

    352
    17
    18
    Jan 4, 2016
    Ratings:
    +44
    Local Time:
    8:47 PM
    10
    Hi

    Did you find the reason?
     
  5. Sunka

    Sunka Well-Known Member

    1,150
    325
    83
    Oct 31, 2015
    Pula, Croatia
    Ratings:
    +525
    Local Time:
    1:47 AM
    Nginx 1.17.9
    MariaDB 10.3.22
    Yep, log files + database backup every day.
    Solved it with logrotator and cron for deleting database backup older than x days.
    Still hard disk space is eaten about +100MB per day
     
    Last edited: Jan 4, 2016
  6. eva2000

    eva2000 Administrator Staff Member

    55,225
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,831
    Local Time:
    9:47 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    part of the science is choosing the right server hardware specs for your needs :D
     
  7. Sunka

    Sunka Well-Known Member

    1,150
    325
    83
    Oct 31, 2015
    Pula, Croatia
    Ratings:
    +525
    Local Time:
    1:47 AM
    Nginx 1.17.9
    MariaDB 10.3.22
    more money thing than science :vulcan:
     
  8. Sunka

    Sunka Well-Known Member

    1,150
    325
    83
    Oct 31, 2015
    Pula, Croatia
    Ratings:
    +525
    Local Time:
    1:47 AM
    Nginx 1.17.9
    MariaDB 10.3.22
    @eva2000 regarding this thread Raising minimum & recommended disk requirements for Centmin Mod LEMP installs | Centmin Mod Community
    Is it safe to delete those backup files? Maybe to leave just newest?
    I have more than 10GB of that.
    Any code to fast delete unnecessary files?

    Code:
    [root@tvor-ocean ~]# du -h --max-depth=1 /usr/local/nginxbackup/
    6.6G    /usr/local/nginxbackup/nginxdirbackup
    4.0M    /usr/local/nginxbackup/confbackup
    6.6G    /usr/local/nginxbackup/
    Code:
    [root@tvor-ocean ~]# du -sh /home/.ccache/
    1.1G    /home/.ccache/
    Code:
    [root@tvor-ocean ~]# ccache -s
    cache directory                     /home/.ccache
    primary config                      /home/.ccache/ccache.conf
    secondary config      (readonly)    /usr/local/etc/ccache.conf
    cache hit (direct)                 13689
    cache hit (preprocessed)           10505
    cache miss                         13974
    called for link                     1186
    called for preprocessing            1132
    multiple source files                  2
    compile failed                       339
    preprocessor error                   217
    bad compiler arguments               264
    unsupported source language          570
    autoconf compile/link               4014
    no input file                        573
    files in cache                     40346
    cache size                           1.1 GB
    max cache size                       2.2 GB
    You have new mail in /var/spool/mail/root
    Code:
    [root@tvor-ocean ~]# du -sh /svr-setup
    3.4G    /svr-setup
     
  9. eva2000

    eva2000 Administrator Staff Member

    55,225
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,831
    Local Time:
    9:47 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    yeah you can just delete nginx backup contents
    Code:
    cd /usr/local/nginxbackup
    rm -rf /usr/local/nginxbackup/nginxdirbackup/*
    rm -rf /usr/local/nginxbackup/confbackup/*
    do ccache clean command above
    Code:
    ccache -C
    and just
    Code:
    cd /svr-setup/
    rm -rf /svr-setup/*
    cd commands just a precaution as rm -rf * wipes out everthing in directory you specify i.e. /svr-setup/* however some folks copy and paste mistakes and if you ran rm -rf * you could wipe out your entire server and OS !

    get into the habit of changing into directory you are going to delete even if you specifying a path to delete from rm command
     
    Last edited: Feb 10, 2016
  10. Sunka

    Sunka Well-Known Member

    1,150
    325
    83
    Oct 31, 2015
    Pula, Croatia
    Ratings:
    +525
    Local Time:
    1:47 AM
    Nginx 1.17.9
    MariaDB 10.3.22
    And that was an answer to first post (y)

    Code:
    [root@tvor-ocean ~]# df -h
    Filesystem      Size  Used Avail Use% Mounted on
    /dev/vda1        59G   39G   18G  69% /
    devtmpfs        2.0G     0  2.0G   0% /dev
    tmpfs           2.0G     0  2.0G   0% /dev/shm
    tmpfs           2.0G   57M  1.9G   3% /run
    tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup
    /dev/loop0      3.7G   99M  3.4G   3% /tmp
    tmpfs           396M     0  396M   0% /run/user/0
    [root@tvor-ocean ~]# cd /usr/local/nginxbackup
    [root@tvor-ocean nginxbackup]# rm -rf /usr/local/nginxbackup/nginxdirbackup/*
    [root@tvor-ocean nginxbackup]# rm -rf /usr/local/nginxbackup/confbackup/*
    [root@tvor-ocean nginxbackup]# ccache -C
    Cleared cache
    [root@tvor-ocean nginxbackup]# cd /svr-setup/
    [root@tvor-ocean svr-setup]# rm -rf /svr-setup/*
    [root@tvor-ocean svr-setup]# df -h
    Filesystem      Size  Used Avail Use% Mounted on
    /dev/vda1        59G   28G   29G  49% /
    devtmpfs        2.0G     0  2.0G   0% /dev
    tmpfs           2.0G     0  2.0G   0% /dev/shm
    tmpfs           2.0G   57M  1.9G   3% /run
    tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup
    /dev/loop0      3.7G   99M  3.4G   3% /tmp
    tmpfs           396M     0  396M   0% /run/user/0
     
  11. dorobo

    dorobo Active Member

    420
    104
    43
    Jun 6, 2014
    Ratings:
    +162
    Local Time:
    7:47 AM
    latest
    latest
    I noticed it just now because I got a disk alert email

    Running out of space "/ (90%)" on server

    and the log files doesn't have logrotate.

    should this be included by default by centminmod?
     
  12. eva2000

    eva2000 Administrator Staff Member

    55,225
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,831
    Local Time:
    9:47 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    yes they do rotate out of the box, checked my servers

    Code (Text):
    ls -lahrt | grep access.log
    -rw-r--r--  1 root  nginx  37M May 19  2015 access.log-20150411.gz
    -rw-r--r--  1 nginx nginx  38M Jun 26  2015 access.log-20150519.gz
    -rw-r--r--  1 nginx nginx  37M Aug  1  2015 access.log-20150626.gz
    -rw-r--r--  1 nginx nginx  43M Sep 11 04:21 access.log-20150801.gz
    -rw-r--r--  1 nginx nginx  43M Oct 19 03:59 access.log-20150911.gz
    -rw-r--r--  1 nginx nginx  43M Dec  1 04:01 access.log-20151019.gz
    -rw-r--r--  1 nginx nginx  45M Jan  7 03:54 access.log-20151201.gz
    -rw-r--r--  1 nginx nginx  46M Feb  2 03:22 access.log-20160107.gz
    -rw-r--r--  1 nginx nginx  42M Feb 24 03:52 access.log-20160202.gz
    -rw-r--r--  1 nginx nginx 516M Feb 24 14:55 access.log-20160224
    -rw-r--r--  1 nginx nginx 242M Mar  6 21:59 access.log
     
  13. SneakyDave

    SneakyDave Member

    84
    14
    8
    Jul 24, 2014
    Ratings:
    +22
    Local Time:
    6:47 PM
    1.0.15
    I noticed my www-php.error.log in /var/log/php-fpm is quite high too. Should centminmod be logrotating that, or should I just set that up myself?
     
  14. eva2000

    eva2000 Administrator Staff Member

    55,225
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,831
    Local Time:
    9:47 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
  15. eva2000

    eva2000 Administrator Staff Member

    55,225
    12,253
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,831
    Local Time:
    9:47 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    edit: oh 123.08stable also has the same fix centminmod/cpcheck.inc at 123.08stable · centminmod/centminmod · GitHub

    so fixed php-fpm logrotate file at /etc/logrotate.d/php-fpm would be like
    Code (Text):
    /var/log/php-fpm/*.log {
            daily
            dateext
            missingok
            rotate 10
            maxsize 500M
            compress
            delaycompress
            notifempty
            postrotate
            /bin/kill -SIGUSR1 $(cat /var/run/php-fpm/php-fpm.pid 2>/dev/null) 2>/dev/null || true
            endscript           
    }