Welcome to Centmin Mod Community
Register Now

access.log is 10gb?

Discussion in 'Nginx, PHP-FPM & MariaDB MySQL' started by moshbit, Aug 23, 2018.

  1. moshbit

    moshbit Member

    51
    10
    8
    Jun 15, 2014
    Ratings:
    +11
    Local Time:
    3:21 AM
    1.15.0
    MariaDB 10.1.33
    • CentOS Version: CentOS 7 64bit ?
    • Centmin Mod Version Installed: i.e. 123.08stable
    • Nginx Version Installed: i.e. 1.15.0
    • PHP Version Installed: 7.2.5
    • When was last time updated Centmin Mod code base ? : 2 months ago
    • Persistent Config:
    Code:
    LETSENCRYPT_DETECT='y'
    MARCH_TARGETNATIVE='n'
    so my website went offline with a 500 error right as i was uploading a file, and after poking around for a minute i noticed that my disk was full. looking around some more, i find that my access.log file is nearly 11gb in size.

    just curious what could cause this if anyone has any ideas, since it's never happened before and i've been using centminmod for a few years now. thanks!
     
  2. eva2000

    eva2000 Administrator Staff Member

    36,054
    7,910
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,192
    Local Time:
    8:21 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    Nginx access.log as outlined on official Configuration outline page, logs visitor/requests to your web site and Centmin Mod already makes it less verbose and smaller in size as static css/js/html files don't get logged by default. But to get to 11GB in size that is huge and really depends on what type of requests are being logged.

    Which specific access.log was it ? site specific i.e. /home/nginx/domains/yourdomain.com/log/access.log or main hostname access.log at /var/log/nginx/localhost.access.log ?

    You can use Goaccess or ngx_top to parse your access logs for Nginx to get some summary info if you want or manually inspect access.log via tail/head/grep/awk commands.

    Example using ngx_top doing nginx service reload to ensure buffered access logs are written to disk first. For large access.log ngx_top or goaccess may take some time to parse and process.

    Top 20 requests, user agents and remote ip addresses in access.log. Don't share this output publicly as it may contain sensitive request data revealed. But just inspect it privately yourself and focus on requests you find suspicious which you can share publicly i.e. bad bots, crawlers, spiders that hit your site and thus logs too frequently etc.
    Code (Text):
    ngxreload
    ngxtop -l /home/nginx/domains/yourdomain.com/log/access.log --no-follow top request http_user_agent remote_addr -n20
    

    Other example queries you can run for ngx_top here.

    Centmin Mod Log Rotation



    Centmin Mod should be auto log rotating your access and error logs via logrotate. Centmin Mod 123.09beta01 has a better logrotate configuration than Centmin Mod 123.08stable due to improvements.

    What's contents of your Nginx logrotate config file at /etc/logrotate.d/nginx ? You can use cat command to output contents to SSH session window and copy and paste that into CODE bbcode tagged post
    Code (Text):
    cat /etc/logrotate.d/nginx
    

    Centmin Mod 123.09beta01's nginx log rotate would have the following log rotate config file /etc/logrotate.d/nginx contents if you server has more than 1153433 bytes of memory and more than 10485760 bytes of disk free space in /var partition
    Code (Text):
    /var/log/nginx/*.log /usr/local/nginx/logs/*.log /home/nginx/domains/*/log/*.log {
            daily
            dateext
            missingok
            rotate 10
            compress
            delaycompress
            notifempty
            postrotate
            /bin/kill -SIGUSR1 $(cat /usr/local/nginx/logs/nginx.pid 2>/dev/null) 2>/dev/null || true
            endscript         
    }
    

    If you server has less than 1153433 bytes of memory and less than 10485760 bytes of disk free space in /var partition
    Code (Text):
    /var/log/nginx/*.log /usr/local/nginx/logs/*.log /home/nginx/domains/*/log/*.log {
            daily
            dateext
            missingok
            rotate 10
            maxsize 500M
            compress
            delaycompress
            notifempty
            postrotate
            /bin/kill -SIGUSR1 $(cat /usr/local/nginx/logs/nginx.pid 2>/dev/null) 2>/dev/null || true
            endscript         
    }
    

    You can do a debug run of logrotate to see what happens when logrortate is meant to rotate your logs using SSH command
    Code (Text):
    logrotate -d /etc/logrotate.d/nginx
    
     
  3. moshbit

    moshbit Member

    51
    10
    8
    Jun 15, 2014
    Ratings:
    +11
    Local Time:
    3:21 AM
    1.15.0
    MariaDB 10.1.33
    so far, the only thing that looks weird in ngxtop is this line:

    Code:
    | POST /job.php HTTP/1.1               | 12912181 |
    i mean, 12 million requests seems like a bit much? all the others in the list are around 5-20 thousand.

    here's my logrotate config file:

    Code:
    cat /etc/logrotate.d/nginx
    /var/log/nginx/*.log /usr/local/nginx/logs/*.log /home/nginx/domains/*/log/*.log {
            daily
            dateext
            missingok
            rotate 10
            maxsize 500M
            compress
            delaycompress
            notifempty
            postrotate
            /bin/kill -SIGUSR1 $(cat /usr/local/nginx/logs/nginx.pid 2>/dev/null) 2>/dev/null || true
            endscript
    }
    
    as far as i can tell i see nothing unusual there.

    it looks like i have a permissions error on the log folder when running the debug of logrotate:

    Code:
    considering log /home/nginx/domains/squattheplanet.com/log/access.log
    error: skipping "/home/nginx/domains/squattheplanet.com/log/access.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation.
    considering log /home/nginx/domains/squattheplanet.com/log/convertimage.log
    error: skipping "/home/nginx/domains/squattheplanet.com/log/convertimage.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation.
    considering log /home/nginx/domains/squattheplanet.com/log/error.log
    error: skipping "/home/nginx/domains/squattheplanet.com/log/error.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation.
    i did make a change to those folder permissions to allow a xenforo addon to create a log in that folder. what should the proper permissions for this folder be?

    thanks for your time in assisting me with this.
     
  4. rdan

    rdan Premium Member Premium Member

    4,255
    1,034
    113
    May 25, 2014
    Ratings:
    +1,486
    Local Time:
    6:21 PM
    Mainline
    10.2
    • Informative Informative x 1
  5. eva2000

    eva2000 Administrator Staff Member

    36,054
    7,910
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,192
    Local Time:
    8:21 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    what are the changed permissions ? they should be owned by nginx user/group and you shouldn't have to change ownership as php-fpm runs as nginx user too

    output for
    Code (Text):
    ls -lah /home/nginx/domains/squattheplanet.com
    

    and
    Code (Text):
    getfacl -L --absolute-names /home/nginx/domains/squattheplanet.com/log
    


    example of default permissions for dummy domain
    Code (Text):
    getfacl -L --absolute-names /home/nginx/domains/domain.com/log
    # file: /home/nginx/domains/domain.com/log
    # owner: nginx
    # group: nginx
    # flags: -s-
    user::rwx
    group::r-x
    other::---
    
     
  6. moshbit

    moshbit Member

    51
    10
    8
    Jun 15, 2014
    Ratings:
    +11
    Local Time:
    3:21 AM
    1.15.0
    MariaDB 10.1.33
    thank you for the suggestion, but it seems that code no longer exists in XF 2.0.9:

    Code:
    <?php
    
    namespace XF\Service\Post;
    
    use XF\Entity\Post;
    use XF\Service\AbstractNotifier;
    
    class Notifier extends AbstractNotifier
    {
        /**
         * @var Post
         */
        protected $post;
    
        protected $actionType;
    
        public function __construct(\XF\App $app, Post $post, $actionType)
        {
            parent::__construct($app);
    
            switch ($actionType)
            {
                case 'reply':
                case 'thread':
                    break;
    
                default:
                    throw new \InvalidArgumentException("Unknown action type '$actionType'");
            }
    
            $this->actionType = $actionType;
            $this->post = $post;
        }
    
        public static function createForJob(array $extraData)
        {
            $post = \XF::app()->find('XF:Post', $extraData['postId'], ['Thread', 'Thread.Forum']);
            if (!$post)
            {
                return null;
            }
    
            return \XF::service('XF:Post\Notifier', $post, $extraData['actionType']);
        }
    
        protected function getExtraJobData()
        {
            return [
                'postId' => $this->post->post_id,
                'actionType' => $this->actionType
            ];
        }
    
        protected function loadNotifiers()
        {
            $notifiers = [
                'quote' => $this->app->notifier('XF:Post\Quote', $this->post),
                'mention' => $this->app->notifier('XF:Post\Mention', $this->post),
                'forumWatch' => $this->app->notifier('XF:Post\ForumWatch', $this->post, $this->actionType),
            ];
    
            // if this is not the last post, then another notification would have been triggered already
            if ($this->post->isLastPost())
            {
                $notifiers['threadWatch'] = $this->app->notifier('XF:Post\ThreadWatch', $this->post, $this->actionType);
            }
    
            return $notifiers;
        }
    
        protected function loadExtraUserData(array $users)
        {
            $permCombinationIds = [];
            foreach ($users AS $user)
            {
                $id = $user->permission_combination_id;
                $permCombinationIds[$id] = $id;
            }
    
            $this->app->permissionCache()->cacheMultipleContentPermsForContent(
                $permCombinationIds,
                'node', $this->post->Thread->node_id
            );
        }
    
        protected function canUserViewContent(\XF\Entity\User $user)
        {
            return \XF::asVisitor(
                $user,
                function() { return $this->post->canView(); }
            );
        }
    
        public function skipUsersWatchingForum(\XF\Entity\Forum $forum)
        {
            $watchers = $this->db()->fetchAll("
                SELECT user_id, send_alert, send_email
                FROM xf_forum_watch
                WHERE node_id = ?
                    AND (send_alert = 1 OR send_email = 1)
            ", $forum->node_id);
    
            foreach ($watchers AS $watcher)
            {
                if ($watcher['send_alert'])
                {
                    $this->setUserAsAlerted($watcher['user_id']);
                }
                if ($watcher['send_email'])
                {
                    $this->setUserAsEmailed($watcher['user_id']);
                }
            }
        }
    }
    Code:
    getfacl -L --absolute-names /home/nginx/domains/squattheplanet.com/log
    # file: /home/nginx/domains/squattheplanet.com/log
    # owner: nginx
    # group: nginx
    # flags: -s-
    user::rwx
    group::rwx
    other::---
     
  7. rdan

    rdan Premium Member Premium Member

    4,255
    1,034
    113
    May 25, 2014
    Ratings:
    +1,486
    Local Time:
    6:21 PM
    Mainline
    10.2
    It exist.
     
  8. moshbit

    moshbit Member

    51
    10
    8
    Jun 15, 2014
    Ratings:
    +11
    Local Time:
    3:21 AM
    1.15.0
    MariaDB 10.1.33
    I'm sorry @rdan but if it exists, I'm not finding it. I've posted the above code to demonstrate that it has changed. If you can post in more detail, that would be more useful.
     
  9. eva2000

    eva2000 Administrator Staff Member

    36,054
    7,910
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,192
    Local Time:
    8:21 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    maybe download fresh zip source and check there. Could be you didn't upload 100% all XF 2 updated files to your server and looking at a older copy of the code ?
     
  10. eva2000

    eva2000 Administrator Staff Member

    36,054
    7,910
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,192
    Local Time:
    8:21 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    what about recursively to see actual files' permissions
    Code (Text):
    getfacl -R -L --absolute-names /home/nginx/domains/squattheplanet.com/log
    
     
  11. moshbit

    moshbit Member

    51
    10
    8
    Jun 15, 2014
    Ratings:
    +11
    Local Time:
    3:21 AM
    1.15.0
    MariaDB 10.1.33
    The XF file health check results come out clean. I wish the solution was that easy, but the issue @rdan is referring to seems to have been fixed after XF 2.0.5 (I'm on 2.0.9).

    Code:
     getfacl -R -L --absolute-names /home/nginx/domains/squattheplanet.com/log
    # file: /home/nginx/domains/squattheplanet.com/log
    # owner: nginx
    # group: nginx
    # flags: -s-
    user::rwx
    group::rwx
    other::---
    
    # file: /home/nginx/domains/squattheplanet.com/log/error.log-20171218.gz
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    # file: /home/nginx/domains/squattheplanet.com/log/error.log
    # owner: nginx
    # group: nginx
    user::rwx
    group::rwx
    other::---
    
    # file: /home/nginx/domains/squattheplanet.com/log/error.log-20171222.gz
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    # file: /home/nginx/domains/squattheplanet.com/log/error.log-20180108.gz
    # owner: nginx
    # group: nginx
    user::rwx
    group::rwx
    other::---
    
    # file: /home/nginx/domains/squattheplanet.com/log/error.log-20180108
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    # file: /home/nginx/domains/squattheplanet.com/log/convertimage.log
    # owner: root
    # group: nginx
    user::rwx
    group::rwx
    other::r-x
    
    # file: /home/nginx/domains/squattheplanet.com/log/access.log-20171223.gz
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    # file: /home/nginx/domains/squattheplanet.com/log/error.log-20171224.gz
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    # file: /home/nginx/domains/squattheplanet.com/log/error.log-20171219.gz
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    # file: /home/nginx/domains/squattheplanet.com/log/error.log-20171221.gz
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    # file: /home/nginx/domains/squattheplanet.com/log/access.log-20171222.gz
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    # file: /home/nginx/domains/squattheplanet.com/log/access.log-20171218.gz
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    # file: /home/nginx/domains/squattheplanet.com/log/access.log-20180108
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    # file: /home/nginx/domains/squattheplanet.com/log/access.log-20171219.gz
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    # file: /home/nginx/domains/squattheplanet.com/log/access.log-20171221.gz
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    # file: /home/nginx/domains/squattheplanet.com/log/access.log-20171226.gz
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    # file: /home/nginx/domains/squattheplanet.com/log/error.log-20171223.gz
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    # file: /home/nginx/domains/squattheplanet.com/log/error.log-20171226.gz
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    # file: /home/nginx/domains/squattheplanet.com/log/access.log
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    # file: /home/nginx/domains/squattheplanet.com/log/error.log-20171220.gz
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    # file: /home/nginx/domains/squattheplanet.com/log/access.log-20171225.gz
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    # file: /home/nginx/domains/squattheplanet.com/log/error.log-20171225.gz
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    # file: /home/nginx/domains/squattheplanet.com/log/access.log-20171224.gz
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    # file: /home/nginx/domains/squattheplanet.com/log/access.log-20180108.gz
    # owner: nginx
    # group: nginx
    user::rwx
    group::rwx
    other::---
    
    # file: /home/nginx/domains/squattheplanet.com/log/access.log-20171220.gz
    # owner: nginx
    # group: nginx
    user::rw-
    group::r--
    other::r--
    
    
     
  12. eva2000

    eva2000 Administrator Staff Member

    36,054
    7,910
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,192
    Local Time:
    8:21 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    did you recently change the log permissions (chmod and chown) ? as most of those logs including access.log show correct user/group and file permissions besides log/convertimage.log which is owned by root user and nginx group with 774 permissions

    what's re-running logrotate test output this time
    Code (Text):
    logrotate -d /etc/logrotate.d/nginx
    
     
  13. moshbit

    moshbit Member

    51
    10
    8
    Jun 15, 2014
    Ratings:
    +11
    Local Time:
    3:21 AM
    1.15.0
    MariaDB 10.1.33
    i did change the permissions for the convertimage.log file, and possibly the parent folder? i don't quite remember. i did this because the Convert Image XF addon by AndyB wasn't able to write the file.

    Code:
    error: skipping "/home/nginx/domains/squattheplanet.com/log/access.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation.
    considering log /home/nginx/domains/squattheplanet.com/log/convertimage.log
    error: skipping "/home/nginx/domains/squattheplanet.com/log/convertimage.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation.
    considering log /home/nginx/domains/squattheplanet.com/log/error.log
    error: skipping "/home/nginx/domains/squattheplanet.com/log/error.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation.
    rotating log /home/nginx/domains/philipderrick.com/log/error.log, log->rotateCount is 10
    dateext suffix '-20180912'
    glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
    compressing log with: /bin/gzip
    renaming /home/nginx/domains/philipderrick.com/log/error.log to /home/nginx/domains/philipderrick.com/log/error.log-20180912
    running postrotate script
    running script with arg /home/nginx/domains/philipderrick.com/log/error.log: "
            /bin/kill -SIGUSR1 $(cat /usr/local/nginx/logs/nginx.pid 2>/dev/null) 2>/dev/null || true
    
     
  14. eva2000

    eva2000 Administrator Staff Member

    36,054
    7,910
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,192
    Local Time:
    8:21 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    and if you change /home/nginx/domains/squattheplanet.com/log back to 750 permissions and re-running logrotate test ?
     
  15. moshbit

    moshbit Member

    51
    10
    8
    Jun 15, 2014
    Ratings:
    +11
    Local Time:
    3:21 AM
    1.15.0
    MariaDB 10.1.33
    Code:
    rotating log /home/nginx/domains/squattheplanet.com/log/access.log, log->rotateCount is 10
    dateext suffix '-20180918'
    glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
    compressing log with: /bin/gzip
    renaming /home/nginx/domains/squattheplanet.com/log/access.log to /home/nginx/domains/squattheplanet.com/log/access.log-20180918
    running postrotate script
    running script with arg /home/nginx/domains/squattheplanet.com/log/access.log: "
            /bin/kill -SIGUSR1 $(cat /usr/local/nginx/logs/nginx.pid 2>/dev/null) 2>/dev/null || true
    "
    removing old log /home/nginx/domains/squattheplanet.com/log/access.log-20171218.gz
    rotating log /home/nginx/domains/squattheplanet.com/log/convertimage.log, log->rotateCount is 10
    dateext suffix '-20180918'
    glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
    glob finding logs to compress failed
    glob finding old rotated logs failed
    renaming /home/nginx/domains/squattheplanet.com/log/convertimage.log to /home/nginx/domains/squattheplanet.com/log/convertimage.log-20180918
    running postrotate script
    running script with arg /home/nginx/domains/squattheplanet.com/log/convertimage.log: "
            /bin/kill -SIGUSR1 $(cat /usr/local/nginx/logs/nginx.pid 2>/dev/null) 2>/dev/null || true
    "
    rotating log /home/nginx/domains/squattheplanet.com/log/error.log, log->rotateCount is 10
    dateext suffix '-20180918'
    glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
    compressing log with: /bin/gzip
    renaming /home/nginx/domains/squattheplanet.com/log/error.log to /home/nginx/domains/squattheplanet.com/log/error.log-20180918
    running postrotate script
    running script with arg /home/nginx/domains/squattheplanet.com/log/error.log: "
            /bin/kill -SIGUSR1 $(cat /usr/local/nginx/logs/nginx.pid 2>/dev/null) 2>/dev/null || true
    "
    removing old log /home/nginx/domains/squattheplanet.com/log/error.log-20171218.gz
    it looks like maybe the glob pattern isn't matching now? it's progress at least. dunno if it helps, but here's the current directory listing:

    Code:
    # ls -al log
    total 7511860
    drwxr-s--- 2 nginx nginx       4096 Sep 12 22:27 .
    drwxrws--- 6 nginx nginx       4096 Mar  7  2018 ..
    -rw-r--r-- 1 nginx nginx 7359714483 Sep 18 01:34 access.log
    -rw-r--r-- 1 nginx nginx    2564687 Dec 18  2017 access.log-20171218.gz
    -rw-r--r-- 1 nginx nginx    2802374 Dec 19  2017 access.log-20171219.gz
    -rw-r--r-- 1 nginx nginx    3497415 Dec 20  2017 access.log-20171220.gz
    -rw-r--r-- 1 nginx nginx    2991545 Dec 21  2017 access.log-20171221.gz
    -rw-r--r-- 1 nginx nginx    3091873 Dec 22  2017 access.log-20171222.gz
    -rw-r--r-- 1 nginx nginx    2931490 Dec 23  2017 access.log-20171223.gz
    -rw-r--r-- 1 nginx nginx    2765194 Dec 24  2017 access.log-20171224.gz
    -rw-r--r-- 1 nginx nginx    2857444 Dec 25  2017 access.log-20171225.gz
    -rw-r--r-- 1 nginx nginx    2566902 Dec 26  2017 access.log-20171226.gz
    -rw-r--r-- 1 nginx nginx   21480139 Jan  8  2018 access.log-20180108
    -rwxrwx--- 1 nginx nginx          0 Jan  9  2018 access.log-20180108.gz
    -rwxrwxr-x 1 root  nginx      14956 Sep 17 18:45 convertimage.log
    -rwxrwx--- 1 nginx nginx  284240382 Sep 18 01:33 error.log
    -rw-r--r-- 1 nginx nginx      36686 Dec 18  2017 error.log-20171218.gz
    -rw-r--r-- 1 nginx nginx      32987 Dec 19  2017 error.log-20171219.gz
    -rw-r--r-- 1 nginx nginx      31446 Dec 20  2017 error.log-20171220.gz
    -rw-r--r-- 1 nginx nginx      34756 Dec 21  2017 error.log-20171221.gz
    -rw-r--r-- 1 nginx nginx      43572 Dec 22  2017 error.log-20171222.gz
    -rw-r--r-- 1 nginx nginx      34619 Dec 23  2017 error.log-20171223.gz
    -rw-r--r-- 1 nginx nginx      32167 Dec 24  2017 error.log-20171224.gz
    -rw-r--r-- 1 nginx nginx      33132 Dec 25  2017 error.log-20171225.gz
    -rw-r--r-- 1 nginx nginx      35876 Dec 26  2017 error.log-20171226.gz
    -rw-r--r-- 1 nginx nginx     249275 Jan  8  2018 error.log-20180108
    -rwxrwx--- 1 nginx nginx          0 Jan  9  2018 error.log-20180108.gz
    i don't remember when i made the change exactly, but december of 2017 or round about then would make sense, since that was when i upgraded to xf2 (and installed andy's convert image addon).
     
  16. eva2000

    eva2000 Administrator Staff Member

    36,054
    7,910
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,192
    Local Time:
    8:21 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    AFAIK, glob pattern for convertimage.log gives error because there wasn't a previous rotated convertimage-2018xxxx.log to rotate so normal do instead of debug run do actual logrotate run now
    Code (Text):
    logrotate -fv /etc/logrotate.d/nginx
    

    and check /home/nginx/domains/squattheplanet.com/log
    Code (Text):
    ls -lah /home/nginx/domains/squattheplanet.com/log
    
     
    • Like Like x 1
  17. moshbit

    moshbit Member

    51
    10
    8
    Jun 15, 2014
    Ratings:
    +11
    Local Time:
    3:21 AM
    1.15.0
    MariaDB 10.1.33
    Code:
    running postrotate script
    rotating log /home/nginx/domains/squattheplanet.com/log/access.log, log->rotateCount is 10
    dateext suffix '-20180918'
    glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
    compressing log with: /bin/gzip
    error: error creating output file /home/nginx/domains/squattheplanet.com/log/access.log-20180108.gz: File exists
    rotating log /home/nginx/domains/squattheplanet.com/log/error.log, log->rotateCount is 10
    dateext suffix '-20180918'
    glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
    compressing log with: /bin/gzip
    error: error creating output file /home/nginx/domains/squattheplanet.com/log/error.log-20180108.gz: File exists
    Removing /var/log/wtmp from state file, because it does not exist and has not been rotated for one year
    Code:
    ls -lah /home/nginx/domains/squattheplanet.com/log
    total 7.4G
    drwxr-s--- 2 nginx nginx 4.0K Sep 18 03:30 .
    drwxrws--- 6 nginx nginx 4.0K Mar  7  2018 ..
    -rw-r--r-- 1 nginx nginx 7.1G Sep 18 15:23 access.log
    -rw-r--r-- 1 nginx nginx 2.5M Dec 18  2017 access.log-20171218.gz
    -rw-r--r-- 1 nginx nginx 2.7M Dec 19  2017 access.log-20171219.gz
    -rw-r--r-- 1 nginx nginx 3.4M Dec 20  2017 access.log-20171220.gz
    -rw-r--r-- 1 nginx nginx 2.9M Dec 21  2017 access.log-20171221.gz
    -rw-r--r-- 1 nginx nginx 3.0M Dec 22  2017 access.log-20171222.gz
    -rw-r--r-- 1 nginx nginx 2.8M Dec 23  2017 access.log-20171223.gz
    -rw-r--r-- 1 nginx nginx 2.7M Dec 24  2017 access.log-20171224.gz
    -rw-r--r-- 1 nginx nginx 2.8M Dec 25  2017 access.log-20171225.gz
    -rw-r--r-- 1 nginx nginx 2.5M Dec 26  2017 access.log-20171226.gz
    -rw-r--r-- 1 nginx nginx  21M Jan  8  2018 access.log-20180108
    -rwxrwx--- 1 nginx nginx    0 Jan  9  2018 access.log-20180108.gz
    -rwxrwxr-x 1 root  nginx    0 Sep 18 03:30 convertimage.log
    -rwxrwxr-x 1 root  nginx  15K Sep 17 18:45 convertimage.log-20180918
    -rwxrwx--- 1 nginx nginx 272M Sep 18 15:20 error.log
    -rw-r--r-- 1 nginx nginx  36K Dec 18  2017 error.log-20171218.gz
    -rw-r--r-- 1 nginx nginx  33K Dec 19  2017 error.log-20171219.gz
    -rw-r--r-- 1 nginx nginx  31K Dec 20  2017 error.log-20171220.gz
    -rw-r--r-- 1 nginx nginx  34K Dec 21  2017 error.log-20171221.gz
    -rw-r--r-- 1 nginx nginx  43K Dec 22  2017 error.log-20171222.gz
    -rw-r--r-- 1 nginx nginx  34K Dec 23  2017 error.log-20171223.gz
    -rw-r--r-- 1 nginx nginx  32K Dec 24  2017 error.log-20171224.gz
    -rw-r--r-- 1 nginx nginx  33K Dec 25  2017 error.log-20171225.gz
    -rw-r--r-- 1 nginx nginx  36K Dec 26  2017 error.log-20171226.gz
    -rw-r--r-- 1 nginx nginx 244K Jan  8  2018 error.log-20180108
    -rwxrwx--- 1 nginx nginx    0 Jan  9  2018 error.log-20180108.gz
    maybe i should download and clear out the current logs?
     
  18. eva2000

    eva2000 Administrator Staff Member

    36,054
    7,910
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,192
    Local Time:
    8:21 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    what happens if you delete these 2 first
    Code (Text):
    cd /home/nginx/domains/squattheplanet.com/log
    rm /home/nginx/domains/squattheplanet.com/log/access.log-20180108.gz
    rm /home/nginx/domains/squattheplanet.com/log/error.log-20180108.gz
    logrotate -fv /etc/logrotate.d/nginx
    
     
  19. moshbit

    moshbit Member

    51
    10
    8
    Jun 15, 2014
    Ratings:
    +11
    Local Time:
    3:21 AM
    1.15.0
    MariaDB 10.1.33
    Code:
    renaming /home/nginx/domains/philipderrick.com/log/wp_updater-190918-160312.log to /home/nginx/domains/philipderrick.com/log/wp_updater-190918-160312.log-20180919
    running postrotate script
    rotating log /home/nginx/domains/squattheplanet.com/log/access.log, log->rotateCount is 10
    dateext suffix '-20180919'
    glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
    compressing log with: /bin/gzip
    renaming /home/nginx/domains/squattheplanet.com/log/access.log to /home/nginx/domains/squattheplanet.com/log/access.log-20180919
    running postrotate script
    removing old log /home/nginx/domains/squattheplanet.com/log/access.log-20171218.gz
    rotating log /home/nginx/domains/squattheplanet.com/log/error.log, log->rotateCount is 10
    dateext suffix '-20180919'
    glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
    compressing log with: /bin/gzip
    renaming /home/nginx/domains/squattheplanet.com/log/error.log to /home/nginx/domains/squattheplanet.com/log/error.log-20180919
    running postrotate script
    removing old log /home/nginx/domains/squattheplanet.com/log/error.log-20171218.gz
    
    Code:
     ls -al
    total 8096800
    drwxr-s--- 2 nginx nginx       4096 Sep 19 18:27 .
    drwxrws--- 6 nginx nginx       4096 Mar  7  2018 ..
    -rw-r--r-- 1 nginx nginx          0 Sep 19 18:27 access.log
    -rw-r--r-- 1 nginx nginx    2802374 Dec 19  2017 access.log-20171219.gz
    -rw-r--r-- 1 nginx nginx    3497415 Dec 20  2017 access.log-20171220.gz
    -rw-r--r-- 1 nginx nginx    2991545 Dec 21  2017 access.log-20171221.gz
    -rw-r--r-- 1 nginx nginx    3091873 Dec 22  2017 access.log-20171222.gz
    -rw-r--r-- 1 nginx nginx    2931490 Dec 23  2017 access.log-20171223.gz
    -rw-r--r-- 1 nginx nginx    2765194 Dec 24  2017 access.log-20171224.gz
    -rw-r--r-- 1 nginx nginx    2857444 Dec 25  2017 access.log-20171225.gz
    -rw-r--r-- 1 nginx nginx    2566902 Dec 26  2017 access.log-20171226.gz
    -rw-r--r-- 1 nginx nginx    2272511 Jan  8  2018 access.log-20180108.gz
    -rw-r--r-- 1 nginx nginx 7977611649 Sep 19 18:27 access.log-20180919
    -rwxrwxr-x 1 root  nginx          0 Sep 19 03:14 convertimage.log
    -rwxrwxr-x 1 root  nginx       4021 Sep 17 18:45 convertimage.log-20180918.gz
    -rwxrwxr-x 1 root  nginx         66 Sep 18 19:57 convertimage.log-20180919
    -rw-r--r-- 1 nginx nginx          0 Sep 19 18:27 error.log
    -rw-r--r-- 1 nginx nginx      32987 Dec 19  2017 error.log-20171219.gz
    -rw-r--r-- 1 nginx nginx      31446 Dec 20  2017 error.log-20171220.gz
    -rw-r--r-- 1 nginx nginx      34756 Dec 21  2017 error.log-20171221.gz
    -rw-r--r-- 1 nginx nginx      43572 Dec 22  2017 error.log-20171222.gz
    -rw-r--r-- 1 nginx nginx      34619 Dec 23  2017 error.log-20171223.gz
    -rw-r--r-- 1 nginx nginx      32167 Dec 24  2017 error.log-20171224.gz
    -rw-r--r-- 1 nginx nginx      33132 Dec 25  2017 error.log-20171225.gz
    -rw-r--r-- 1 nginx nginx      35876 Dec 26  2017 error.log-20171226.gz
    -rw-r--r-- 1 nginx nginx      25555 Jan  8  2018 error.log-20180108.gz
    -rwxrwx--- 1 nginx nginx  287363742 Sep 19 18:25 error.log-20180919
    
    so it looks like it rotated out, but didn't compress. maybe cause it's so huge?
     
  20. eva2000

    eva2000 Administrator Staff Member

    36,054
    7,910
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,192
    Local Time:
    8:21 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    first rotated log isn't compressed, on 2nd rotation it will be so if you re-run forced logrotate one more time should be compressed
     
..