Discover Centmin Mod today
Register Now

Security CSF Firewall native fail2ban functionality

Discussion in 'System Administration' started by eva2000, Apr 12, 2018.

  1. eva2000

    eva2000 Administrator Staff Member

    33,704
    7,464
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,475
    Local Time:
    1:09 AM
    Nginx 1.13.x
    MariaDB 5.5
    With inspiration from @ethanpil work and my fail2ban work, I am now doing some testing for CSF Firewall native layer 7 application level protections similar to what fail2ban would do but without needing fail2ban to be installed. The thresholds for banning IPs in CSF Firewall for these native rules will need fine tuning and testing so best you test on test Centmin Mod server. Testing and feedback will help improve the rule sets and eventually an addon would be created so folks can optionally install and/or uninstall this CSF Firewall extended functionality ;)

    CSF Firewall Layer 7 Application Level Banning



    CSF Firewall allows custom server log level regex match banning of visitor IP addresses via customising the file at /usr/local/csf/bin/regex.custom.pm and defining up to 9 custom logs in /etc/csf/csf.conf for CSF Firewall's lfd daemon to monitor and scan. There is no Cloudflare Firewall API support added yet, will do that later.

    To setup and test this, please try on test server first with Centmin Mod 123.09beta01. The SSH commands to run to setup are below:
    This will
    1. backup csf firewall /etc/csf/csf.conf config to profile suffix labelled as backup-b4-customregex
    2. backup /usr/local/csf/bin/regex.custom.pm to /usr/local/csf/bin/regex.custom.pm.bak
    3. setup 4 custom log definitions in /etc/csf/csf.conf config file
    4. wget download the regex.custom.pm from hosted gist file
    5. restart csf firewall and lfd daemon
    6. grep for new custom log definitions in lfd log
    Code (Text):
    csf --profile backup backup-b4-customregex
    cp -a /usr/local/csf/bin/regex.custom.pm /usr/local/csf/bin/regex.custom.pm.bak
    egrep 'CUSTOM1_LOG|CUSTOM2_LOG|CUSTOM3_LOG|CUSTOM4_LOG' /etc/csf/csf.conf
    sed -i "s|CUSTOM1_LOG = .*|CUSTOM1_LOG = \"/home/nginx/domains/\*/log/access.log\"|" /etc/csf/csf.conf
    sed -i "s|CUSTOM2_LOG = .*|CUSTOM2_LOG = \"/home/nginx/domains/\*/log/error.log\"|" /etc/csf/csf.conf
    sed -i "s|CUSTOM3_LOG = .*|CUSTOM3_LOG = \"/var/log/nginx/localhost.access.log\"|" /etc/csf/csf.conf
    sed -i "s|CUSTOM4_LOG = .*|CUSTOM4_LOG = \"/var/log/nginx/localhost.error.log\"|" /etc/csf/csf.conf
    egrep 'CUSTOM1_LOG|CUSTOM2_LOG|CUSTOM3_LOG|CUSTOM4_LOG' /etc/csf/csf.conf
    wget -O /usr/local/csf/bin/regex.custom.pm https://gist.github.com/centminmod/f5551b92b8aba768c3b4db84c57e756d/raw/regex.custom.pm
    csf -ra
    fgrep 'nginx' /var/log/lfd.log | tail -100
    

    So 4 custom logs setup
    Code (Text):
    egrep 'CUSTOM1_LOG|CUSTOM2_LOG|CUSTOM3_LOG|CUSTOM4_LOG' /etc/csf/csf.conf
    CUSTOM1_LOG = "/home/nginx/domains/*/log/access.log"
    CUSTOM2_LOG = "/home/nginx/domains/*/log/error.log"
    CUSTOM3_LOG = "/var/log/nginx/localhost.access.log"
    CUSTOM4_LOG = "/var/log/nginx/localhost.error.log"
    

    check for lfd.log log to see if it picked up and is watching the 4 defined custom logs
    Code (Text):
    fgrep 'nginx' /var/log/lfd.log | tail -100
    Apr 12 07:48:26 host lfd[12904]: Watching /home/nginx/domains/demodomain.com/log/access.log...
    Apr 12 07:52:50 host lfd[13141]: Watching /home/nginx/domains/demodomain.com/log/error.log...
    Apr 12 07:52:50 host lfd[13141]: Watching /var/log/nginx/localhost.access.log...
    Apr 12 07:52:50 host lfd[13141]: Watching /var/log/nginx/localhost.error.log...
    


    Custom CSF Firewall Regexs



    Always check for most up to date regex rule sets for the custom /usr/local/csf/bin/regex.custom.pm that is used here.

    You can always download and use latest regex.custom.pm using below 2 commands
    Code (Text):
    wget -O /usr/local/csf/bin/regex.custom.pm https://gist.github.com/centminmod/f5551b92b8aba768c3b4db84c57e756d/raw/regex.custom.pm
    csf -ra
    


    Below is current example regex rule sets for
    1. nginx HTTP 444 status errors ban if more than 5 entries in logs over 24hrs (86400 seconds)
    2. access forbidden by rule matches ban if more than 40 entries entries in logs over 24hrs (86400 seconds). High threshold used to account for new web apps/new nginx users configuring their web apps/scripts so they don't run into issues.
    3. 404 not found matches ban if more than 50 entries in logs over 24hrs (86400 seconds). High threshold used to account for new web apps/new nginx users configuring their web apps/scripts so they don't run into issues.
    4. attempts to access .htaccess or .htpasswd ban if more than 1 entries in logs over 24hrs (86400 seconds)
    5. wordpress plugin wp-fail2ban-redux logged entries ban if more than 2 entries in logs over 24hrs (86400 seconds)
    Code (Text):
    # /home/nginx/domains/*/log/access.log
    # Nginx 444  (Default: 5 errors bans for 24 hours)
    if (($globlogs{CUSTOM1_LOG}{$lgfile}) and ($line =~ /(\S+) -.*[GET|POST|HEAD].*(444)/)) {
        return ("Nginx 444",$1,"nginx_444","5","80,443","86400","0");
    }
    
    # /var/log/nginx/localhost.access.log
    # Nginx 444  (Default: 5 errors bans for 24 hours)
    if (($globlogs{CUSTOM3_LOG}{$lgfile}) and ($line =~ /(\S+) -.*[GET|POST|HEAD].*(444)/)) {
        return ("Nginx 444",$1,"nginx_444","5","80,443","86400","0");
    }
    
    # /home/nginx/domains/*/log/error.log
    # NginX security rules trigger (Default: 40 errors bans for 24 hours)
    if (($globlogs{CUSTOM2_LOG}{$lgfile}) and ($line =~ /.*access forbidden by rule, client: (\S+).*/)) {
        return ("NGINX Security rule triggered from",$1,"nginx_security","40","80,443","86400","0");
    }
    
    # /var/log/nginx/localhost.error.log
    # NginX security rules trigger (Default: 40 errors bans for 24 hours)
    if (($globlogs{CUSTOM4_LOG}{$lgfile}) and ($line =~ /.*access forbidden by rule, client: (\S+).*/)) {
        return ("NGINX Security rule triggered from",$1,"nginx_security","40","80,443","86400","0");
    }
    
    # /home/nginx/domains/*/log/error.log
    # NginX 404 errors (Default: 50 errors bans for 24 hours)
    if (($globlogs{CUSTOM2_LOG}{$lgfile}) and ($line =~ /.*No such file or directory\), client: (\S+),.*/)) {
        return ("NGINX Security rule triggered from",$1,"nginx_404s","50","80,443","86400","0");
    }
    
    # /var/log/nginx/localhost.error.log
    # NginX 404 errors (Default: 50 errors bans for 24 hours)
    if (($globlogs{CUSTOM4_LOG}{$lgfile}) and ($line =~ /.*No such file or directory\), client: (\S+),.*/)) {
        return ("NGINX Security rule triggered from",$1,"nginx_404s","50","80,443","86400","0");
    }
    
    # /home/nginx/domains/*/log/access.log
    #Trying to download htaccess or htpasswd  (Default: 1 error bans for 24 hours)
    if (($globlogs{CUSTOM1_LOG}{$lgfile}) and ($line =~ /.*\.(htpasswd|htaccess).*client: (\S+),.*GET/)) {
        return ("Trying to download .ht files",$1,"nginx_htfiles","1","80,443","86400","0");
    }
    
    # /var/log/nginx/localhost.access.log
    #Trying to download htaccess or htpasswd  (Default: 1 error bans for 24 hours)
    if (($globlogs{CUSTOM3_LOG}{$lgfile}) and ($line =~ /.*\.(htpasswd|htaccess).*client: (\S+),.*GET/)) {
        return ("Trying to download .ht files",$1,"nginx_htfiles","1","80,443","86400","0");
    }
    
    # Wordpress fail2ban plugin https://wordpress.org/plugins/wp-fail2ban-redux/
    # (Default: 2 errors bans for 24 hours)
    if (($globlogs{SYSLOG_LOG}{$lgfile}) and ($line =~ /.*Authentication attempt for unknown user .* from (.*)\n/)) {
      return ("Wordpress unknown user from",$1,"fail2ban_unknownuser","2","80,443","86400","0");
    }
    
    # Wordpress fail2ban plugin https://wordpress.org/plugins/wp-fail2ban-redux/
    # (Default: 2 errors bans for 24 hours)
    if (($globlogs{SYSLOG_LOG}{$lgfile}) and ($line =~ /.*Blocked user enumeration attempt from (.*)\n/)) {
      return ("WordPress user enumeration attempt from",$1,"fail2ban_userenum","2","80,443","86400","0");
    }
    
    # Wordpress fail2ban plugin https://wordpress.org/plugins/wp-fail2ban-redux/
    # (Default: 2 errors bans for 24 hours)
    if (($globlogs{SYSLOG_LOG}{$lgfile}) and ($line =~ /.*Pingback error .* generated from (.*)\n/)) {
      return ("WordPress pingback error",$1,"fail2ban_pingback","2","80,443","86400","0");
    }
    
    # Wordpress fail2ban plugin https://wordpress.org/plugins/wp-fail2ban-redux/
    # (Default: 2 errors bans for 24 hours)
    if (($globlogs{SYSLOG_LOG}{$lgfile}) and ($line =~ /.*Spammed comment from (.*)\n/)) {
      return ("WordPress spam comments from",$1,"fail2ban_spam","2","80,443","86400","0");
    }
    
    # Wordpress fail2ban plugin https://wordpress.org/plugins/wp-fail2ban-redux/
    # (Default: 2 errors bans for 24 hours)
    if (($globlogs{SYSLOG_LOG}{$lgfile}) and ($line =~ /.*XML-RPC multicall authentication failure (.*)\n/)) {
      return ("WordPress XML-RPC multicall fail from",$1,"fail2ban_xmlrpc","5","80,443","86400","0");
    }

    Uninstallation



    Now if you want to remove this custom testing configuration from CSF Firewall follow these SSH command steps where:
    1. profile list will list csf profile backups look for listing name containing _backup_b4_customregex i.e. 1523525787_backup_b4_customregex
    2. then restore /etc/csf/csf.conf config backup
    3. restore default /usr/local/csf/bin/regex.custom.pm.bak to /usr/local/csf/bin/regex.custom.pm
    4. restart csf firewall and lfd daemon
    Code (Text):
    csf --profile list
    csf --profile restore 1523525787_backup_b4_customregex
    \cp -af /usr/local/csf/bin/regex.custom.pm.bak /usr/local/csf/bin/regex.custom.pm
    csf -ra
    


    CSF Firewall Layer 7 Application Level Ban Notes


    • Centmin Mod by default buffers access log writes so they are not written to access logs in real time which would skew CSF Firewall as well as fail2ban usage unless you disable access log buffering (which will increase nginx usage loads).

    Unban IP From CSF Firewall



    CSF Firewall does a temp IP ban first and if temp IP ban records more than 4 occurrences, CSF Firewall does a permanent ban. To unblock an IP you need to remove it from CSF Firewall's permanent and temporarily ban lists using below commands:

    To very if IP is banned for a grep i.e. ip = xxx.xxx.xxx.xxx
    Code (Text):
    csf -g xxx.xxx.xxx.xxx
    

    To unban from temp ban list
    Code (Text):
    csf -tr xxx.xxx.xxx.xxx
    

    To unban form permanent ban list
    Code (Text):
    csf -dr xxx.xxx.xxx.xxx
    

    To verify grep unbanned ip again
    Code (Text):
    csf -g xxx.xxx.xxx.xxx
    
     
    • Winner Winner x 2
    • Informative Informative x 1
  2. eva2000

    eva2000 Administrator Staff Member

    33,704
    7,464
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,475
    Local Time:
    1:09 AM
    Nginx 1.13.x
    MariaDB 5.5
    Example of blocking at CSF Firewall lfd daemon level in /var/log/lfd.log. All custom regex blocks are tagged with label LF_CUSTOMTRIGGER so easy to use fgrep or grep to filter for
    Code (Text):
    fgrep 'LF_CUSTOMTRIGGER' /var/log/lfd.log                         
    Apr 12 08:44:21 hostname lfd[14003]: (nginx_404s) NGINX Security rule triggered from xxx.xxx.xxx.xxx (US/United States/-): 50 in the last 3600 secs - *Blocked in csf* for 86400 secs [LF_CUSTOMTRIGGER]
    Apr 12 09:04:42 hostname lfd[15422]: (nginx_444) Nginx 444 xxx.xxx.xxx.xxx (US/United States/-): 5 in the last 3600 secs - *Blocked in csf* for 86400 secs [LF_CUSTOMTRIGGER]
    Apr 12 09:08:23 hostname lfd[15525]: (PERMBLOCK) xxx.xxx.xxx.xxx (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - *Blocked in csf* [LF_CUSTOMTRIGGER]
    Apr 12 09:09:22 hostname lfd[15605]: (nginx_444) Nginx 444 xxx.xxx.xxx.xxx (US/United States/-): 5 in the last 3600 secs - *Blocked in csf* for 86400 secs [LF_CUSTOMTRIGGER]
    Apr 12 09:32:34 hostname lfd[16537]: (nginx_404s) NGINX Security rule triggered from xxx.xxx.xxx.xxx (US/United States/-): 50 in the last 3600 secs - *Blocked in csf* for 86400 secs [LF_CUSTOMTRIGGER]
    
     
    • Like Like x 1
  3. pamamolf

    pamamolf Well-Known Member

    3,075
    293
    83
    May 31, 2014
    Ratings:
    +524
    Local Time:
    6:09 PM
    Nginx-1.13.x
    MariaDB 10.1.x
    That will be great :)
     
  4. eva2000

    eva2000 Administrator Staff Member

    33,704
    7,464
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,475
    Local Time:
    1:09 AM
    Nginx 1.13.x
    MariaDB 5.5
    Well lets perfect the CSF Firewall regex rules first before looking at Cloudflare Firewall API support :)
     
    • Agree Agree x 1
  5. eva2000

    eva2000 Administrator Staff Member

    33,704
    7,464
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,475
    Local Time:
    1:09 AM
    Nginx 1.13.x
    MariaDB 5.5
..