Join the community today
Become a Member

fail2ban & nginx

Discussion in 'Nginx, PHP-FPM & MariaDB MySQL' started by Oxide, Mar 29, 2015.

Tags:
  1. Oxide

    Oxide Active Member

    502
    29
    28
    Mar 19, 2015
    Ratings:
    +54
    Local Time:
    5:23 AM
    From what I've heard, fail2ban is extremly good with nginx. This can block Layer-7 DDoS.

    Have anyone used it with centminmod? I tried to follow some tutorials but for some reason it's not picking up any traffic. I am also using CloudFlare CDN.. but that shouldnt matter, because even CloudFlare didn't get blocked.

    I'm trying to make it so basically if they reach req limit, it will block them for 7200 seconds. As well as if they have invalid user agent (blank, wordpress) it'll block it for 7200 seconds..

    Anyone know how we can do this? I'd be able to pay a small amount to help me out setting it up
     
  2. eva2000

    eva2000 Administrator Staff Member

    29,016
    6,584
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,774
    Local Time:
    5:23 AM
    Nginx 1.13.x
    MariaDB 5.5
    Fail2ban in it's native configuration out of the box can conflict with CSF Firewall that Centmin Mod installs by default CSF Firewall - Centmin Mod - Menu based Nginx installer for CentOS servers as there is overlap in functionality and features between Fail2ban and CSF Firewall.

    I have been working on Fail2ban config that will eventually work with CSF Firewall for Centmin Mod so Fail2ban can do the layer 7 stuff that CSF can't, but it isn't ready for production use and may or may not end up as another Premium Member first feature.

    Unfortunately, I only work on Centmin Mod in my free time and there's not much of that free time right now.

    Of course, if other members here have working Fail2ban + CSF Firewall working, they are free to share their experiences and know how :)
     
    • Like Like x 2
  3. eva2000

    eva2000 Administrator Staff Member

    29,016
    6,584
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,774
    Local Time:
    5:23 AM
    Nginx 1.13.x
    MariaDB 5.5
    Also reason why Fail2ban didn't pick up any traffic is Centmin Mod Nginx is configured to buffer access.log in memory for certain amount of data so it isn't real time logging.

    In you nginx site's vhost yourdomain.com.conf file you will see a buffer log setting on access.log line

    Code:
    buffer=256k;
    
    or something

    removing that part but leaving semi-colon and restarting nginx server will allow real time access.log logging of traffic

    for example first 256KB of data is buffered in memory and flushed to access.log every 60 minutes

    change from

    Code:
    access_log /home/nginx/domains/yourdomain.com/log/access.log combined buffer=256k flush=60m;
    to
    Code:
    access_log /home/nginx/domains/yourdomain.com/log/access.log combined;
    now access.log data is written to log file in real time
     
  4. Oxide

    Oxide Active Member

    502
    29
    28
    Mar 19, 2015
    Ratings:
    +54
    Local Time:
    5:23 AM
    I'll wait and see if someone else has done it, hopefully they have. I really want this, would be great to block out ddos.
     
  5. Oxide

    Oxide Active Member

    502
    29
    28
    Mar 19, 2015
    Ratings:
    +54
    Local Time:
    5:23 AM
    Question, what is CSF Used for by default?
     
  6. eva2000

    eva2000 Administrator Staff Member

    29,016
    6,584
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,774
    Local Time:
    5:23 AM
    Nginx 1.13.x
    MariaDB 5.5
    read what CSF does at CSF Firewall - Centmin Mod - Menu based Nginx installer for CentOS servers it protects entire server and all it's ports from brute force logins etc and also has login failure daemon (lfd)
    Of course you can do alot more as per docs below including banner by country range/extension, banning my known spam lists etc see /etc/csf/csf.conf config file for clues to what else it can do at TCP/UDP levels too including port flooding attacks etc and even lfd clustering so multiple servers using CSF Firewall can share it's banned ips etc across the cluster too :)

    CSF Documentation Links

    readme.txt has sections dealing with the following
     
  7. Oxide

    Oxide Active Member

    502
    29
    28
    Mar 19, 2015
    Ratings:
    +54
    Local Time:
    5:23 AM
    ah so i dont really need it, because i use dome9 and i block all ports to my ip and cloudflare?
     
  8. eva2000

    eva2000 Administrator Staff Member

    29,016
    6,584
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,774
    Local Time:
    5:23 AM
    Nginx 1.13.x
    MariaDB 5.5
    never used dome9 so if it's a firewall, then yes may even conflict with CSF Firewall.. but I've been using CSF Firewall for nearly 10 yrs now so never run a server without CSF Firewall in place .. hence why it's default installed on Centmin Mod stacks :)

    edit: http://www.dome9.com/ interesting service :) expensive if you have more than 40+ servers though heh
     
  9. cloud9

    cloud9 Premium Member Premium Member

    140
    54
    28
    Oct 6, 2015
    England
    Ratings:
    +75
    Local Time:
    8:23 PM
    1.11.x
    10.x
    @eva2000 Did you get any further with the fail2ban config ?
     
  10. eva2000

    eva2000 Administrator Staff Member

    29,016
    6,584
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,774
    Local Time:
    5:23 AM
    Nginx 1.13.x
    MariaDB 5.5
  11. Colin

    Colin Premium Member Premium Member

    108
    32
    28
    Oct 7, 2015
    Sheffield UK
    Ratings:
    +80
    Local Time:
    8:23 PM
    1.11.#
    MariaDB 10.1.#
    @eva2000 add this to the list of 5 things after install ;) But maybe I can cross it off the list :D
     
    • Like Like x 1
  12. eva2000

    eva2000 Administrator Staff Member

    29,016
    6,584
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,774
    Local Time:
    5:23 AM
    Nginx 1.13.x
    MariaDB 5.5
    yeah after letsencrypt addons/acmetool.sh testing :)
     
  13. ethanpil

    ethanpil Active Member

    168
    47
    28
    Nov 8, 2015
    Ratings:
    +87
    Local Time:
    5:23 AM
    Checking in on fail2ban! :)
     
    • Like Like x 1
  14. eva2000

    eva2000 Administrator Staff Member

    29,016
    6,584
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,774
    Local Time:
    5:23 AM
    Nginx 1.13.x
    MariaDB 5.5
    no updates - not there yet :)
     
  15. eva2000

    eva2000 Administrator Staff Member

    29,016
    6,584
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,774
    Local Time:
    5:23 AM
    Nginx 1.13.x
    MariaDB 5.5
  16. ethanpil

    ethanpil Active Member

    168
    47
    28
    Nov 8, 2015
    Ratings:
    +87
    Local Time:
    5:23 AM
    I have recently discovered that CSF/LFD supports custom regex configuration which can be used to add additional checks & blocks that arent setup out of the box. For example, here:
    Defeating Brute Force Attacks by Custom Regex in CSF
    and
    Custom REGEX rules for CSF. - ConfigServer Community Forum

    I have experienced lately a lot of coordinated attacks where one IP or a set of IPs will try common expliot script URLs and receive repeated 404s (Like hundreds or thousands) I am working on a custom regex which will allow me to ban an IP with LFD after 3 404s. Perhaps we can also use this for additional security and features in CMM as well.

    For example, here is a way to ban IP addresses with failed wp-login using LFD:
    regex.custom.pm for WordPress wp-login brute force - ConfigServer Community Forum

    Here are some more great ideas:
    Login Failure Custom Triggers | Juggernaut Security and Firewall Documentation

    Does anyone have some suggestions?

    It doesnt seem very complex at all. Here is the documentation from /usr/local/csf/bin/regex.custom.pm

    Code:
    #!/usr/bin/perl
    ###############################################################################
    # Copyright 2006-2015, Way to the Web Limited
    # URL: http://www.configserver.com
    # Email: sales@waytotheweb.com
    ###############################################################################
    sub custom_line {
        my $line = shift;
        my $lgfile = shift;
    
    # Do not edit before this point
    ###############################################################################
    #
    # Custom regex matching can be added to this file without it being overwritten
    # by csf upgrades. The format is slightly different to regex.pm to cater for
    # additional parameters. You need to specify the log file that needs to be
    # scanned for log line matches in csf.conf under CUSTOMx_LOG. You can scan up
    # to 9 custom logs (CUSTOM1_LOG .. CUSTOM9_LOG)
    #
    # The regex matches in this file will supercede the matches in regex.pm
    #
    # Example:
    #    if (($globlogs{CUSTOM1_LOG}{$lgfile}) and ($line =~ /^\S+\s+\d+\s+\S+ \S+ pure-ftpd: \(\?\@(\d+\.\d+\.\d+\.\d+)\) \[WARNING\] Authentication failed for user/)) {
    #        return ("Failed myftpmatch login from",$1,"myftpmatch","5","20,21","1");
    #    }
    #
    # The return values from this example are as follows:
    #
    # "Failed myftpmatch login from" = text for custom failure message
    # $1 = the offending IP address
    # "myftpmatch" = a unique identifier for this custom rule, must be alphanumeric and have no spaces
    # "5" = the trigger level for blocking
    # "20,21" = the ports to block the IP from in a comma separated list, only used if LF_SELECT enabled. To specify the protocol use 53;udp,53;tcp
    # "1" = n/temporary (n = number of seconds to temporarily block) or 1/permanant IP block, only used if LF_TRIGGER is disabled
    
    
    
    # If the matches in this file are not syntactically correct for perl then lfd
    # will fail with an error. You are responsible for the security of any regex
    # expressions you use. Remember that log file spoofing can exploit poorly
    # constructed regex's
    ###############################################################################
    # Do not edit beyond this point
    
        return 0;
    }
    
    1;
    
    
     
    Last edited: May 13, 2017
    • Informative Informative x 1
  17. eva2000

    eva2000 Administrator Staff Member

    29,016
    6,584
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,774
    Local Time:
    5:23 AM
    Nginx 1.13.x
    MariaDB 5.5
  18. ethanpil

    ethanpil Active Member

    168
    47
    28
    Nov 8, 2015
    Ratings:
    +87
    Local Time:
    5:23 AM
    It doesnt seem complex. I will share what I create. I think it can be a great simple security addon for the system.
     
    • Like Like x 1
  19. eva2000

    eva2000 Administrator Staff Member

    29,016
    6,584
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,774
    Local Time:
    5:23 AM
    Nginx 1.13.x
    MariaDB 5.5
    found some other discussions for this How to set up protection on Wordpress