Security fail2ban for Centmin Mod + CSF Firewall / Cloudflare API

Code:

017-08-21 19:18:13,187 fail2ban.failmanager    [14513]: DEBUG   Total # of detected failures: 1604. Current failures from 1 IPs (IP:count): remoteserverip:1
2017-08-21 19:18:13,187 fail2ban.filter         [14513]: DEBUG   Processing line with time:1503343093.0 and ip:remoteserverip
2017-08-21 19:18:13,188 fail2ban.filter         [14513]: INFO    [nginx-req-limit] Found remoteserverip - 2017-08-21 19:18:13
2017-08-21 19:18:13,191 fail2ban.failmanager    [14513]: DEBUG   Total # of detected failures: 1605. Current failures from 1 IPs (IP:count): remoteserverip:2
2017-08-21 19:18:13,415 fail2ban.actions        [14513]: DEBUG   [nginx-req-limit] remoteserverip already banned
2017-08-21 19:18:13,415 fail2ban.actions        [14513]: DEBUG   [nginx-req-limit] remoteserverip already banned
2017-08-21 19:18:13,416 fail2ban.actions        [14513]: DEBUG   [nginx-req-limit] remoteserverip already banned
2017-08-21 19:18:13,417 fail2ban.actions        [14513]: NOTICE  [nginx-req-limit] remoteserverip already banned
2017-08-21 19:18:13,418 fail2ban.actions        [14513]: NOTICE  [nginx-req-limit] remoteserverip already banned

Log seems to be ok but again nothing with Cloudflare

---

 

Maybe something related in the code with a variable like $ip as if i use direct ip it works .....?

Or maybe a local fix that you forgot to push to Github ?

 

This is what i have with Cloudflare enable:

Code:

2017-08-22 04:50:06,793 fail2ban.filter         [14513]: INFO    [nginx-req-limit] Found remoteserverip - 2017-08-22 04:50:06
2017-08-22 04:50:06,796 fail2ban.actions        [14513]: NOTICE  [nginx-req-limit] Ban remoteserverip
2017-08-22 04:50:06,810 fail2ban.filter         [14513]: INFO    [nginx-req-limit-repeat] Found remoteserverip - 2017-08-22 04:50:06
2017-08-22 04:50:06,826 fail2ban.filter         [14513]: INFO    [nginx-req-limit] Found remoteserverip - 2017-08-22 04:50:06
2017-08-22 04:50:06,827 fail2ban.filter         [14513]: INFO    [nginx-req-limit] Found remoteserverip - 2017-08-22 04:50:06

With the csf firewall as default action i have the same output as yours and it works....

The only issue is with the Cloudflare as default action ......

As you can see from the log it does ban the ip but not push correctly to Cloudflare....

But as i can push manually an access rule with no problems on Cloudflare i thought that the issue may be in the code maybe ....

Thank you !!!

 

Yes i am using debug log level .....

I just notice also that the Cloudflare firewall block the server for browser user agent siege as not valid.....

At firewall events ....

Ok anyway i will reinstall the vps and try again.....

Wondering if the limit of 200 rules for free plan is related to firewall events or for access rules or both combined ......

But it should be for Access Rules only !

 

Ok i just get 2 new vps's one for main server and one for attack.....

I follow the instructions and now i can see on fail2ban log:

Code:

2017-08-22 09:39:01,695 fail2ban.filterpyinotify[10217]: DEBUG   <_RawEvent cookie=0 mask=0x2 name='' wd=2 >
2017-08-22 09:39:01,696 fail2ban.filterpyinotify[10217]: DEBUG   Event queue size: 16
2017-08-22 09:39:01,697 fail2ban.filterpyinotify[10217]: DEBUG   <_RawEvent cookie=0 mask=0x2 name='' wd=2 >
2017-08-22 09:39:01,697 fail2ban.filterpyinotify[10217]: DEBUG   Event queue size: 16
2017-08-22 09:39:01,697 fail2ban.filterpyinotify[10217]: DEBUG   <_RawEvent cookie=0 mask=0x2 name='' wd=2 >

But it seems that the ip is banned:

Code:

Status for the jail: nginx-req-limit
|- Filter
|  |- Currently failed: 0
|  |- Total failed:     536
|  `- File list:        /home/nginx/domains/testdomain.com/log/error.log
`- Actions
   |- Currently banned: 1
   |- Total banned:     1
   `- Banned IP list:   remoteserveriphere

Code:

fail2ban-regex error.log /etc/fail2ban/filter.d/nginx-req-limit.conf

Running tests
=============

Use   failregex filter file : nginx-req-limit, basedir: /etc/fail2ban
Use      datepattern : Default Detectors
Use         log file : error.log
Use         encoding : UTF-8


Results
=======

Failregex: 536 total
|-  #) [# of hits] regular expression
|   1) [536] ^\s*\[error\] \d+#\d+: \*\d+ limiting requests, excess: [\d\.]+ by zone "(?:[^"]+)", client: <HOST>,
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
|  [536] {^LN-BEG}ExYear(?P<_sep>[-/.])Month(?P=_sep)Day[T ]24hour:Minute:Second(?:[.,]Microseconds)?(?:\s*Zone offset)?
`-

Lines: 536 lines, 0 ignored, 536 matched, 0 missed

All looks ok but no Cloudflare entry at all

I know that it works for you but i am sure that other users also may have issues with it as i did a fresh install and follow exactly the how to.....

I can provide you if you want server access for both test servers so you can check...

If there is a bug or an issue somewhere other users may benefit from this....

Thank you

At the moment i will have to use the csf action as it works perfect and i will manually add them to Cloudflare

 

Ok so i start testing the csfdeny action

It works great and i got a ban when needed but i can't clear my ip for ever

What i do is clear all logs access and error logs and then remove my ip from csfdeny file and then restart the csf and then i restart also the fail2ban and boom i am banned again when fail2ban started

Don't know where fail2ban store that previous info and use them after restarting it

Any help so i can go ahead with tests?

I also edit the:

Code:

/etc/fail2ban/fail2ban.conf

and set dbpurgeage = to 10 so after 10 seconds it should clear the banned ip's but nothing

Also the default value is:

Code:

# Values: [ SECONDS ] Default: 86400 (24hours)
dbpurgeage = 1d

but it should be at seconds?

 

Ok i did ot using the hard way

Stop fail2ban

then run:

Code:

mv /var/lib/fail2ban/fail2ban.sqlite3 /var/lib/fail2ban/fail2ban.olda

and then start fail2ban ....

That works but don't know why dbpurgeage = 10 doesn't work

 

I note also that nginx-get-f5 is commented (disable) at jails and i rename also the related config file from .conf to .old and then i restart fail2ban and try to scan using Acunetix and i got a ban and at the csf deny entry there is:

Code:

# Added by Fail2Ban for nginx-get-f5 - Wed Aug 23 13:02:55 2017

Also that seems a nice code that you may want to add on your script for an easy way to delete all banned ip's?

fail2ban-remove-ban/fail2ban-remove-ban.sh at master · extremeshok/fail2ban-remove-ban · GitHub

 

Just by removing them from csf deny file ...

Then i read the documentation and it seems the only way to do that is to set the dbpurgeage = 1d to a short value like dbpurgeage = 20 so that should clear all fail2ban banned ip's from the fail2ban.sqlite3 but for me it didn't work

I edit:

Code:

nano /etc/fail2ban/jail.local

and i set there:

Code:

#[nginx-get-f5]
#enabled = true
#filter = nginx-get-f5
#action = csfdeny[name=nginx-get-f5]
##action   = cloudflare
#logpath = /home/nginx/domains/*/log/access.log
#          /usr/local/nginx/logs/*access*.log
#port   = http,https
#maxretry = 15
#findtime = 1
#bantime = 600

and i just rename also the:

Code:

/root/tools/centminmod-fail2ban/filter.d/nginx-get-f5.conf

to

Code:

/root/tools/centminmod-fail2ban/filter.d/nginx-get-f5.old

 

I am wondering also if bantime = 600 for example works also and auto remove the ban.... ?

Anyway i will do more tests next time

 

I think i do the correct steps.....

I will ask for help from a freelancer......

Thanks anyway !!!

 

Well it seems that i found what the problem is

Install using:

Code:

mkdir -p /root/tools
cd /root/tools
git clone https://github.com/centminmod/centminmod-fail2ban
cd centminmod-fail2ban
./fail2ban.sh install

Then i did some settings at this directory:

Code:

/etc/fail2ban/

and some at:

Code:

/root/tools/centminmod-fail2ban/

George can you please verify that all changes must be done at this directory?

Code:

/etc/fail2ban/

Thank you

 

Well as i reinstall the server the only installation was done from /root/tools/centminmod-fail2ban/ but don't know why i did the mistake and i was trying to edit filters there and the cloudflare config file also

Now that i did the changes at the correct path /etc/fail2ban/ all working perfect !!!!!

Now i am thinking of a good value for the nginx get f5 ....

Code:

#maxretry = 15
#findtime = 1
#bantime = 600

Does that mean if a user request / of the site (main page) by hitting the F5 (don't have any idea how the system will now that the request will be from F5 ) more than 15 times then is banned?

Thank you !!!!!