Want more timely Centmin Mod News Updates?
Become a Member

Beta Branch revise inc/csftweaks.inc

Discussion in 'Centmin Mod Github Commits' started by eva2000, May 22, 2017.

  1. eva2000

    eva2000 Administrator Staff Member

    36,424
    7,994
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,308
    Local Time:
    3:03 AM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    revise inc/csftweaks.inc

    - More dynamic formula for non-openvz system based DENY_IP_LIMIT & DENY_TEMP_IP_LIMIT settings based on system resources available.
    - Add tools/csf-advancetweaks.sh which can be manually run on non-openvz systems which have linux kernel that supports IPSET to update existing Centmin Mod LEMP system's CSF Firewall settings for DENY_IP_LIMIT & DENY_TEMP_IP_LIMIT with these committed changes. The script also extends /etc/csf/csf.blocklists for blocklist.de additional block ip lists which are narrower than CSF default DBEALL chain provided. These additional blocklist.de chains are taken from http://www.blocklist.de/en/export.html

    Continue reading...

    123.09beta01 branch
     
  2. eva2000

    eva2000 Administrator Staff Member

    36,424
    7,994
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,308
    Local Time:
    3:03 AM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    For existing Centmin Mod 123.09beta01+ and higher LEMP stack users wanting to update their CSF Firewall's DENY_IP_LIMIT & DENY_TEMP_IP_LIMIT with the commit's updated values, you can run the newly added tools/csf-advancetweaks.sh script after you update centmin mod code via centmin.sh menu option 23 submenu option 2 and then exit centmin.sh and change into /usr/local/src/centminmod/tools directory to run the script.

    Additionally, the script also extends /etc/csf/csf.blocklists for blocklist.de additional block ip lists which are narrower than CSF default DBEALL chain provided. CSF by default disables all blocklists listed in /etc/csf/csf.blocklists. The default CSF provided DBEALL chain, when manually enabled uses all ips listed in blocklist.de database and that can be 33,000+ IPs per 48hr period found. Adding too many blocked IPs to a firewall like CSF Firewall or IPTables will slow a system down alot if it doesn't support IPSET hashed ips (only non-OpenVZ systems support IPSET as long as Linux Kernel supports IPSET). For example, enabling DBESTRONGIPS and DBEBRUTEFORCE chains only instead of DBEALL, means DBESTRONGIPS and DBEBRUTEFORCE chains only contain 395 and 119 IP addresses respectively for below example run.

    These additional blocklist.de chains are taken from www.blocklist.de -- Export all Attacker-IPs from the last 48 Hours.

    The extended listing now includes these additional blocklists in /etc/csf/csf.blocklists
    Code (Text):
    # If you do not want to use Blocklist.de large IP list from second list for
    # BDEALL iptables chain name list, you can use one of these listings
    # for narrower set of IPs to block for specific attack types outlined
    # here http://www.blocklist.de/en/export.html
    # DO NOT enable second list BDEALL as well as duplicating IP blocks by
    # enabling below lists. Use second list OR one if the below narrower
    # lists NOT both
    
    #IP addresses which have been reported within the last 48 hours as
    #having run attacks on the service SSH.
    #BDESSH|86400|0|https://lists.blocklist.de/lists/ssh.txt
    
    #IP addresses which have been reported within the last 48 hours as
    #having run attacks on the service Mail, Postfix.
    #BDEMAIL|86400|0|https://lists.blocklist.de/lists/mail.txt
    
    #IP addresses which have been reported within the last 48 hours as
    #having run attacks on the service Apache, Apache-DDOS, RFI-Attacks
    #BDEAPACHE|86400|0|https://lists.blocklist.de/lists/apache.txt
    
    #IP addresses which have been reported within the last 48 hours
    #for attacks on the Service imap, sasl, pop3
    #BDEIMAP|86400|0|https://lists.blocklist.de/lists/imap.txt
    
    #IP addresses which have been reported within the last 48 hours
    #for attacks on the Service FTP.
    #BDEFTP|86400|0|https://lists.blocklist.de/lists/ftp.txt
    
    #IP addresses that tried to login in a SIP-, VOIP- or Asterisk-Server
    #and are inclueded in the IPs-List from http://www.infiltrated.net/
    #BDESIP|86400|0|https://lists.blocklist.de/lists/sip.txt
    
    #IP addresses which have been reported within the last 48 hours as
    #having run attacks attacks on the RFI-Attacks, REG-Bots, IRC-Bots
    #or BadBots
    #BDEBOTS|86400|0|https://lists.blocklist.de/lists/bots.txt
    
    #IP addresses older then 2 month & have more then 5.000 attacks.
    #BDESTRONGIPS|86400|0|https://lists.blocklist.de/lists/strongips.txt
    
    #IP addresses for ircbot
    #BDEIRCBOT|86400|0|https://lists.blocklist.de/lists/ircbot.txt
    
    #IP addresses which attacks Joomlas, Wordpress and other Web-Logins
    #with Brute-Force Logins
    #BDEBRUTEFORCE|86400|0|https://lists.blocklist.de/lists/bruteforcelogin.txt
    

    If you manually enable one of these extended lists you need to restart CSF Firewall and lfd daemon service.
    Code (Text):
    csf -r
    service lfd restart
    

    or just
    Code (Text):
    csf -ra
    

    So example enable BDESTRONGIPS and DBEBRUTEFORCE chains uncomment by removing hash # in front of the line only (not the comment)

    within /etc/csf/csf.blocklists change from
    Code (Text):
    #IP addresses older then 2 month & have more then 5.000 attacks.
    #BDESTRONGIPS|86400|0|https://lists.blocklist.de/lists/strongips.txt
    
    #IP addresses which attacks Joomlas, Wordpress and other Web-Logins
    #with Brute-Force Logins
    #BDEBRUTEFORCE|86400|0|https://lists.blocklist.de/lists/bruteforcelogin.txt
    

    change to
    Code (Text):
    #IP addresses older then 2 month & have more then 5.000 attacks.
    BDESTRONGIPS|86400|0|https://lists.blocklist.de/lists/strongips.txt
    
    #IP addresses which attacks Joomlas, Wordpress and other Web-Logins
    #with Brute-Force Logins
    BDEBRUTEFORCE|86400|0|https://lists.blocklist.de/lists/bruteforcelogin.txt
    

    restart CSF Firewall and lfd daemon service.
    Code (Text):
    csf -r
    service lfd restart
    

    or just
    Code (Text):
    csf -ra
    

    If you are on non-openvz system with linux kernel that supports IPSET, then these extended blocklist.de lists are added to /etc/csf/csf.blocklists and when you restart CSF Firewall, you can see the IPSET chains being created as well the reported number of IP addresses blocked and listed by those enabled chains

    i.e. DBESTRONGIPS and DBEBRUTEFORCE chains
    Code (Text):
    csf -ra
    csf: IPSET creating set bl_BDEBRUTEFORCE
    DROP all opt -- in * out * 0.0.0.0/0 -> 0.0.0.0/0 match-set bl_BDEBRUTEFORCE src
    csf: IPSET loading set bl_BDEBRUTEFORCE with 395 entries
    BDEBRUTEFORCE all opt -- in !lo out * 0.0.0.0/0 -> 0.0.0.0/0
    csf: IPSET creating set bl_BDESTRONGIPS
    DROP all opt -- in * out * 0.0.0.0/0 -> 0.0.0.0/0 match-set bl_BDESTRONGIPS src
    csf: IPSET loading set bl_BDESTRONGIPS with 119 entries
    BDESTRONGIPS all opt -- in !lo out * 0.0.0.0/0 -> 0.0.0.0/0
    tcp opt -- in !lo out * 0.0.0.0/0 -> 0.0.0.0/0 tcp dpt:21 state NEW recent: SET name: 21 side: source mask: 255.255.255.255
    

    You can use IPSET list command to inspect and list the IPs being blocked by the particular chain. Note CSF Firewall prefixes the chain name with bl_ in front so to view BDESTRONGIPS list via IPSET list command
    Code (Text):
    ipset list bl_BDESTRONGIPS
    Name: bl_BDESTRONGIPS
    Type: hash:net
    Revision: 3
    Header: family inet hashsize 1024 maxelem 65536
    Size in memory: 20432
    References: 1
    Members:
    221.194.47.236
    116.31.116.52
    193.201.224.11
    116.31.116.46
    116.31.116.53
    59.45.175.67
    59.45.175.86
    116.31.116.23
    59.45.175.56
    116.31.116.38
    116.31.116.37
    58.218.198.162
    59.45.175.62
    91.200.12.6
    116.31.116.14
    146.185.223.173
    46.161.9.48
    221.194.47.242
    123.183.209.134
    221.194.44.212
    218.241.238.141
    121.18.238.119
    116.31.116.9
    91.200.12.10
    46.161.9.14
    59.45.175.64
    116.31.116.26
    163.121.188.3
    221.194.44.240
    116.31.116.18
    91.200.12.119
    91.200.12.133
    59.45.175.88
    116.31.116.43
    191.239.161.236
    116.31.116.34
    91.200.12.136
    91.200.12.142
    218.65.30.38
    91.200.12.125
    113.195.145.21
    116.31.116.45
    58.57.65.113
    91.200.12.11
    218.65.30.156
    46.161.9.18
    85.185.244.101
    31.168.198.79
    91.200.12.137
    59.45.175.33
    146.185.223.82
    116.31.116.48
    91.200.12.81
    91.200.12.140
    91.200.12.168
    194.247.190.248
    221.214.208.226
    61.177.172.44
    91.197.232.109
    121.18.238.125
    193.201.224.224
    146.185.223.197
    218.65.30.46
    116.31.116.27
    116.31.116.28
    91.200.12.29
    116.31.116.20
    116.31.116.25
    187.94.99.194
    91.200.12.90
    193.201.224.227
    116.31.116.44
    116.31.116.4
    91.200.12.74
    93.170.122.30
    116.31.116.39
    221.194.47.252
    91.197.232.107
    91.200.12.169
    91.200.13.3
    116.31.116.33
    116.31.116.24
    84.17.27.245
    46.119.116.11
    91.200.12.80
    195.22.126.189
    61.177.172.32
    91.200.12.139
    116.31.116.10
    58.218.198.146
    218.65.30.210
    121.18.238.123
    91.200.12.1
    218.65.30.251
    116.31.116.47
    179.127.175.210
    116.31.116.50
    116.31.116.17
    91.200.12.165
    116.31.116.49
    116.31.116.15
    116.31.116.6
    123.183.209.136
    121.18.238.106
    91.200.12.146
    59.45.175.66
    46.161.9.3
    91.197.232.104
    91.200.12.150
    89.248.171.31
    221.194.47.233
    123.183.209.140
    91.200.12.164
    116.31.116.41
    58.57.65.114
    89.248.171.223
    116.31.116.30
    116.31.116.5
    217.70.28.155
    

    From the IPSET listing checking 1st IP listed via CSF grep command to see if the IP is blocked. Below output confirms such via IPSET line Match
    Code (Text):
    csf -g 221.194.47.236
    
    Chain            num   pkts bytes target     prot opt in     out     source               destination    
    No matches found for 221.194.47.236 in iptables
    
    IPSET: Set:bl_BDESTRONGIPS Match:221.194.47.236 Setting:BDESTRONGIPS file:/etc/csf/csf.blocklists
    
    ip6tables:
    
    Chain            num   pkts bytes target     prot opt in     out     source               destination    
    No matches found for 221.194.47.236 in ip6tables
    


    Notes


    • Be carefull enabling /etc/csf/csf.blocklists lists as these can have false positive IPs listed or dynamic ISPs' IP addresses listed so some of your visitors could automatically be blocked from your server. Especially, if you visitor traffic is Asian region based as alot of ISPs use dynamic IPs and the Asian region has alot of spammers, attackers etc from that region so chances dynamic ISP IPs are listed in enabled blocklists.
    • If you try to run tools/csf-advancetweaks.sh on OpenVZ system or non-OpenVZ system which has a Linux Kernel that doesn't support IPSET, you will get this message
      Code (Text):
      ./csf-advancetweaks.sh                    
      
      Detected either openvz system and/or linux kernel that doesn't support ipset
      Aborted csf tweaks for DENY_IP_LIMIT & DENY_TEMP_IP_LIMIT & blocklist.de extension
    • This can complement, the fail2ban work outlined here Security - fail2ban for Centmin Mod + CSF Firewall / Cloudflare API
     
    Last edited: May 22, 2017
    • Like Like x 2
  3. eva2000

    eva2000 Administrator Staff Member

    36,424
    7,994
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,308
    Local Time:
    3:03 AM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    A quick revisit of what enabling some of the advanced CSF Firewall blocklists entails as per above instructions.

    List all blocklists that have been created with IPSET and save list to csf-blocklist.txt
    Code (Text):
    csf -r 2>&1 | grep 'IPSET loading set' > csf-blocklist.txt
    

    file contents
    Code (Text):
    cat csf-blocklist.txt
    csf: IPSET loading set bl_BDESSH with 14293 entries
    csf: IPSET loading set bl_6_BDESSH with 0 entries
    csf: IPSET loading set bl_BDESTRONGIPS with 112 entries
    csf: IPSET loading set bl_6_BDESTRONGIPS with 0 entries
    csf: IPSET loading set bl_ALTTOR with 253 entries
    csf: IPSET loading set bl_6_ALTTOR with 0 entries
    csf: IPSET loading set bl_BDESIP with 102 entries
    csf: IPSET loading set bl_6_BDESIP with 0 entries
    csf: IPSET loading set bl_STOPFORUMSPAM with 6639 entries
    csf: IPSET loading set bl_6_STOPFORUMSPAM with 0 entries
    csf: IPSET loading set bl_BDEBRUTEFORCE with 558 entries
    csf: IPSET loading set bl_6_BDEBRUTEFORCE with 0 entries
    csf: IPSET loading set bl_HONEYPOT with 48 entries
    csf: IPSET loading set bl_6_HONEYPOT with 0 entries
    csf: IPSET loading set bl_BDEAPACHE with 8687 entries
    csf: IPSET loading set bl_6_BDEAPACHE with 0 entries
    csf: IPSET loading set bl_BDEMAIL with 6240 entries
    csf: IPSET loading set bl_6_BDEMAIL with 0 entries
    csf: IPSET loading set bl_BDEBOTS with 136 entries
    csf: IPSET loading set bl_6_BDEBOTS with 0 entries
    csf: IPSET loading set bl_BDEFTP with 847 entries
    csf: IPSET loading set bl_6_BDEFTP with 0 entries
    csf: IPSET loading set bl_BDEIMAP with 0 entries
    csf: IPSET loading set bl_6_BDEIMAP with 0 entries
    csf: IPSET loading set bl_TOR with 341 entries
    csf: IPSET loading set bl_6_TOR with 0 entries
    csf: IPSET loading set bl_CIARMY with 14663 entries
    csf: IPSET loading set bl_6_CIARMY with 0 entries
    

    Now lets look at some IPSET list info stats which include IPSET name, hash type, hashsize, and memory limits and usage and number of blocked IPs per IPSET list
    Code (Text):
    while read b; do n=$(echo $b | awk '{print $5}'); bn=$(echo $n | sed -e 's|bl_||g' -e 's|bl_6_||g'); c=$(echo $b | awk '{print $7}'); echo "------------"; i=$(ipset list $n | head -n5 | xargs); echo "$i number of ips: $c"; u=$(grep "^$bn" /etc/csf/csf.blocklists); echo "$u";done < csf-blocklist.txt
    

    Code (Text):
    while read b; do n=$(echo $b | awk '{print $5}'); bn=$(echo $n | sed -e 's|bl_||g' -e 's|bl_6_||g'); c=$(echo $b | awk '{print $7}'); echo "------------"; i=$(ipset list $n | head -n5 | xargs); echo "$i number of ips: $c"; u=$(grep "^$bn" /etc/csf/csf.blocklists); echo "$u";done < csf-blocklist.txt
    ------------
    Name: bl_BDESSH Type: hash:net Revision: 3 Header: family inet hashsize 8192 maxelem 65536 Size in memory: 355984 number of ips: 14293
    BDESSH|86400|0|https://lists.blocklist.de/lists/ssh.txt
    ------------
    Name: bl_6_BDESSH Type: hash:net Revision: 3 Header: family inet6 hashsize 1024 maxelem 65536 Size in memory: 17552 number of ips: 0
    
    ------------
    Name: bl_BDESTRONGIPS Type: hash:net Revision: 3 Header: family inet hashsize 1024 maxelem 65536 Size in memory: 19856 number of ips: 112
    BDESTRONGIPS|86400|0|https://lists.blocklist.de/lists/strongips.txt
    ------------
    Name: bl_6_BDESTRONGIPS Type: hash:net Revision: 3 Header: family inet6 hashsize 1024 maxelem 65536 Size in memory: 17552 number of ips: 0
    
    ------------
    Name: bl_ALTTOR Type: hash:net Revision: 3 Header: family inet hashsize 1024 maxelem 65536 Size in memory: 24080 number of ips: 253
    ALTTOR|86400|0|http://torstatus.blutmagie.de/ip_list_exit.php/Tor_ip_list_EXIT.csv
    ------------
    Name: bl_6_ALTTOR Type: hash:net Revision: 3 Header: family inet6 hashsize 1024 maxelem 65536 Size in memory: 17552 number of ips: 0
    
    ------------
    Name: bl_BDESIP Type: hash:net Revision: 3 Header: family inet hashsize 1024 maxelem 65536 Size in memory: 19856 number of ips: 102
    BDESIP|86400|0|https://lists.blocklist.de/lists/sip.txt
    ------------
    Name: bl_6_BDESIP Type: hash:net Revision: 3 Header: family inet6 hashsize 1024 maxelem 65536 Size in memory: 17552 number of ips: 0
    
    ------------
    Name: bl_STOPFORUMSPAM Type: hash:net Revision: 3 Header: family inet hashsize 2048 maxelem 65536 Size in memory: 111504 number of ips: 6639
    STOPFORUMSPAM|86400|0|http://www.stopforumspam.com/downloads/listed_ip_1.zip
    ------------
    Name: bl_6_STOPFORUMSPAM Type: hash:net Revision: 3 Header: family inet6 hashsize 1024 maxelem 65536 Size in memory: 17552 number of ips: 0
    
    ------------
    Name: bl_BDEBRUTEFORCE Type: hash:net Revision: 3 Header: family inet hashsize 1024 maxelem 65536 Size in memory: 27728 number of ips: 558
    BDEBRUTEFORCE|86400|0|https://lists.blocklist.de/lists/bruteforcelogin.txt
    ------------
    Name: bl_6_BDEBRUTEFORCE Type: hash:net Revision: 3 Header: family inet6 hashsize 1024 maxelem 65536 Size in memory: 17552 number of ips: 0
    
    ------------
    Name: bl_HONEYPOT Type: hash:net Revision: 3 Header: family inet hashsize 1024 maxelem 65536 Size in memory: 18288 number of ips: 48
    HONEYPOT|86400|0|http://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1
    ------------
    Name: bl_6_HONEYPOT Type: hash:net Revision: 3 Header: family inet6 hashsize 1024 maxelem 65536 Size in memory: 17552 number of ips: 0
    
    ------------
    Name: bl_BDEAPACHE Type: hash:net Revision: 3 Header: family inet hashsize 2048 maxelem 65536 Size in memory: 127408 number of ips: 8687
    BDEAPACHE|86400|0|https://lists.blocklist.de/lists/apache.txt
    ------------
    Name: bl_6_BDEAPACHE Type: hash:net Revision: 3 Header: family inet6 hashsize 1024 maxelem 65536 Size in memory: 17552 number of ips: 0
    
    ------------
    Name: bl_BDEMAIL Type: hash:net Revision: 3 Header: family inet hashsize 2048 maxelem 65536 Size in memory: 108624 number of ips: 6240
    BDEMAIL|86400|0|https://lists.blocklist.de/lists/mail.txt
    ------------
    Name: bl_6_BDEMAIL Type: hash:net Revision: 3 Header: family inet6 hashsize 1024 maxelem 65536 Size in memory: 17552 number of ips: 0
    
    ------------
    Name: bl_BDEBOTS Type: hash:net Revision: 3 Header: family inet hashsize 1024 maxelem 65536 Size in memory: 21840 number of ips: 136
    BDEBOTS|86400|0|https://lists.blocklist.de/lists/bots.txt
    ------------
    Name: bl_6_BDEBOTS Type: hash:net Revision: 3 Header: family inet6 hashsize 1024 maxelem 65536 Size in memory: 17552 number of ips: 0
    
    ------------
    Name: bl_BDEFTP Type: hash:net Revision: 3 Header: family inet hashsize 1024 maxelem 65536 Size in memory: 35184 number of ips: 847
    BDEFTP|86400|0|https://lists.blocklist.de/lists/ftp.txt
    ------------
    Name: bl_6_BDEFTP Type: hash:net Revision: 3 Header: family inet6 hashsize 1024 maxelem 65536 Size in memory: 17552 number of ips: 0
    
    ------------
    Name: bl_BDEIMAP Type: hash:net Revision: 3 Header: family inet hashsize 1024 maxelem 65536 Size in memory: 16784 number of ips: 0
    BDEIMAP|86400|0|https://lists.blocklist.de/lists/imap.txt
    ------------
    Name: bl_6_BDEIMAP Type: hash:net Revision: 3 Header: family inet6 hashsize 1024 maxelem 65536 Size in memory: 17552 number of ips: 0
    
    ------------
    Name: bl_TOR Type: hash:net Revision: 3 Header: family inet hashsize 1024 maxelem 65536 Size in memory: 26384 number of ips: 341
    TOR|86400|0|https://check.torproject.org/cgi-bin/TorBulkExitList.py?ip=1.2.3.4
    ------------
    Name: bl_6_TOR Type: hash:net Revision: 3 Header: family inet6 hashsize 1024 maxelem 65536 Size in memory: 17552 number of ips: 0
    
    ------------
    Name: bl_CIARMY Type: hash:net Revision: 3 Header: family inet hashsize 4096 maxelem 65536 Size in memory: 232432 number of ips: 14663
    CIARMY|86400|0|http://www.ciarmy.com/list/ci-badguys.txt
    ------------
    Name: bl_6_CIARMY Type: hash:net Revision: 3 Header: family inet6 hashsize 1024 maxelem 65536 Size in memory: 17552 number of ips: 0
    
     
    • Like Like x 1
..