Welcome to Centmin Mod Community
Become a Member

Beta Branch csfcf.sh - automate Cloudflare Nginx & CSF Firewall setups

Discussion in 'Beta release code' started by eva2000, Feb 15, 2016.

Thread Status:
Not open for further replies.
  1. eva2000

    eva2000 Administrator Staff Member

    29,034
    6,589
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,783
    Local Time:
    10:55 PM
    Nginx 1.13.x
    MariaDB 5.5
    For 123.09beta01 looking to add a script to automate the setup for Cloudflare IPv4 and IPv6 and Nginx and CSF Firewall configurations outlined at:
    Command to add to cronjob would be with auto option:
    Code (Text):
    /usr/local/src/centminmod/tools/csfcf.sh auto
    

    example every 36 hrs. You can set it to whatever you want i.e. daily, weekly or monthly. Note part of the routine will restart nginx at cron run time.
    Code (Text):
    23 */36 * * * /usr/local/src/centminmod/tools/csfcf.sh auto >/dev/null 2>&1
    

    use crontab -e command to invoke nano text editor see guide at HowTo: Add Jobs To cron Under Linux or UNIX?

    When the csfcf.sh auto command runs it will end up whitelisting CSF Firewall's know listed IPv4 and IPv6 IP addresses + populating include file /usr/local/nginx/conf/cloudflare.conf which you should include in nginx.conf and/or your individual nginx vhost's server contexts.

    so your nginx.conf http{} context would look like this with added include file /usr/local/nginx/conf/cloudflare.conf on 123.09beta01
    Code:
    http {
    map_hash_bucket_size 128;
    map_hash_max_size 2048;
    server_names_hash_bucket_size 128;
    server_names_hash_max_size 2048;
    
    limit_req_zone $binary_remote_addr zone=xwplogin:16m rate=40r/m;
    #limit_conn_zone $binary_remote_addr zone=xwpconlimit:16m;
    
    more_set_headers "Server: nginx centminmod";
    more_set_headers "X-Powered-By: centminmod";
    
    include /usr/local/nginx/conf/cloudflare.conf;
    include /usr/local/nginx/conf/maintenance.conf;
    include /usr/local/nginx/conf/vts_http.conf;
    include /usr/local/nginx/conf/geoip.conf;
    #include /usr/local/nginx/conf/pagespeedadmin.conf;
    include /usr/local/nginx/conf/fastcgi_param_https_map.conf;
    Code:
    ./csfcf.sh nginx                         
    created /usr/local/nginx/conf/cloudflare.conf include file
    contents of /usr/local/nginx/conf/cloudflare.conf
    Code:
    set_real_ip_from  103.21.244.0/22;
    set_real_ip_from  103.22.200.0/22;
    set_real_ip_from  103.31.4.0/22;
    set_real_ip_from  104.16.0.0/12;
    set_real_ip_from  108.162.192.0/18;
    set_real_ip_from  131.0.72.0/22;
    set_real_ip_from  141.101.64.0/18;
    set_real_ip_from  162.158.0.0/15;
    set_real_ip_from  172.64.0.0/13;
    set_real_ip_from  173.245.48.0/20;
    set_real_ip_from  188.114.96.0/20;
    set_real_ip_from  190.93.240.0/20;
    set_real_ip_from  197.234.240.0/22;
    set_real_ip_from  198.41.128.0/17;
    set_real_ip_from  199.27.128.0/21;
    #set_real_ip_from  2400:cb00::/32;
    #set_real_ip_from  2405:8100::/32;
    #set_real_ip_from  2405:b500::/32;
    #set_real_ip_from  2606:4700::/32;
    #set_real_ip_from  2803:f800::/32;
    real_ip_header CF-Connecting-IP;
    adding Cloudflare IPs to CSF Firewall whitelisting
    Code:
    ./csfcf.sh csf
    --------------------------------------------
    Add Cloudflare IP list to CSF
    from: https://www.cloudflare.com/ips-v4
    from: https://www.cloudflare.com/ips-v6
    --------------------------------------------
    
    --------------------------------------------
      Add to /etc/csf/csf.allow
    --------------------------------------------
    Adding 103.21.244.0/22 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [103.21.244.0/22] to set [chain_ALLOW]
    Adding 103.22.200.0/22 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [103.22.200.0/22] to set [chain_ALLOW]
    Adding 103.31.4.0/22 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [103.31.4.0/22] to set [chain_ALLOW]
    Adding 104.16.0.0/12 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [104.16.0.0/12] to set [chain_ALLOW]
    Adding 108.162.192.0/18 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [108.162.192.0/18] to set [chain_ALLOW]
    Adding 131.0.72.0/22 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [131.0.72.0/22] to set [chain_ALLOW]
    Adding 141.101.64.0/18 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [141.101.64.0/18] to set [chain_ALLOW]
    Adding 162.158.0.0/15 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [162.158.0.0/15] to set [chain_ALLOW]
    Adding 172.64.0.0/13 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [172.64.0.0/13] to set [chain_ALLOW]
    Adding 173.245.48.0/20 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [173.245.48.0/20] to set [chain_ALLOW]
    Adding 188.114.96.0/20 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [188.114.96.0/20] to set [chain_ALLOW]
    Adding 190.93.240.0/20 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [190.93.240.0/20] to set [chain_ALLOW]
    Adding 197.234.240.0/22 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [197.234.240.0/22] to set [chain_ALLOW]
    Adding 198.41.128.0/17 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [198.41.128.0/17] to set [chain_ALLOW]
    Adding 199.27.128.0/21 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [199.27.128.0/21] to set [chain_ALLOW]
    Adding 2400:cb00::/32 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [2400:cb00::/32] to set [chain_6_ALLOW]
    Adding 2405:8100::/32 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [2405:8100::/32] to set [chain_6_ALLOW]
    Adding 2405:b500::/32 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [2405:b500::/32] to set [chain_6_ALLOW]
    Adding 2606:4700::/32 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [2606:4700::/32] to set [chain_6_ALLOW]
    Adding 2803:f800::/32 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [2803:f800::/32] to set [chain_6_ALLOW]
     
    Last edited: Mar 24, 2017
    • Winner Winner x 4
    • Like Like x 1
  2. eva2000

    eva2000 Administrator Staff Member

    29,034
    6,589
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,783
    Local Time:
    10:55 PM
    Nginx 1.13.x
    MariaDB 5.5
  3. eva2000

    eva2000 Administrator Staff Member

    29,034
    6,589
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,783
    Local Time:
    10:55 PM
    Nginx 1.13.x
    MariaDB 5.5
    cloudflare's new web site design messages up csfcf.sh scripts parsing of cloudflare's listed ips at

    as the lists are no longer plain text but html pages so html code is being inserted into csfcf.sh parsed ip lists !

    edit: oh seems it's due to curl and I used without ending forward slash ! so 301 redirect

    Code (Text):
    /usr/bin/curl -s ${CURL_TIMEOUTS} https://www.cloudflare.com/ips-v4
    <html>
    <head><title>301 Moved Permanently</title></head>
    <body bgcolor="white">
    <center><h1>301 Moved Permanently</h1></center>
    <hr><center>nginx/1.6.2</center>
    </body>
    </html>
     
    Last edited: Sep 29, 2016
  4. eva2000

    eva2000 Administrator Staff Member

    29,034
    6,589
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,783
    Local Time:
    10:55 PM
    Nginx 1.13.x
    MariaDB 5.5
    fixed auto run
    Code (Text):
    /usr/local/src/centminmod/tools/csfcf.sh auto                     
    --------------------------------------------
     Add Cloudflare IP list to CSF
     from: https://www.cloudflare.com/ips-v4/
     from: https://www.cloudflare.com/ips-v6/
    --------------------------------------------
    
    --------------------------------------------
      Add to /etc/csf/csf.allow
    --------------------------------------------
    Adding 103.21.244.0/22 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [103.21.244.0/22] to set [chain_ALLOW]
    Adding 103.22.200.0/22 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [103.22.200.0/22] to set [chain_ALLOW]
    Adding 103.31.4.0/22 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [103.31.4.0/22] to set [chain_ALLOW]
    Adding 104.16.0.0/12 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [104.16.0.0/12] to set [chain_ALLOW]
    Adding 108.162.192.0/18 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [108.162.192.0/18] to set [chain_ALLOW]
    Adding 131.0.72.0/22 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [131.0.72.0/22] to set [chain_ALLOW]
    Adding 141.101.64.0/18 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [141.101.64.0/18] to set [chain_ALLOW]
    Adding 162.158.0.0/15 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [162.158.0.0/15] to set [chain_ALLOW]
    Adding 172.64.0.0/13 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [172.64.0.0/13] to set [chain_ALLOW]
    Adding 173.245.48.0/20 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [173.245.48.0/20] to set [chain_ALLOW]
    Adding 188.114.96.0/20 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [188.114.96.0/20] to set [chain_ALLOW]
    Adding 190.93.240.0/20 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [190.93.240.0/20] to set [chain_ALLOW]
    Adding 197.234.240.0/22 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [197.234.240.0/22] to set [chain_ALLOW]
    Adding 198.41.128.0/17 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [198.41.128.0/17] to set [chain_ALLOW]
    Adding 199.27.128.0/21 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [199.27.128.0/21] to set [chain_ALLOW]
    Adding 2400:cb00::/32 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [2400:cb00::/32] to set [chain_6_ALLOW]
    Adding 2405:8100::/32 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [2405:8100::/32] to set [chain_6_ALLOW]
    Adding 2405:b500::/32 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [2405:b500::/32] to set [chain_6_ALLOW]
    Adding 2606:4700::/32 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [2606:4700::/32] to set [chain_6_ALLOW]
    Adding 2803:f800::/32 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [2803:f800::/32] to set [chain_6_ALLOW]
    Adding 2c0f:f248::/32 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [2c0f:f248::/32] to set [chain_6_ALLOW]
    Adding 2a06:98c0::/29 to csf.allow and iptables ACCEPT...
    csf: IPSET adding [2a06:98c0::/29] to set [chain_6_ALLOW]
    
    created /usr/local/nginx/conf/cloudflare.conf include file


    contents of include file /usr/local/nginx/conf/cloudflare.conf generated by tools/csfcf.sh auto run
    Code (Text):
    set_real_ip_from 103.21.244.0/22;
    set_real_ip_from 103.22.200.0/22;
    set_real_ip_from 103.31.4.0/22;
    set_real_ip_from 104.16.0.0/12;
    set_real_ip_from 108.162.192.0/18;
    set_real_ip_from 131.0.72.0/22;
    set_real_ip_from 141.101.64.0/18;
    set_real_ip_from 162.158.0.0/15;
    set_real_ip_from 172.64.0.0/13;
    set_real_ip_from 173.245.48.0/20;
    set_real_ip_from 188.114.96.0/20;
    set_real_ip_from 190.93.240.0/20;
    set_real_ip_from 197.234.240.0/22;
    set_real_ip_from 198.41.128.0/17;
    set_real_ip_from 199.27.128.0/21;
    #set_real_ip_from 2400:cb00::/32;
    #set_real_ip_from 2405:8100::/32;
    #set_real_ip_from 2405:b500::/32;
    #set_real_ip_from 2606:4700::/32;
    #set_real_ip_from 2803:f800::/32;
    #set_real_ip_from 2c0f:f248::/32;
    #set_real_ip_from 2a06:98c0::/29;
    real_ip_header CF-Connecting-IP;
     
  5. eva2000

    eva2000 Administrator Staff Member

    29,034
    6,589
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,783
    Local Time:
    10:55 PM
    Nginx 1.13.x
    MariaDB 5.5
    FYI, just updated 123.09beta01's tools/csfcf.sh auto routine to add an include file into cloudflare.conf called cloudflare_customips.conf so you can still run tools/csfcf.sh via cronjob and not overwrite your custom added ips in cloudflare_customips.conf

    Code (Text):
    cat /usr/local/nginx/conf/cloudflare.conf
    
    include /usr/local/nginx/conf/cloudflare_customips.conf;
    set_real_ip_from 103.21.244.0/22;
    set_real_ip_from 103.22.200.0/22;
    set_real_ip_from 103.31.4.0/22;
    set_real_ip_from 104.16.0.0/12;
    set_real_ip_from 108.162.192.0/18;
    set_real_ip_from 131.0.72.0/22;
    set_real_ip_from 141.101.64.0/18;
    set_real_ip_from 162.158.0.0/15;
    set_real_ip_from 172.64.0.0/13;
    set_real_ip_from 173.245.48.0/20;
    set_real_ip_from 188.114.96.0/20;
    set_real_ip_from 190.93.240.0/20;
    set_real_ip_from 197.234.240.0/22;
    set_real_ip_from 198.41.128.0/17;
    set_real_ip_from 199.27.128.0/21;
    set_real_ip_from 2400:cb00::/32;
    set_real_ip_from 2405:8100::/32;
    set_real_ip_from 2405:b500::/32;
    set_real_ip_from 2606:4700::/32;
    set_real_ip_from 2803:f800::/32;
    set_real_ip_from 2c0f:f248::/32;
    set_real_ip_from 2a06:98c0::/29;
    real_ip_header CF-Connecting-IP;


    so stick your custom ips etc into /usr/local/nginx/conf/cloudflare_customips.conf
    Code (Text):
    set_real_ip_from YOURCUSTOM_IPADDRESS;
    


    Used to address https://community.centminmod.com/posts/44475/ and needing to add custom ips.
     
    • Like Like x 1
Thread Status:
Not open for further replies.