Welcome to Centmin Mod Community
Register Now

Nginx A straight forward tutorial to install test-cookie?

Discussion in 'Nginx, PHP-FPM & MariaDB MySQL' started by Oxide, Jun 25, 2015.

  1. rdan

    rdan Well-Known Member

    4,713
    1,137
    113
    May 25, 2014
    Ratings:
    +1,691
    Local Time:
    8:46 AM
    Mainline
    10.2
    I'm just inspecting every config, and eliminating useless code :D.
     
  2. rdan

    rdan Well-Known Member

    4,713
    1,137
    113
    May 25, 2014
    Ratings:
    +1,691
    Local Time:
    8:46 AM
    Mainline
    10.2
    Tried it live now, but Google Bot can't crawled :(.
     
  3. rdan

    rdan Well-Known Member

    4,713
    1,137
    113
    May 25, 2014
    Ratings:
    +1,691
    Local Time:
    8:46 AM
    Mainline
    10.2
    What are your list of IP white listed?
     
  4. Oxide

    Oxide Active Member

    516
    29
    28
    Mar 19, 2015
    Ratings:
    +54
    Local Time:
    10:46 AM
    Code:
    testcookie_whitelist {
        # PayPal
        173.0.88.66;
        173.0.88.98;
        173.0.84.66;
        173.0.84.98;
        66.211.168.91;
        173.0.92.23;
        173.0.93.23;
        64.4.249.23;
        64.4.248.23;
        173.0.81.1;
        173.0.81.33;
        66.211.170.66;
        173.0.84.8;
        173.0.84.40;
        173.0.88.8;
        173.0.88.40;
        173.0.92.8;
        173.0.93.8;
        64.4.249.8;
        64.4.248.8;
        66.211.168.93;
        173.0.84.161;
        173.0.84.198;
        173.0.88.161;
        173.0.88.198;
    
        # Google Bots
        64.233.160.0/19;
        66.102.0.0/20;
        66.249.64.0/19;
        72.14.192.0/18;
        74.125.0.0/16;
        209.85.128.0;
        216.239.32.0/19;
        8.8.8.8/32;
    }
     
    • Informative Informative x 1
  5. rdan

    rdan Well-Known Member

    4,713
    1,137
    113
    May 25, 2014
    Ratings:
    +1,691
    Local Time:
    8:46 AM
    Mainline
    10.2
    Where did you find those google bot ip?
    Is it really valid and updated? :)
    Thanks!
     
  6. rdan

    rdan Well-Known Member

    4,713
    1,137
    113
    May 25, 2014
    Ratings:
    +1,691
    Local Time:
    8:46 AM
    Mainline
    10.2
    Google, Facebook, and Bing bot is the only thing important to me.
     
  7. rdan

    rdan Well-Known Member

    4,713
    1,137
    113
    May 25, 2014
    Ratings:
    +1,691
    Local Time:
    8:46 AM
    Mainline
    10.2
    When I try: Google Webmaster Tools > Crawl > Fetch as Google

    Works now after I add the Google Bot list of IP you have :D
    Thanks!
     
    • Like Like x 1
  8. rdan

    rdan Well-Known Member

    4,713
    1,137
    113
    May 25, 2014
    Ratings:
    +1,691
    Local Time:
    8:46 AM
    Mainline
    10.2
    To white list Facebook Bots:
    The Facebook Crawler
    1. Whitelist the IP addresses used by the crawler, which is more secure:
    Run this command to get a current list of IP addresses the crawler uses.
     
  9. rdan

    rdan Well-Known Member

    4,713
    1,137
    113
    May 25, 2014
    Ratings:
    +1,691
    Local Time:
    8:46 AM
    Mainline
    10.2
    To compare the changes.
    Here's the last 12 hour snapshot via LongView:
    upload_2015-7-10_9-9-34.png

    Will update this tomorrow.
    I even disable all Cloudflare Firewall now.
     
  10. Oxide

    Oxide Active Member

    516
    29
    28
    Mar 19, 2015
    Ratings:
    +54
    Local Time:
    10:46 AM
    Cheers mate, i guess we can write some kind of bash script that constantly check for new facebook ips, google and other that are necessary.. i would think they would provide plain text version that would be easy to work with somewhere..

    did you also change the encryption key or whatever is neccesary, including firewall name? so it fits your site, i would replace firewallname with check or something so they don't get confused if using 301 redirects.
     
  11. rdan

    rdan Well-Known Member

    4,713
    1,137
    113
    May 25, 2014
    Ratings:
    +1,691
    Local Time:
    8:46 AM
    Mainline
    10.2
  12. eva2000

    eva2000 Administrator Staff Member

    42,048
    9,488
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +14,599
    Local Time:
    10:46 AM
    Nginx 1.17.x
    MariaDB 5.5/10.x
    • Like Like x 1
  13. rdan

    rdan Well-Known Member

    4,713
    1,137
    113
    May 25, 2014
    Ratings:
    +1,691
    Local Time:
    8:46 AM
    Mainline
    10.2
    Too bad, they attacking again now.
    Looks like even bypass this mitigation :/
     
  14. Oxide

    Oxide Active Member

    516
    29
    28
    Mar 19, 2015
    Ratings:
    +54
    Local Time:
    10:46 AM
    Do you know it's Layer-7, how?


    if ($http_user_agent ~* "PHP|curl|Wget|HTTrack|Nmap|Verifying|PingBack|Pingdom|Joomla|Wordpress") { return 444; }
    if ($http_user_agent = "") { return 444; }
    if ($http_user_agent = " ") { return 444; }
    if ($http_user_agent = "-") { return 444; }

    add this also to prevent many attacks, bad bots..
     
  15. rdan

    rdan Well-Known Member

    4,713
    1,137
    113
    May 25, 2014
    Ratings:
    +1,691
    Local Time:
    8:46 AM
    Mainline
    10.2
    Because PHP-FPM is almost consuming all the CPU power.
    I have 12 CPU core.
    If it's layer 3/4 my Network will die.
     
  16. Oxide

    Oxide Active Member

    516
    29
    28
    Mar 19, 2015
    Ratings:
    +54
    Local Time:
    10:46 AM
    do you have nginx access logs and error logs, mind uploading them somewhere?

    to take down 12 cores, requires a lot of power.. a lot.. especially when using test cookie lol

    install nload and check incoming traffic also, in case.
     
  17. rdan

    rdan Well-Known Member

    4,713
    1,137
    113
    May 25, 2014
    Ratings:
    +1,691
    Local Time:
    8:46 AM
    Mainline
    10.2
    They already take down several forum here on my country.
    Mostly large VB4 forum that has 20M+ post.
    Most them are already dead for more than 1 week now.

    Mine is just small having 600K post.
     
  18. Oxide

    Oxide Active Member

    516
    29
    28
    Mar 19, 2015
    Ratings:
    +54
    Local Time:
    10:46 AM
    if you show me your access logs, you should have attack logged there..

    i dont belive its layer-7 though, might be TCP

    even though i'd disable access logs, as they also use lots of cpu when attack.. but nice to identify which attack
     
  19. rdan

    rdan Well-Known Member

    4,713
    1,137
    113
    May 25, 2014
    Ratings:
    +1,691
    Local Time:
    8:46 AM
    Mainline
    10.2
    By the way, I tried to remove this cookie mitigation, and nothing change.
    So this module didn't help me :/.
     
  20. rdan

    rdan Well-Known Member

    4,713
    1,137
    113
    May 25, 2014
    Ratings:
    +1,691
    Local Time:
    8:46 AM
    Mainline
    10.2
    I will forward later access logs, a little bit busy right now :|.