Want to subscribe to topics you're interested in?
Become a Member

Was community.centminmod down?

Discussion in 'Forum Feedback & Suggestions' started by Revenge, Apr 15, 2016.

  1. Revenge

    Revenge Active Member

    469
    93
    28
    Feb 21, 2016
    Portugal
    Ratings:
    +354
    Local Time:
    4:25 AM
    1.9.x
    10.1.x
    For some time, i was not able to enter the community, but i could enter the main site.

    Problem with server? I don't think it was on my end.


    [​IMG]
     
  2. eva2000

    eva2000 Administrator Staff Member

    50,891
    11,793
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,248
    Local Time:
    2:25 PM
    Nginx 1.25.x
    MariaDB 10.x
  3. eva2000

    eva2000 Administrator Staff Member

    50,891
    11,793
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,248
    Local Time:
    2:25 PM
    Nginx 1.25.x
    MariaDB 10.x
    Ah checking my logs, it seems it was a wordpress pingback attack directed against my forum server :LOL: Or could of been a bit of both - Linode maintenance + pingback attack ?

    Seems the attack came from 4,731 wordpress sites on 4,730 ip addresses - probably from insecure wordpress sites with still have pingback enabled.
    Code (Text):
    awk -v filter='verifying' '$14 == 'filter' { print $1, $13 }' access.log | sort | uniq -c | sort -n | wc -l
    4731
    
    awk -v filter='verifying' '$14 == 'filter' { print $1 }' access.log | sort | uniq -c | sort -n | wc -l
    4730

    Code (Text):
    grep 'verifying pingback' access.log | tail -10
    104.155.215.31 - - [15/Apr/2016:04:45:02 +0000] "GET / HTTP/1.0" 200 131664 "-" "WordPress/4.1; http://x.hello-air.com; verifying pingback from 89.248.167.131"
    104.155.215.31 - - [15/Apr/2016:04:44:23 +0000] "GET / HTTP/1.0" 200 131664 "-" "WordPress/4.1; http://x.hello-air.com; verifying pingback from 89.248.167.131"
    104.155.215.31 - - [15/Apr/2016:04:44:25 +0000] "GET / HTTP/1.0" 200 131664 "-" "WordPress/4.1; http://x.hello-air.com; verifying pingback from 89.248.167.131"
    104.155.215.31 - - [15/Apr/2016:04:44:30 +0000] "GET / HTTP/1.0" 200 131664 "-" "WordPress/4.1; http://x.hello-air.com; verifying pingback from 89.248.167.131"
    104.155.215.31 - - [15/Apr/2016:04:44:45 +0000] "GET / HTTP/1.0" 200 131664 "-" "WordPress/4.1; http://x.hello-air.com; verifying pingback from 89.248.167.131"
    104.155.215.31 - - [15/Apr/2016:04:45:00 +0000] "GET / HTTP/1.0" 200 131664 "-" "WordPress/4.1; http://x.hello-air.com; verifying pingback from 89.248.167.131"
    104.155.215.31 - - [15/Apr/2016:04:45:02 +0000] "GET / HTTP/1.0" 200 131664 "-" "WordPress/4.1; http://x.hello-air.com; verifying pingback from 89.248.167.131"
    104.155.215.31 - - [15/Apr/2016:04:44:54 +0000] "GET / HTTP/1.0" 200 131664 "-" "WordPress/4.1; http://x.hello-air.com; verifying pingback from 89.248.167.131"
    104.155.215.31 - - [15/Apr/2016:04:44:56 +0000] "GET / HTTP/1.0" 200 131664 "-" "WordPress/4.1; http://x.hello-air.com; verifying pingback from 89.248.167.131"
    104.155.215.31 - - [15/Apr/2016:04:44:58 +0000] "GET / HTTP/1.0" 200 131664 "-" "WordPress/4.1; http://x.hello-air.com; verifying pingback from 89.248.167.131"


    Checking out profile of WordPress versions that were a part of the pingback attack. Some very very old WordPress versions out there !

    Code (Text):
    awk -v filter='verifying' '$14 == 'filter' { print $12 }' access.log | sort | uniq -c | sort -n
          1 "WordPress/107;
          1 "WordPress/109;
          1 "WordPress/115;
          1 "WordPress/121;
          1 "WordPress/124;
          1 "WordPress/133;
          1 "WordPress/1359;
          1 "WordPress/137;
          1 "WordPress/140;
          1 "WordPress/141;
          1 "WordPress/148;
          1 "WordPress/155;
          1 "WordPress/158;
          1 "WordPress/164;
          1 "WordPress/170;
          1 "WordPress/173;
          1 "WordPress/176;
          1 "WordPress/180;
          1 "WordPress/185;
          1 "WordPress/186;
          1 "WordPress/193;
          1 "WordPress/197;
          1 "WordPress/199;
          1 "WordPress/200;
          1 "WordPress/2075;
          1 "WordPress/208;
          1 "WordPress/214;
          1 "WordPress/216;
          1 "WordPress/225;
          1 "WordPress/226;
          1 "WordPress/228;
          1 "WordPress/238;
          1 "WordPress/239;
          1 "WordPress/2507;
          1 "WordPress/258;
          1 "WordPress/261;
          1 "WordPress/262;
          1 "WordPress/266;
          1 "WordPress/274;
          1 "WordPress/2769;
          1 "WordPress/280;
          1 "WordPress/284;
          1 "WordPress/285;
          1 "WordPress/286;
          1 "WordPress/288;
          1 "WordPress/290;
          1 "WordPress/291;
          1 "WordPress/310;
          1 "WordPress/311;
          1 "WordPress/313;
          1 "WordPress/316;
          1 "WordPress/320;
          1 "WordPress/323;
          1 "WordPress/328;
          1 "WordPress/330;
          1 "WordPress/332;
          1 "WordPress/334;
          1 "WordPress/337;
          1 "WordPress/338;
          1 "WordPress/341;
          1 "WordPress/345;
          1 "WordPress/348;
          1 "WordPress/349;
          1 "WordPress/351;
          1 "WordPress/353;
          1 "WordPress/354;
          1 "WordPress/359;
          1 "WordPress/361;
          1 "WordPress/366;
          1 "WordPress/379;
          1 "WordPress/389;
          1 "WordPress/391;
          1 "WordPress/394;
          1 "WordPress/397;
          1 "WordPress/401;
          1 "WordPress/418;
          1 "WordPress/419;
          1 "WordPress/420;
          1 "WordPress/421;
          1 "WordPress/422;
          1 "WordPress/427;
          1 "WordPress/428;
          1 "WordPress/429;
          1 "WordPress/430;
          1 "WordPress/433;
          1 "WordPress/436;
          1 "WordPress/438;
          1 "WordPress/441;
          1 "WordPress/443;
          1 "WordPress/445;
          1 "WordPress/447;
          1 "WordPress/455;
          1 "WordPress/456;
          1 "WordPress/461;
          1 "WordPress/4782;
          1 "WordPress/482;
          1 "WordPress/493;
          1 "WordPress/500;
          1 "WordPress/6499;
          1 "WordPress/7542;
          1 "WordPress/8178;
          1 "WordPress/9094;
          2 "WordPress/117;
          2 "WordPress/150;
          2 "WordPress/169;
          2 "WordPress/179;
          2 "WordPress/201;
          2 "WordPress/247;
          2 "WordPress/253;
          2 "WordPress/267;
          2 "WordPress/278;
          2 "WordPress/306;
          2 "WordPress/312;
          2 "WordPress/370;
          2 "WordPress/372;
          2 "WordPress/388;
          2 "WordPress/457;
          2 "WordPress/491;
          6 "WordPress/218;
          9 "WordPress/3.9.4;
         10 "WordPress/4;
         13 "WordPress/4.2-alpha-31302;
         15 "WordPress/4.0.4;
         17 "WordPress/3.5.2;
         28 "WordPress/277;
         28 "WordPress/3.7.4;
         31 "WordPress/3.7.11;
         34 "WordPress/163;
         45 "WordPress/157;
         49 "WordPress/3.6.1;
         50 "WordPress/4.1-alpha;
         54 "WordPress/378;
         58 "WordPress/3.7.3;
         69 "WordPress/4.0.9;
         70 "WordPress/270;
         70 "WordPress/403;
         75 "WordPress/4.0.7;
         87 "WordPress/4.1.6;
         97 "WordPress/4.0.6;
        102 "WordPress/400;
        118 "WordPress/4.1.3;
        121 "WordPress/3.7.8;
        126 "WordPress/4.0.3;
        128 "WordPress/363;
        137 "WordPress/3.9.5;
        149 "WordPress/143;
        167 "WordPress/3.8.10;
        190 "WordPress/4.3-alpha-32812;
        192 "WordPress/126;
        199 "WordPress/3.9.10;
        220 "WordPress/4.5-RC3-37177;
        265 "WordPress/3.9.8;
        270 "WordPress/4.1.7;
        284 "WordPress/4.1.4;
        295 "WordPress/4.2;
        302 "WordPress/4.1.9;
        327 "WordPress/3.8.2;
        359 "WordPress/3.7.5;
        375 "WordPress/4.1.2;
        378 "WordPress/3.8.11;
        387 "WordPress/4.0.8;
        462 "WordPress/3.9.9;
        508 "WordPress/4.0.5;
        539 "WordPress/3.8.4;
        549 "WordPress/4.2.6;
        631 "WordPress/3.9.6;
        702 "WordPress/4.1.5;
        790 "WordPress/3.9.3;
        796 "WordPress/4.2.1;
        946 "WordPress/3.8.5;
       1081 "WordPress/3.7.13;
       1224 "WordPress/4.1.8;
       1469 "WordPress/4.3.2;
       1697 "WordPress/4.2.3;
       1722 "WordPress/3.8.3;
       1991 "WordPress/3.9;
       2618 "WordPress/4.2.4;
       2788 "WordPress/3.9.2;
       2799 "WordPress/4.4;
       2816 "WordPress/4.2.5;
       2871 "WordPress/3.8.13;
       4336 "WordPress/4.0.1;
       4431 "WordPress/4.4.1;
       4545 "WordPress/4.3;
       6606 "WordPress/3.9.1;
       7016 "WordPress/3.9.11;
       7145 "WordPress/4.1.1;
       7452 "WordPress/4.5;
       7633 "WordPress/4.1;
       7910 "WordPress/4.0.10;
       7927 "WordPress/4.0;
      12095 "WordPress/4.2.2;
      15669 "WordPress/4.1.10;
      19476 "WordPress/4.3.1;
      26677 "WordPress/4.2.7;
      37821 "WordPress/4.3.3;
      49593 "WordPress/4.4.2;


    Top 20 unique ips that took part in the Wordpress pingback attack
    Code (Text):
    awk -v filter='verifying' '$14 == 'filter' { print $1 }' access.log | sort | uniq -c | sort -n | tail -20
        307 209.160.24.14
        308 137.117.145.49
        309 104.43.20.190
        312 52.24.81.156
        314 104.197.137.99
        322 23.21.136.106
        329 69.13.208.100
        330 130.211.139.202
        333 130.211.148.235
        333 161.202.15.135
        340 104.236.188.129
        358 52.1.28.168
        364 173.230.153.62
        366 52.69.112.242
        372 104.193.9.61
        410 54.75.246.142
        420 104.154.41.88
        426 104.131.134.150
        474 192.99.101.208
        483 123.30.153.88

    Top 10 attacker originating IPs themselves with 253,000+ instances from IP 89.248.167.131
    Code (Text):
    awk -v filter='verifying' '$14 == 'filter' { print $17 }' access.log | sort | uniq -c | sort -n | tail -10
         79 10.0.0.4"
         82 207.244.76.218"
         83 10.12.1.2"
         89 162.243.61.161"
        102 10.0.2.2"
        144 67.199.71.56"
        149 172.17.0.1"
        411 ::ffff:89.248.167.131"
       5995 127.0.0.1"
    253061 89.248.167.131"
    

    Total hits from Wordpress pingback attack = 261,602 entries
    Code (Text):
    grep -c 'verifying pingback' access.log   
    261602


    Can also use ngxtop to get similar stats too Nginx - ngxtop real time metrics for Nginx | Centmin Mod Community
    Code (Text):
    cat access.log | grep 'verifying pingback' | grep WordPress | ngxtop --no-follow
    running for 12 seconds, 261602 records processed: 21833.89 req/sec
    
    Summary:
    |   count |   avg_bytes_sent |   2xx |   3xx |    4xx |   5xx |
    |---------+------------------+-------+-------+--------+-------|
    |  261602 |         4993.304 | 16202 |     0 | 242708 |  2692 |
    
    Detailed:
    | request_path   |   count |   avg_bytes_sent |   2xx |   3xx |    4xx |   5xx |
    |----------------+---------+------------------+-------+-------+--------+-------|
    | /              |  261602 |         4993.304 | 16202 |     0 | 242708 |  2692 |
    

    Code (Text):
    cat access.log | grep 'verifying pingback' | grep WordPress | ngxtop --no-follow --group-by remote_addr
    running for 13 seconds, 261602 records processed: 19874.48 req/sec
    
    Summary:
    |   count |   avg_bytes_sent |   2xx |   3xx |    4xx |   5xx |
    |---------+------------------+-------+-------+--------+-------|
    |  261602 |         4993.304 | 16202 |     0 | 242708 |  2692 |
    
    Detailed:
    | remote_addr     |   count |   avg_bytes_sent |   2xx |   3xx |   4xx |   5xx |
    |-----------------+---------+------------------+-------+-------+-------+-------|
    | 123.30.153.88   |     483 |        78988.685 |   483 |     0 |     0 |     0 |
    | 192.99.101.208  |     474 |         3161.977 |    19 |     0 |   455 |     0 |
    | 104.131.134.150 |     426 |         1477.784 |     8 |     0 |   418 |     0 |
    | 104.154.41.88   |     420 |         2067.036 |    11 |     0 |   409 |     0 |
    | 54.75.246.142   |     410 |         2874.644 |    15 |     0 |   395 |     0 |
    | 104.193.9.61    |     372 |         5302.680 |    25 |     0 |   347 |     0 |
    | 52.69.112.242   |     366 |        79489.000 |   366 |     0 |     0 |     0 |
    | 173.230.153.62  |     364 |         1080.431 |     5 |     0 |   359 |     0 |
    | 52.1.28.168     |     358 |         2867.553 |    13 |     0 |   345 |     0 |
    | 104.236.188.129 |     340 |         3237.476 |    14 |     0 |   326 |     0 |
    

    Code (Text):
    cat access.log | grep 'verifying pingback' | grep WordPress | ngxtop --no-follow --group-by http_user_agent
    running for 13 seconds, 261602 records processed: 19863.16 req/sec
    
    Summary:
    |   count |   avg_bytes_sent |   2xx |   3xx |    4xx |   5xx |
    |---------+------------------+-------+-------+--------+-------|
    |  261602 |         4993.304 | 16202 |     0 | 242708 |  2692 |
    
    Detailed:
    | http_user_agent                                                                         |   count |   avg_bytes_sent |   2xx |   3xx |   4xx |   5xx |
    |-----------------------------------------------------------------------------------------+---------+------------------+-------+-------+-------+-------|
    | WordPress/4.1; http://online.yea.edu.vn; verifying pingback from 89.248.167.131         |     483 |        78988.685 |   483 |     0 |     0 |     0 |
    | WordPress/4.4.2; http://blog.ingen-lab.com; verifying pingback from 89.248.167.131      |     474 |         3161.977 |    19 |     0 |   455 |     0 |
    | WordPress/4.2.2; http://www.awdlaw.com; verifying pingback from 89.248.167.131          |     426 |         1477.784 |     8 |     0 |   418 |     0 |
    | WordPress/4.3.3; http://104.154.41.88; verifying pingback from 89.248.167.131           |     420 |         2067.036 |    11 |     0 |   409 |     0 |
    | WordPress/4.0; http://www.bluecrestwellness.com; verifying pingback from 89.248.167.131 |     410 |         2874.644 |    15 |     0 |   395 |     0 |
    | WordPress/4.0.1; http://mobile.nancynet2.com; verifying pingback from 89.248.167.131    |     372 |         5302.680 |    25 |     0 |   347 |     0 |
    | WordPress/4.3.3; http://jiritu-sinkei.com; verifying pingback from 89.248.167.131       |     366 |        79489.000 |   366 |     0 |     0 |     0 |
    | WordPress/4.1; http://www.agnesshiu.info; verifying pingback from 89.248.167.131        |     364 |         1080.431 |     5 |     0 |   359 |     0 |
    | WordPress/4.2.7; http://52.1.28.168; verifying pingback from 89.248.167.131             |     358 |         2867.553 |    13 |     0 |   345 |     0 |
    | WordPress/4.4.2; http://www.americannails.net; verifying pingback from 89.248.167.131   |     340 |         3237.476 |    14 |     0 |   326 |     0 |
    

    Added WordPress to my bad bot blocking and rate limiting function at Blocking bad or aggressive bots | Centmin Mod Community and deployed on my server :)
    Code (Text):
    curl -I -A "WordPress" https://community.centminmod.com
    curl: (52) Empty reply from server

    added to centminmod.com server cluster too
    Code (Text):
    curl -I -A "WordPress" http://centminmod.com
    curl: (52) Empty reply from server
     
    Last edited: Apr 16, 2016
  4. eva2000

    eva2000 Administrator Staff Member

    50,891
    11,793
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,248
    Local Time:
    2:25 PM
    Nginx 1.25.x
    MariaDB 10.x
    Looks like wordpress pingback attack round 2 :LOL: This time with bad bot blocking in place so access.log logs them all as 444 status entries

    Almost 4 million entries logged
    Code (Text):
    cat access.log | grep 'verifying pingback' | grep WordPress | ngxtop --no-follow
    running for 188 seconds, 3983383 records processed: 21194.36 req/sec
    
    Summary:
    |   count |   avg_bytes_sent |   2xx |   3xx |     4xx |   5xx |
    |---------+------------------+-------+-------+---------+-------|
    | 3983383 |            4.625 |   143 |     0 | 3983240 |     0 |
    
    Detailed:
    | request_path   |   count |   avg_bytes_sent |   2xx |   3xx |     4xx |   5xx |
    |----------------+---------+------------------+-------+-------+---------+-------|
    | /              | 3983383 |            4.625 |   143 |     0 | 3983240 |     0 |
    

    Top IP of the wordpress sites doing the pingback
    Code (Text):
    cat access.log | grep 'verifying pingback' | grep WordPress | ngxtop --no-follow --group-by remote_addr
    running for 193 seconds, 3984386 records processed: 20625.43 req/sec
    
    Summary:
    |   count |   avg_bytes_sent |   2xx |   3xx |     4xx |   5xx |
    |---------+------------------+-------+-------+---------+-------|
    | 3984386 |            4.624 |   143 |     0 | 3984243 |     0 |
    
    Detailed:
    | remote_addr     |   count |   avg_bytes_sent |   2xx |   3xx |   4xx |   5xx |
    |-----------------+---------+------------------+-------+-------+-------+-------|
    | 172.246.236.90  |    4821 |            0.000 |     0 |     0 |  4821 |     0 |
    | 193.180.115.103 |    4612 |            0.000 |     0 |     0 |  4612 |     0 |
    | 174.129.36.212  |    4538 |            0.000 |     0 |     0 |  4538 |     0 |
    | 185.53.129.239  |    4502 |            0.000 |     0 |     0 |  4502 |     0 |
    | 62.210.100.209  |    4365 |            0.000 |     0 |     0 |  4365 |     0 |
    | 84.246.225.122  |    4079 |            0.000 |     0 |     0 |  4079 |     0 |
    | 213.246.38.41   |    4066 |            0.000 |     0 |     0 |  4066 |     0 |
    | 144.76.159.199  |    3759 |            0.000 |     0 |     0 |  3759 |     0 |
    | 45.55.62.73     |    3690 |            0.000 |     0 |     0 |  3690 |     0 |
    | 174.37.215.195  |    3687 |            0.000 |     0 |     0 |  3687 |     0 |
    

    User Agents and Wordpress versions - seems to be the same originating attack IP as before 89.248.167.131
    Code (Text):
    cat access.log | grep 'verifying pingback' | grep WordPress | ngxtop --no-follow --group-by http_user_agent
    running for 190 seconds, 3984718 records processed: 20950.53 req/sec
    
    Summary:
    |   count |   avg_bytes_sent |   2xx |   3xx |     4xx |   5xx |
    |---------+------------------+-------+-------+---------+-------|
    | 3984718 |            4.624 |   143 |     0 | 3984575 |     0 |
    
    Detailed:
    | http_user_agent                                                                     |   count |   avg_bytes_sent |   2xx |   3xx |   4xx |   5xx |
    |-------------------------------------------------------------------------------------+---------+------------------+-------+-------+-------+-------|
    | WordPress/4.4.2; http://www.monkon.co.uk; verifying pingback from 127.0.0.1         |    4821 |            0.000 |     0 |     0 |  4821 |     0 |
    | WordPress/4.4; http://174.129.36.212; verifying pingback from 89.248.167.131        |    4538 |            0.000 |     0 |     0 |  4538 |     0 |
    | WordPress/4.4.1; http://bethelmusic.com; verifying pingback from 89.248.167.131     |    4511 |            0.000 |     0 |     0 |  4511 |     0 |
    | WordPress/4.4.2; http://bedroomcoders.co.uk; verifying pingback from 89.248.167.131 |    4502 |            0.000 |     0 |     0 |  4502 |     0 |
    | WordPress/4.3.1; http://www.alfamitoclub.it; verifying pingback from 89.248.167.131 |    4432 |            0.000 |     0 |     0 |  4432 |     0 |
    | WordPress/4.4.2; http://www.alsim.com; verifying pingback from 89.248.167.131       |    4079 |            0.000 |     0 |     0 |  4079 |     0 |
    | WordPress/4.3.1; http://www.early-birds.fr; verifying pingback from 89.248.167.131  |    3906 |            0.000 |     0 |     0 |  3906 |     0 |
    | WordPress/4.2.5; http://www.serendio.com; verifying pingback from 89.248.167.131    |    3759 |            0.000 |     0 |     0 |  3759 |     0 |
    | WordPress/4.4.2; http://ideasandcars.com; verifying pingback from 89.248.167.131    |    3690 |            0.000 |     0 |     0 |  3690 |     0 |
    | WordPress/4.2.2; http://poadigital.com; verifying pingback from 89.248.167.131      |    3687 |            0.000 |     0 |     0 |  3687 |     0 |
    

    Code (Text):
    cat access.log| grep '\"WordPress\/' | ngxtop --no-follow
    running for 236 seconds, 4564458 records processed: 19359.82 req/sec
    
    Summary:
    |   count |   avg_bytes_sent |   2xx |   3xx |     4xx |   5xx |
    |---------+------------------+-------+-------+---------+-------|
    | 4564458 |            4.037 |   143 |     0 | 4564315 |     0 |
    
    Detailed:
    | request_path                                   |   count |   avg_bytes_sent |   2xx |   3xx |     4xx |   5xx |
    |------------------------------------------------+---------+------------------+-------+-------+---------+-------|
    | /                                              | 4564456 |            4.037 |   143 |     0 | 4564313 |     0 |
    

    Newrelic stats ~1.33Gbps in..

    newrelic-525am-00.png

    Nginx access.log entries per minute charted :)

    access-log-hits-chart-1.png
     
  5. Revenge

    Revenge Active Member

    469
    93
    28
    Feb 21, 2016
    Portugal
    Ratings:
    +354
    Local Time:
    4:25 AM
    1.9.x
    10.1.x
    I don't know if blocking those requests with just a 444 response, is the best solution. The requests will still hit Nginx, and if the attack is big enough, it will put Nginx to its knees. The graph you showed of New Relic, i can see the Cpu reached 100%, even with Nginx returning 444 code.

    I would suggest adding another layer of security, fail2ban. It would be possible to ban the IP's with iptables of all the requests of WordPress. That way, the requests would never reach nginx. Of course, if the attack is really BIG! it would put iptables on its knees. But its a far better solution.

    I never received an attack like this, but we could create this configuration.

    Creste a filter named pingback.conf with this content:

    Code:
    [Definition]
    
    failregex = ^<HOST> -.*WordPress.*/
    
    Then, add this piece of code to the fail2ban jail:

    Code:
    [pingback]
    enabled = true
    port = http,https
    filter = pingback
    logpath = /home/nginx/domains/example.com/log/access.log
    backend  = polling
    journalmatch =
    maxretry = 1
    findtime = 1
    bantime = 86400
    action = iptables-allports
    This way, the first requests of a Wordpress hacked server would hit Nginx, but it would respond with a 444 code. At the same time, fail2ban would ban the server IP using iptables.
    The second time that Hacked server would try to make a request, it would get a deny from iptables and Nginx would not even know.
     
  6. eva2000

    eva2000 Administrator Staff Member

    50,891
    11,793
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,248
    Local Time:
    2:25 PM
    Nginx 1.25.x
    MariaDB 10.x
    @Revenge yeah that's what I've done as I have a script passing the pingback ips to CSF firewall + IPSET which is blocking them like a champ + bad bot botlimit setup :) The pingback attack is still actually happening while I type this reply :D

    You can see clearly when I enabled CSF Firewall banning
    newrelic-856am-00.png

    I have been working on a fail2ban configuration for Centmin Mod + CSF Firewall too. Something planned for later on :)
     
  7. Revenge

    Revenge Active Member

    469
    93
    28
    Feb 21, 2016
    Portugal
    Ratings:
    +354
    Local Time:
    4:25 AM
    1.9.x
    10.1.x
    @eva2000 i really like fail2ban, it has really a lot of potential. It will be a really good addition to centminmod (y)
     
  8. eva2000

    eva2000 Administrator Staff Member

    50,891
    11,793
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,248
    Local Time:
    2:25 PM
    Nginx 1.25.x
    MariaDB 10.x
    yeah I've been working on fail2ban + CSF Firewall configuration for months on and off - rather than fail2ban passing ips to iptables, they're passed to CSF Firewall which is basically a wrapper to iptables too just nicer command line interface :)
     
  9. eva2000

    eva2000 Administrator Staff Member

    50,891
    11,793
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,248
    Local Time:
    2:25 PM
    Nginx 1.25.x
    MariaDB 10.x
    what fun got null routed by Linode despite the server itself handling it well Forum DDOS Attacked - Linode null routed | Centmin Mod Community

    busy time ahead for me !

    by the numbers
    Code (Text):
    cat access.log | grep 'verifying pingback' | grep WordPress | ngxtop --no-follow
    running for 242 seconds, 5101795 records processed: 21124.78 req/sec
    
    Summary:
    |   count |   avg_bytes_sent |   2xx |   3xx |     4xx |   5xx |
    |---------+------------------+-------+-------+---------+-------|
    | 5101795 |            3.611 |   143 |     0 | 5101652 |     0 |
    
    Detailed:
    | request_path   |   count |   avg_bytes_sent |   2xx |   3xx |     4xx |   5xx |
    |----------------+---------+------------------+-------+-------+---------+-------|
    | /              | 5101795 |            3.611 |   143 |     0 | 5101652 
    

    Code (Text):
    cat access.log | grep 'verifying pingback' | grep WordPress | ngxtop --no-follow --group-by remote_addr
    running for 242 seconds, 5101795 records processed: 21110.20 req/sec
    
    Summary:
    |   count |   avg_bytes_sent |   2xx |   3xx |     4xx |   5xx |
    |---------+------------------+-------+-------+---------+-------|
    | 5101795 |            3.611 |   143 |     0 | 5101652 |     0 |
    
    Detailed:
    | remote_addr     |   count |   avg_bytes_sent |   2xx |   3xx |   4xx |   5xx |
    |-----------------+---------+------------------+-------+-------+-------+-------|
    | 62.210.100.209  |    7141 |            0.000 |     0 |     0 |  7141 |     0 |
    | 185.53.129.239  |    6508 |            0.000 |     0 |     0 |  6508 |     0 |
    | 174.129.36.212  |    6137 |            0.000 |     0 |     0 |  6137 |     0 |
    | 213.246.38.41   |    5638 |            0.000 |     0 |     0 |  5638 |     0 |
    | 84.246.225.122  |    4996 |            0.000 |     0 |     0 |  4996 |     0 |
    | 162.243.222.188 |    4892 |            0.000 |     0 |     0 |  4892 |     0 |
    | 206.131.38.124  |    4889 |            0.000 |     0 |     0 |  4889 |     0 |
    | 172.246.236.90  |    4821 |            0.000 |     0 |     0 |  4821 |     0 |
    | 104.155.13.96   |    4804 |            0.000 |     0 |     0 |  4804 |     0 |
    | 168.61.43.210   |    4744 |            0.000 |     0 |     0 |  4744 |     0 |
    

    Code (Text):
    cat access.log | grep 'verifying pingback' | grep WordPress | ngxtop --no-follow --group-by http_user_agent
    running for 246 seconds, 5101795 records processed: 20744.55 req/sec
    
    Summary:
    |   count |   avg_bytes_sent |   2xx |   3xx |     4xx |   5xx |
    |---------+------------------+-------+-------+---------+-------|
    | 5101795 |            3.611 |   143 |     0 | 5101652 |     0 |
    
    Detailed:
    | http_user_agent                                                                     |   count |   avg_bytes_sent |   2xx |   3xx |   4xx |   5xx |
    |-------------------------------------------------------------------------------------+---------+------------------+-------+-------+-------+-------|
    | WordPress/4.3.1; http://www.alfamitoclub.it; verifying pingback from 89.248.167.131 |    7141 |            0.000 |     0 |     0 |  7141 |     0 |
    | WordPress/4.4.2; http://bedroomcoders.co.uk; verifying pingback from 89.248.167.131 |    6508 |            0.000 |     0 |     0 |  6508 |     0 |
    | WordPress/4.4.1; http://bethelmusic.com; verifying pingback from 89.248.167.131     |    6408 |            0.000 |     0 |     0 |  6408 |     0 |
    | WordPress/4.4; http://174.129.36.212; verifying pingback from 89.248.167.131        |    6137 |            0.000 |     0 |     0 |  6137 |     0 |
    | WordPress/4.3.1; http://www.early-birds.fr; verifying pingback from 89.248.167.131  |    5478 |            0.000 |     0 |     0 |  5478 |     0 |
    | WordPress/4.4.2; http://www.alsim.com; verifying pingback from 89.248.167.131       |    4996 |            0.000 |     0 |     0 |  4996 |     0 |
    | WordPress/4.3; http://101healthyskin.com; verifying pingback from 89.248.167.131    |    4892 |            0.000 |     0 |     0 |  4892 |     0 |
    | WordPress/4.2.7; http://blogs.district279.org; verifying pingback from 127.0.0.1    |    4889 |            0.000 |     0 |     0 |  4889 |     0 |
    | WordPress/4.4.2; http://www.monkon.co.uk; verifying pingback from 127.0.0.1         |    4821 |            0.000 |     0 |     0 |  4821 |     0 |
    | WordPress/4.3.3; http://104.155.13.96; verifying pingback from 89.248.167.131       |    4804 |            0.000 |     0 |     0 |  4804 |     0 |
    

    Code (Text):
    cat access.log| grep '\"WordPress\/' | ngxtop --no-follow
    running for 275 seconds, 6078320 records processed: 22092.98 req/sec
    
    Summary:
    |   count |   avg_bytes_sent |   2xx |   3xx |     4xx |   5xx |
    |---------+------------------+-------+-------+---------+-------|
    | 6078320 |            3.031 |   143 |     0 | 6078177 |     0 |
    
    Detailed:
    | request_path                                   |   count |   avg_bytes_sent |   2xx |   3xx |     4xx |   5xx |
    |------------------------------------------------+---------+------------------+-------+-------+---------+-------|
    | /                                              | 6078318 |            3.031 |   143 |     0 | 6078175 |     0 |
    
     
  10. Moe

    Moe New Member

    17
    4
    3
    Dec 4, 2015
    Jupiter
    Ratings:
    +7
    Local Time:
    6:25 AM
    1.9.7
    MariaDB 10.x
    Seems its better to transfer forum to ovh ddos protected cloud,test their new models seems very cheap for spec they offer,2 core and 7 gb ram for 34$ if paid monthly
     
  11. deltahf

    deltahf Premium Member Premium Member

    565
    251
    63
    Jun 8, 2014
    Ratings:
    +463
    Local Time:
    11:25 PM
    This makes me absolutely furious. Who would want to target Centminmod, a great open-source tool, with something like this? :mad:

    My sympathies are with your family at this time, George, I will keep your mother in my thoughts and prayers. If we can take something positive away from it, maybe it will be a learning experience and will help you understand how to make Centminmod even more resilient to DDoS attacks in the future!
     
  12. hardousse

    hardousse Active Member

    165
    35
    28
    Dec 15, 2015
    Sweden
    Ratings:
    +56
    Local Time:
    6:25 AM
    1.11.*
    10.1*
    Even open source project its target with DDos really its something i can not undestand.
    George God give you power and energy to handle all because only good people do what you do ,i prayer for your mother because mine its far for me 2000 km.
    Best regards and God with you
     
  13. pamamolf

    pamamolf Well-Known Member

    4,028
    421
    83
    May 31, 2014
    Ratings:
    +817
    Local Time:
    6:25 AM
    Nginx-1.17.x
    MariaDB 10.3.x
    I hope all the best for you and for your mother !
     
  14. Moe

    Moe New Member

    17
    4
    3
    Dec 4, 2015
    Jupiter
    Ratings:
    +7
    Local Time:
    6:25 AM
    1.9.7
    MariaDB 10.x
    i believe brothers the problem is not open source or not,and whether its for profit or for people, it depends on the peoples' moralities and such descent society that @eva2000 "George" built in the past few years will not vanish by a DDOS attack or any other similar evil methods
     
  15. eva2000

    eva2000 Administrator Staff Member

    50,891
    11,793
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,248
    Local Time:
    2:25 PM
    Nginx 1.25.x
    MariaDB 10.x
    Indeed, looking into my options with regards to DDOS protection with a few things in the works hopefully :)

    thanks @deltahf for the thoughts and prayers for my mother :) Oh yes, Centmin Mod will get more improvements through this experience :)

    thanks @hardousse for your prayers for my mother :)

    cheers @pamamolf much appreciated :)

    Indeed @Moe. The real shame is with DDOS protection in front of my forum's Centmin Mod LEMP stack install, it won't really be show casing the pure performance of Centmin Mod LEMP setup but partially the performance of the DDOS protection in front of it. To date, I've been very proud to show case what pure Centmin Mod LEMP stack install can handle in terms of speed and performance for both HTTP and HTTPS based web sites.
     
  16. pamamolf

    pamamolf Well-Known Member

    4,028
    421
    83
    May 31, 2014
    Ratings:
    +817
    Local Time:
    6:25 AM
    Nginx-1.17.x
    MariaDB 10.3.x
    Checking a few more info about fail2ban on the net it seems an amazing option for Centminmod :)
     
  17. pamamolf

    pamamolf Well-Known Member

    4,028
    421
    83
    May 31, 2014
    Ratings:
    +817
    Local Time:
    6:25 AM
    Nginx-1.17.x
    MariaDB 10.3.x