Cloudflare Firewall > Rate Limiting: Optimal settings?

It would vary depending on how your site is laid out too i.e. number of images, css, js being called per page too.

You'd have to inspect site's access.log in /home/nginx/domains/yourdomain.com/log/access.log and maybe rotated gzip compressed logs too as outlined in Config page Centmin Mod Configuration Files - CentminMod.com LEMP Nginx web stack for CentOS and maybe use zcat instead of cat or Centmin Mod 123.09beta01 also has pzcat if you have more than 2 cpu cores which uses multi-threaded pigz instead of single threaded gzip (zcat) for inspecting compressed access logs Sysadmin - zcat compressed access log processing benchmarks

To use pzcat on both uncompressed and compressed logs, need to use -f force flag with pzcat i.e.

Code (Text):

cd /home/nginx/domains/yourdomain.com/log/
pzcat -f access.log access.log-20180428.gz

which will output contents of both uncompressed access.log and compressed log and then pipe them through and use awk,grep,fgrep and other shell commands to manipulate the date i.e. filter IP addresses and date to calculate the rate of requests/s.

That's what I do to calculate Nginx's request rate for a point in time at Forum DDOS Attacked - Linode null routed

For example to inspect main hostname's nginx log which is at separate location at /var/log/nginx there are both uncompressed current localhost.access.log and compressed logs with suffix -201805xx.gz date timestamps

Code (Text):

ls -lah /var/log/nginx
total 728K
drwxr-xr-x 2 root  root 4.0K May 11 03:10 .
drwxr-xr-x 8 root  root 4.0K Apr 11 04:59 ..
-rw-rw---- 1 nginx root  22K May 11 12:16 localhost.access.log
-rw-rw---- 1 root  root  20K May  4 03:06 localhost.access.log-20180504.gz
-rw-rw---- 1 nginx root  15K May  5 03:08 localhost.access.log-20180505.gz
-rw-rw---- 1 nginx root  18K May  6 03:27 localhost.access.log-20180506.gz
-rw-rw---- 1 nginx root  40K May  7 03:28 localhost.access.log-20180507.gz
-rw-rw---- 1 nginx root  37K May  8 03:49 localhost.access.log-20180508.gz
-rw-rw---- 1 nginx root  23K May  9 03:35 localhost.access.log-20180509.gz
-rw-rw---- 1 nginx root  15K May 10 03:29 localhost.access.log-20180510.gz
-rw-rw---- 1 nginx root  88K May 11 03:10 localhost.access.log-20180511
-rw-rw---- 1 nginx root  45K May 11 12:15 localhost.error.log
-rw-rw---- 1 root  root  25K May  4 03:05 localhost.error.log-20180504.gz
-rw-rw---- 1 nginx root  19K May  5 03:07 localhost.error.log-20180505.gz
-rw-rw---- 1 nginx root  22K May  6 03:26 localhost.error.log-20180506.gz
-rw-rw---- 1 nginx root  50K May  7 03:27 localhost.error.log-20180507.gz
-rw-rw---- 1 nginx root  46K May  8 03:49 localhost.error.log-20180508.gz
-rw-rw---- 1 nginx root  28K May  9 03:34 localhost.error.log-20180509.gz
-rw-rw---- 1 nginx root  17K May 10 03:28 localhost.error.log-20180510.gz
-rw-rw---- 1 nginx root 141K May 11 03:07 localhost.error.log-20180511

so to inspect it for a particular IP address i.e. 174.129.1.66

just current non-rotated access log reload nginx to flush access log buffer to disk and cd into log directory and use cat command piped to a grep filter for IP entry

Code (Text):

ngxreload
cd /var/log/nginx
cat localhost.access.log | grep '174.129.1.66'

example output has 3 entries, you may have 1000s

Code (Text):

ngxreload
cd /var/log/nginx
cat localhost.access.log | grep '174.129.1.66'
174.129.1.66 - - [11/May/2018:03:20:44 +0000] "HEAD /haircut-places-in-san-antonio HTTP/1.1" 404 0 "-" "MBCrawler/1.0 (https://monitorbacklinks.com)"
174.129.1.66 - - [11/May/2018:03:20:44 +0000] "GET /haircut-places-in-san-antonio HTTP/1.1" 404 162 "-" "MBCrawler/1.0 (https://monitorbacklinks.com)"
174.129.1.66 - - [11/May/2018:11:52:21 +0000] "HEAD /haircut-places-in-san-antonio HTTP/1.1" 404 0 "-" "MBCrawler/1.0 (https://monitorbacklinks.com)"
174.129.1.66 - - [11/May/2018:11:52:21 +0000] "GET /haircut-places-in-san-antonio HTTP/1.1" 404 162 "-" "MBCrawler/1.0 (https://monitorbacklinks.com)"

to count number in just current log pipe previous command through wc -l to count number of entries matching that grep filtered IP address

Code (Text):

ngxreload
cd /var/log/nginx
cat localhost.access.log | grep '174.129.1.66' | wc -l
4

Now if you need to look through all rotated gzip compressed logs and current uncompressed logs use pzcat if you have 2 or more cpu cores or use zcat if you have 1 cpu core

Code (Text):

ngxreload
cd /var/log/nginx
pzcat -f localhost.access.log localhost.access.log-* | grep '174.129.1.66' | wc -l
40

which shows there's 40 matching entries from all rotated gzip compressed logs and current uncompressed logs dating back to May 4th log rotation time for localhost.access.log-20180504.gz which has a start date of May 3 at 8:31 AM

Code (Text):

ngxreload
cd /var/log/nginx
pzcat localhost.access.log-20180504.gz | head -n1
93.158.161.176 - - [03/May/2018:08:31:23 +0000] "GET /should-i-grow-my-hair-out-men/phenomenalhaircare-sisterlocks-showing-length-in-bundles-and-rods HTTP/1.1" 404 162 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"

Convert that May 3 time into linux epoch time change forward slashs / to dashes - and putting space between year and time

Code (Text):

date -d "03-May-2018 08:31:23" +%s
1525336283

current date time in epoch time

Code (Text):

date +%s
1526043594

Time period is over 1526043594 - 1525336283 = 707,311 seconds

Code (Text):

echo $((1526043594 - 1525336283))
707311

Or calculate number of 404 entries requested for url /haircut-places-in-san-antonio via grep filter - shows 84 entries

Code (Text):

ngxreload
cd /var/log/nginx
pzcat -f localhost.access.log localhost.access.log-* | grep '/haircut-places-in-san-antonio' | wc -l      
84

Then you can go further to filter using date command and some bash calculation/maths for just a point in time period and work out number of hits for that filtered IP over that time interval to work out the request rate specifically.

Probably other tools you can use too. Google is your friend

 

haha power of GUI and cloudflare

 

Wrong calculation rate limiting is per request processed through rate limiting rules iirc Billing for Cloudflare Rate Limiting

 

interesting no where in their doc/pricing info do they mention that part. Cloudflare support confirmed this ?

 

yeah liked it until saw the pricing