Learn about Centmin Mod LEMP Stack today
Register Now

Nginx Blocking Domains Instead Of IPs Nginx

Discussion in 'Nginx, PHP-FPM & MariaDB MySQL' started by skringjer, May 1, 2022.

Tags:
  1. skringjer

    skringjer NoobMaster69 Premium Member

    179
    23
    18
    Apr 21, 2019
    Ratings:
    +38
    Local Time:
    2:11 PM
    Nginx 1.21.6
    MariaDB 10.3.x
    Please fill in any relevant information that applies to you:
    • CentOS Version: CentOS 7 64bit
    • Centmin Mod Version Installed: 123.09beta01
    • Nginx Version Installed: 1.21.6
    • PHP Version Installed: 7.4.27
    • MariaDB MySQL Version Installed: 10.3.32
    • When was last time updated Centmin Mod code base ? : Automatic Cron
    • Persistent Config: Do you have any persistent config file options set in
    Greetings everyone, i was wondering if it is possible to block domains instead of IP from accessing the website / vhost.


    I have done some research and found this HTTP rDNS | NGINX but couldnt figure it out.

    So anyone with tips, please help out.
     
  2. eva2000

    eva2000 Administrator Staff Member

    48,440
    11,102
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +17,281
    Local Time:
    7:11 PM
    Nginx 1.21.x
    MariaDB 10.x
    What usage case do you have that requires such? Domains resolve to IP addresses and that is what Nginx would see.
     
  3. skringjer

    skringjer NoobMaster69 Premium Member

    179
    23
    18
    Apr 21, 2019
    Ratings:
    +38
    Local Time:
    2:11 PM
    Nginx 1.21.6
    MariaDB 10.3.x
    I would like to block certain websites, Because they have made exact clones of mine, and since i provide downloads on my website, they have scrapped those download links and serving them to their users, my images, content everything.

    Since they are behind CloudFlare i can not ban their IP unfortunately.
     
  4. eva2000

    eva2000 Administrator Staff Member

    48,440
    11,102
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +17,281
    Local Time:
    7:11 PM
    Nginx 1.21.x
    MariaDB 10.x
    Yeah you can't block domains from scraping your web sites exactly.

    In theory if they're using Cloudflare to proxy to origin IP, you can use an advanced form of Cloudflare Authenticated Origin pull https://community.centminmod.com/th...d-origin-pulls-protecting-your-origins.13847/ but instead of the link's outlined Cloudflare Origin pull CA certificate, you need to create your Certificate Authority (CA) root certificate and private key and then create and sign your own client TLS SSL certificate and upload that client TLS SSL certificate to Cloudflare via Cloudflare API and then enable Cloudflare Authenticated origin pull and setup the link's Nginx ssl_client_certificate path to your created CA root certificate and set ssl_verify_client on in your Nginx vhost.

    The advanced Cloudflared Authenticated Origin Pull set slightly differs for zone level apex domain.com or per hostname *.domain.com as outlined at https://developers.cloudflare.com/ssl/origin-configuration/authenticated-origin-pull/set-up/ 2nd and 3rd links below. But essentially does this same thing.

    1st link is Cloudflare Authenticated Origin Pull using Cloudflare provided CA certificate as outlined at https://community.centminmod.com/th...d-origin-pulls-protecting-your-origins.13847/. But that doesn't protect you against another Cloudflare protected/proxied orange cloud site from pointing to your origin real IP address as Cloudflare provided CA certificate will authenticate and allow the default Cloudflare client TLS certificate that is sent in requests to your origin.

    2nd & 3rd linked methods are advanced in that you create your own private CA certificate, CSR and private key and use that to create and sign your own client TLS certificate and upload that to CF via API. So when Cloudflare edge servers connect to your origin, it sends that uploaded custom client TLS certificate which can only be verified and validated against your custom created CA root certificate. Thus protecting your origin server from other Cloudflare users who proxy to your origin server's real IP address.

    The custom Cloudflare Authenticated Origin Pull requires knowledge of creating your own CA root certificate, CSR and private key and optionally create your own intermediate root certificate and private key and then use either of them to create your own self-signed client TLS certificates which are uploaded to Cloudflare via their API.

    Cloudflare has tools like cfssl to make that job easier and I wrote my own tool using cfssl to do just that at GitHub - centminmod/cfssl-ca-ssl but I don't provide any support so you'd be on your own figuring it out.

    Examples of creating own CA root and Intermediate root certificates GitHub - centminmod/cfssl-ca-ssl

    Examples of creating own client TLS certificates signed using created CA root/Intermediate root certificates and uploading custom per hostname *.domain.com based Authenticated Origin Pull certs GitHub - centminmod/cfssl-ca-ssl. For apex domain.com, you'd have to look at https://developers.cloudflare.com/s...ull/set-up/#zone-level--customer-certificates - difference is the Cloudflare API endpoint to upload your custom client TLS certificates to. For per hostname *.domain.com it's
    https://api.cloudflare.com/client/v4/zones/$cfzoneid/origin_tls_client_auth/hostnames/certificates and for apex domain.com it's

    https://api.cloudflare.com/client/v4/zones/$cfzoneid/origin_tls_client_auth

    But this only helps if those web scrapers are just using Cloudflare to proxy to your origin server IP. If they only setting up their sites behind Cloudflare and are in fact using their own origin server as a reverse proxy to your domain, then the custom Cloudflare Authenticated Origin pull certificate setups won't help.

    You can also inspect your Nginx logs and Cloudflare web analytics to see if can find patterns of access to restrict. Extending Nginx logs i.e. json logging for Cloudflare might give you more details https://community.centminmod.com/th...r-json-based-access-logging.19641/#post-86817. Then use Cloudflare Firewall's fields to create Firewall rules to restrict access https://developers.cloudflare.com/ruleset-engine/rules-language/fields/ as there are more advanced static, dynamic fields and request header, body fields and response fields you can create Cloudflare Firewall rules from. You can then use the insights you find from inspection to tailor Cloudflare Firewall rules in conjunction with using Cloudflare Transform rules https://developers.cloudflare.com/rules/transform that can modify the request and response headers before they hit Cloudflare. So if for instance you know that requests to a specific URL will always have a valid referrer request header, you can setup Cloudflare Firewall rules to look for that in determining if a request is valid.

    Cloudflare Firewall request body fields is one but that is Cloudflare Enterprise plan only https://developers.cloudflare.com/ruleset-engine/rules-language/fields/#http-request-body-fields. Cloudflare's Bot Management product can also protect against web scraping but it's also a Cloudflare Enterprise plan feature https://www.cloudflare.com/en-au/learning/bots/what-is-content-scraping/ and https://www.cloudflare.com/products/bot-management/





    .
     
    Last edited: May 1, 2022