Welcome to Centmin Mod Community
Become a Member

Cloudflare Cloudflare Authenticated Origin Pulls and "localhost": how to deal with local API access?

Discussion in 'Web Development & Web Performance' started by deltahf, May 29, 2021.

  1. deltahf

    deltahf Premium Member Premium Member

    423
    186
    43
    Jun 8, 2014
    Ratings:
    +327
    Local Time:
    6:14 AM
    I am redesigning my WordPress installation to make extensive use of the XenForo 2 API, so I can easily access threads, users, forums, etc. from WordPress, with a few simple curl requests in PHP. This works great in my local development environment, but as I move closer to production, I see a problem.

    On my production site, I use Cloudflare Authenticated Origin Pulls with Centminmod. Of course, this verifies the server will only ever respond to authenticated requests from Cloudflare. This means that all requests to my XenForo API will have to traverse the Internet through Cloudflare before they come all the way back to my server again.


    Obviously, this is not ideal. I want to be able to make requests to the API on my own "localhost", but from my understanding of Authenticated Origin Pulls, that won't work by design.

    What is the best way to get around this?
     
  2. eva2000

    eva2000 Administrator Staff Member

    46,851
    10,627
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +16,493
    Local Time:
    8:14 PM
    Nginx 1.19.x
    MariaDB 5.5/10.x
    You mean localhost on the production server ? Instead of localhost, you can setup a Cloudflare Argo tunnel https://blog.centminmod.com/2021/02/09/2250/how-to-setup-cloudflare-argo-tunnel-on-centos-7/ to tie a hostname to your localhost/port instance so it can be called from the hostname. Then you can just use CF Access to allow/deny IPs that can access that hostname.

    Or probably have to use Cloudflare API Shield instead Introducing API Shield and docs https://developers.cloudflare.com/firewall/cf-firewall-rules/api-shield

     
  3. deltahf

    deltahf Premium Member Premium Member

    423
    186
    43
    Jun 8, 2014
    Ratings:
    +327
    Local Time:
    6:14 AM
    Yes. WordPress and XenForo are on the same domain and server. If I connect to the XenForo API from WordPress, I want the connection to stay on the server and not get routed through Cloudflare. Routing the request through Cloudflare only causes additional latency because it would be reaching out to a Cloudflare edge server just to communicate with itself.

    Hmm, OK. Lots of questions here...

    I think I understand how Cloudflare Tunnel is supposed to work, and it looks like I probably need to upgrade to Tunnel (from Authenticated Origin Pulls) now that it's a free product for someone like me on the Pro plan. Would you recommend that?

    So, let's assume that I set up CF Tunnel. Let me run through everything as a sanity check to make sure I understand this correctly...

    I will have something like this ~/.cloudflared/config.yml file in your setup guide...

    Code (Text):
    tunnel: your_tunnelid
    credentials-file: /root/.cloudflared/your_tunnelid.json
    origincert: /root/.cloudflared/cert-tun.domain.com.pem
    protocol: http2
    originRequest:
      connectTimeout: 30s
    
    metrics: localhost:5432
    #tag: cmm=blog
    pidfile: /var/run/cmm-test-argo.pid
    autoupdate-freq: 24h
    loglevel: info
    logfile: /var/log/cloudflared.log
    
    ingress:
      - hostname: tun.domain.com
       service: https://localhost:443
       originRequest:
         connectTimeout: 10s
         noTLSVerify: true
      - hostname: hostname.domain.com
       service: https://localhost:443
       originRequest:
         connectTimeout: 10s
         noTLSVerify: true
      - hostname: host.example.com
       service: https://localhost:443
       originRequest:
         connectTimeout: 10s
         noTLSVerify: true
      - service: http_status:404
    


    So, cloudflared handles the incoming requests and then passes them along to nginx by calling https://localhost:443. Nginx matches the hostname with a server_name in a vhost config file and handles the request from there.

    Question 1:

    If cloudflared and nginx are both running on localhost, why do we need to access localhost via HTTPS? Why do we want the additional overhead of encrypting the traffic between the two processes on the same server? If nginx will now always use cloudflared to communicate with the outside internet, do we really need to mess with certificates or HTTPS configurations in vhost.conf settings at all?

    Question 2:

    If my site, we'll say tun.domain.com in this example, is accessible via http(s)://localhost, why would I need to set up CF Access or API Shield? Wouldn't the traffic stay on the server and never touch Cloudflare in this configuration?
     
  4. eva2000

    eva2000 Administrator Staff Member

    46,851
    10,627
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +16,493
    Local Time:
    8:14 PM
    Nginx 1.19.x
    MariaDB 5.5/10.x
    yup

    up to you if you want to over HTTPS or non-HTTPS with you communicate over HTTPS to HTTPS that is over TLSv1.3 which saves one round trip time between responses.

    Yeah you wouldn't but with Argo Tunnel you now have 3 segments, one between visitor and CF edge server, one between CF edge server and Cloudflared instance and one between Cloudflared instance and your origin. Again HTTPS with TLSv1.3 saves one round trip time between responses. On mobile that can be up to 200-400ms saved.

    You won't unless you want to be able to access API URL from the internet.
     
  5. deltahf

    deltahf Premium Member Premium Member

    423
    186
    43
    Jun 8, 2014
    Ratings:
    +327
    Local Time:
    6:14 AM
    Awesome, thank you so much for pointing me in the right direction. Everything is working great now with CloudFlare Tunnel!

    For future reference or anyone else curious about this:

    Switching from Authenticated Origin Pulls to CloudFlare Tunnel, I moved from CloudFlare's provided certificate and Full (Strict) SSL/TLS mode to a self-signed certificate (created using these Centminmod instructions) and "Full (non-strict)" SSL/TLS mode in the CF dashboard. I also added my site's domain to point to 127.0.0.1 in my server's /etc/hosts file.

    Now that my site was presenting a self-signed certificate and requests to my API were being properly routed directly back into the machine, curl can be configured to ignore certificate warnings without having to make any changes to the API endpoint URLs.

    Adding this PHP code before making my curl requests allows it ignore the self-signed certificate warnings:

    PHP:
    curl_setopt($chCURLOPT_SSL_VERIFYHOST0);
    curl_setopt($chCURLOPT_SSL_VERIFYPEER0);
    (Via StackOverflow)

    This method did not work with Cloudflare's provided Authenticated Origin Pull certificates, so if you want to access an API on your own site, I would strongly recommend CF Tunnels instead of Authenticated Origin Pull.
     
  6. eva2000

    eva2000 Administrator Staff Member

    46,851
    10,627
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +16,493
    Local Time:
    8:14 PM
    Nginx 1.19.x
    MariaDB 5.5/10.x
    Thanks for sharing your tips - glad to hear it all worked out :D