Join the community today
Register Now

Varnish in front of nGINX

Discussion in 'Other Centmin Mod Installed software' started by pamamolf, Jun 3, 2014.

Tags:
  1. pamamolf

    pamamolf Premium Member Premium Member

    4,074
    427
    83
    May 31, 2014
    Ratings:
    +833
    Local Time:
    8:22 AM
    Nginx-1.25.x
    MariaDB 10.3.x
    Hi


    Do we have any tutorial for this?

    Does it realy helps?

    Thank you
     
  2. eva2000

    eva2000 Administrator Staff Member

    54,110
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,739
    Local Time:
    4:22 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    no tutorials from myself, but other members here are free to offer advice :)

    Varnish is available since 1.2.3-eva2000.04 http://centminmod.com/changelog.html#123eva200004

    Reason is Varnish vcl configuration is a fine art needing a proper understanding of all variables (some change over Varnish cache versions) and like Nginx, Varnish VCL configuration is web app specific so you need to understand the workings of your web app i.e. wordpress, forum software etc to properly configure it. Hence, why I left installation to the end user who knows and understands how to configure Varnish Cache. It isn't something I'd provide support for free.

    But yes Varnish Cache helps ALOT, you can see may benchmarks on my blog at http://vbtechsupport.com/category/webtech/varnish/ i.e. http://vbtechsupport.com/26/


    [​IMG]

    Moving this thread to other software forums :)
     
  3. rdan

    rdan Well-Known Member

    5,443
    1,402
    113
    May 25, 2014
    Ratings:
    +2,194
    Local Time:
    2:22 PM
    Mainline
    10.2
    I hope you'll test it soon with XenForo Board :)
     
  4. eva2000

    eva2000 Administrator Staff Member

    54,110
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,739
    Local Time:
    4:22 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
  5. rdan

    rdan Well-Known Member

    5,443
    1,402
    113
    May 25, 2014
    Ratings:
    +2,194
    Local Time:
    2:22 PM
    Mainline
    10.2
    Dis regard Apache and OpenLiteSpeed :D
    Lets focus on Nginx :love:
     
  6. eva2000

    eva2000 Administrator Staff Member

    54,110
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,739
    Local Time:
    4:22 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    nah I need them for my own purposes as well i.e. extensive testing and comparison benchmarks all in one package = Centmin Mod :D
     
  7. rdan

    rdan Well-Known Member

    5,443
    1,402
    113
    May 25, 2014
    Ratings:
    +2,194
    Local Time:
    2:22 PM
    Mainline
    10.2
    Apache will never defeat Nginx anytime :p
     
  8. eva2000

    eva2000 Administrator Staff Member

    54,110
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,739
    Local Time:
    4:22 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    No some folks due to htaccess/rewrites might require nginx as a reverse proxy to apache so need to support that in the long term :)

    End goal is to be able to support on a Centmin Mod server the ability to have some sites running from pure Nginx, some on Apache and some on OpenLiteSpeed as well as support mixing them if needed.

    i.e.

    siteA = Nginx + PHP-FPM
    siteB = Apache 2.4 event + PHP-FPM
    siteC = Nginx reverse proxy to Apache 2.4 + PHP-FPM
    siteD = OpenLiteSpeed + LSAPI PHP

    All available simultaneously :D
     
  9. rdan

    rdan Well-Known Member

    5,443
    1,402
    113
    May 25, 2014
    Ratings:
    +2,194
    Local Time:
    2:22 PM
    Mainline
    10.2
    It takes years to develop such thing if you do it all alone :)
     
  10. eva2000

    eva2000 Administrator Staff Member

    54,110
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,739
    Local Time:
    4:22 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Individually each web server is already done with exception of OpenLiteSpeed stuff as it's still has development bugs etc. Just putting it all together and alot of testing. Hey that's what this forum and beta testers are for eventually ;) :D :happy:

    Before I started Centmin Mod, I had started development of an Apache equivalent called CentMAP, so the ground work was already there for an Apache + MariaDB + PHP-FPM auto installer, some of the features were ported to Centmin Mod i.e command short cuts etc. I abandoned CentMAP so can focus on Centmin Mod :)
     
  11. rdan

    rdan Well-Known Member

    5,443
    1,402
    113
    May 25, 2014
    Ratings:
    +2,194
    Local Time:
    2:22 PM
    Mainline
    10.2
    I think this config: My Varnish Configuration (default.vcl) | Live Free Die Travelling
    Code:
    # We only have one backend to define: NGINX
    backend default {
        .host = "127.0.0.1";
        .port = "8080";
    }
    # Only allow purging from specific IPs
    acl purge {
        "localhost";
        "127.0.0.1";
    }
    sub vcl_recv {
        /* Before anything else we need to fix gzip compression */
        if (req.http.Accept-Encoding) {
            if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
                # No point in compressing these
                remove req.http.Accept-Encoding;
            } else if (req.http.Accept-Encoding ~ "gzip") {
                set req.http.Accept-Encoding = "gzip";
            } else if (req.http.Accept-Encoding ~ "deflate") {
                set req.http.Accept-Encoding = "deflate";
            } else {
                # unknown algorithm
                unset req.http.Accept-Encoding;
            }
        }
        # Set client IP
        if (req.http.x-forwarded-for) {
            set req.http.X-Forwarded-For =
            req.http.X-Forwarded-For + ", " + client.ip;
        } else {
            set req.http.X-Forwarded-For = client.ip;
        }
        # Check if we may purge (only localhost)
        if (req.request == "PURGE") {
            if (!client.ip ~ purge) {
                error 405 "Not allowed.";
            }
            return(lookup);
        }
        if (req.request != "GET" &&
            req.request != "HEAD" &&
            req.request != "PUT" &&
            req.request != "POST" &&
            req.request != "TRACE" &&
            req.request != "OPTIONS" &&
            req.request != "DELETE") {
                # /* Non-RFC2616 or CONNECT which is weird. */
                return (pipe);
        }
        if (req.request != "GET" && req.request != "HEAD") {
            # /* We only deal with GET and HEAD by default */
            return (pass);
        }
        # admin users, facebook logged in users, XenForo users and comment authors always miss the cache
        if( req.http.Cookie ~ "xf_session_admin" || req.http.Cookie ~ "xf_session" || req.http.Cookie ~ "wordpress_logged_in_" || req.http.Cookie ~ "fbsr_537171586310880" || req.http.Cookie ~ "wp-postpass" || req.http.Cookie ~ "comment_author_"
         ){
                return (pass);
        }
        # Remove cookies set by Google Analytics (pattern: '__utmABC')
        if (req.http.Cookie) {
            set req.http.Cookie = regsuball(req.http.Cookie,
                "(^|; ) *__utm.=[^;]+;? *", "\1");
            if (req.http.Cookie == "") {
                remove req.http.Cookie;
            }
        }
        # Remove empty cookies.
        if (req.http.Cookie ~ "^\s*$") {
            unset req.http.Cookie;
        }
        # always pass through POST requests and those with basic auth
        if (req.http.Authorization && req.request == "POST") {
            return (pass);
        }
        # don't cache ajax requests and admin.php from XenForo
        if(req.http.X-Requested-With == "XMLHttpRequest" || req.url ~ "nocache" || req.url ~
    "(control.php|wp-comments-post.php|wp-login.php|register.php|admin.php)") {
            return (pass);
        }
        # Do not cache these paths
        if (req.url ~ "^/wp-cron\.php$" ||
            req.url ~ "^/xmlrpc\.php$" ||
            req.url ~ "^/apcstats\.php$" ||
            req.url ~ "^/wp-admin/.*$" ||
            req.url ~ "^/wp-includes/.*$" ||
            req.url ~ "\?s=" ||
            req.url ~ ".*fbconnect.*" ||
            req.url ~ ".*facebook.*" ||
            req.url ~ ".*fblink.*"  ||
            req.url ~ "/community/.*$" ||
            req.url ~ "^/admin\.php$" ) {
                return (pass);
        }
        # Define the default grace period to serve cached content
          if (req.backend.healthy)     {
                set req.grace = 60s;
                          }
        else     {
                set req.grace = 1h;
              }
        # remove ?ver=xxxxx strings from urls so css and js files are cached.
        # Watch out when upgrading WordPress, need to restart Varnish or flush cache.
        set req.url = regsub(req.url, "\?ver=.*$", "");
        # By ignoring any other cookies, it is now ok to get a page
        unset req.http.Cookie;
        return (lookup);
    }
    sub vcl_fetch {
        # remove some headers we never want to see
        unset beresp.http.Server;
        unset beresp.http.X-Powered-By;
        # don't cache admin or forums
        if (req.url ~ "wp-(login|admin)|community") {
        return (hit_for_pass);
        }
        # If WordPress or Facebook OAuth cookies found then page is not cacheable
        if (req.http.Cookie ~"(wp-postpass|wordpress_logged_in|xf_session_admin|xf_session|comment_author_|fbsr_537171586310880)") {
        #beresp.ttl>0 is cacheable so 0 will not be cached
        set beresp.ttl = 0s;
        } else {
           # set beresp.cacheable = true;
        set beresp.ttl=24h;#cache for 24hrs
        }
        # don't cache response to posted requests or those with basic auth
        if ( req.request == "POST" && req.http.Authorization ) {
            return (hit_for_pass);
        }
        # don't cache search results or XML RPC
        if (req.url ~ "\?s=" ||
            req.url ~ "\?P3_NOCACHE" ||
            req.url ~ "xmlrpc.php" ||
            req.url ~ "admin-ajax.php" ) {
                return (hit_for_pass);
        }
        # Handle ESI enabled AdRotate widget
        if (req.url ~ "esihandler.php") {
        set beresp.ttl = 30s;
         }
        else {
        set beresp.do_esi = true;
        set beresp.ttl = 24h;
        }
        # only cache status ok
        if ( beresp.status != 200 ) {
            return (hit_for_pass);
        }
        # If our backend returns 5xx status this will reset the grace time
        # set in vcl_recv so that cached content will be served and
        # the unhealthy backend will not be hammered by requests
        if (beresp.status == 500) {
            set beresp.grace = 1h;
            return (restart);
        }
        # GZip the cached content if possible
        if (beresp.http.content-type ~ "text") {
            set beresp.do_gzip = true;
        }
        # if nothing above matched it is now ok to cache the response
        set beresp.ttl = 24h;
        return (deliver);
    }
    sub vcl_deliver {
        # remove some headers added by varnish
        unset resp.http.Via;
        unset resp.http.X-Varnish;
    }
    sub vcl_hit {
        # Set up invalidation of the cache so purging gets done properly
        if (req.request == "PURGE") {
            purge;
            error 200 "Purged.";
        }
        return (deliver);
    }
    sub vcl_miss {
        # Set up invalidation of the cache so purging gets done properly
        if (req.request == "PURGE") {
            purge;
            error 200 "Purged.";
        }
        return (fetch);
    }
    sub vcl_error {
        if (obj.status == 503) {
                    # set obj.http.location = req.http.Location;
                    set obj.status = 404;
            set obj.response = "Not Found";
                    return (deliver);
        }
    }
    
    Is good enough to start :/
     
  12. eva2000

    eva2000 Administrator Staff Member

    54,110
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,739
    Local Time:
    4:22 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+