Discover Centmin Mod today
Register Now

Wordpress Bug in instructions re: nginx Fastcgi cache for wordpress

Discussion in 'Blogs & CMS usage' started by jeffwidman, Jun 4, 2015.

  1. jeffwidman

    jeffwidman Active Member

    152
    27
    28
    Dec 3, 2014
    Ratings:
    +51
    Local Time:
    12:48 AM
    Nginx Wordpress Configuration Examples for rewrite rules & FastCGI Caching

    I'm not understanding why under the advanced guide it says to create a wpnocache file. That file is never included later on in the vhost, and the contents of it are already included in the wpcache file.


    I'm guessing it's just a leftover copy/paste from the basic setup part; although why it's even included int he basic setup part I don't understand, since the basic setup doesn't include setting up any kind of caching.
     
    Last edited: Jun 4, 2015
  2. eva2000

    eva2000 Administrator Staff Member

    55,197
    12,251
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,829
    Local Time:
    5:48 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    basic has wpnocache as it's a redirect to admin adding the ending forward slash but yes fastcgi cache version already includes that rewrite so no needed for wpnocache

    cheers (y)

    edit: ok i remember why it's included in fastcgi cache one.. if you want to temp disable fastcgi cache you can comment out the wpcache.conf and still retain the admin/ rewrite
     
  3. jeffwidman

    jeffwidman Active Member

    152
    27
    28
    Dec 3, 2014
    Ratings:
    +51
    Local Time:
    12:48 AM
    Might want to add that as a note in the docs under the advanced fastcgi section.

    Afraid I'm confused though--what's the point of ending wp-admin urls with a forward slash?? Sites seem to work fine without that... Is there some cache I'm unaware of that this forces a bypass on?
     
  4. eva2000

    eva2000 Administrator Staff Member

    55,197
    12,251
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,829
    Local Time:
    5:48 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    It's something that Apache users are more accustomed to I guess :)
     
  5. jeffwidman

    jeffwidman Active Member

    152
    27
    28
    Dec 3, 2014
    Ratings:
    +51
    Local Time:
    12:48 AM
    Finished getting this working today--really appreciated @eva2000 including the centminmod fastcgi docs examples, but there's a few gotchas that I'm jotting down here for anyone else who works on this:

    - while `fastcgi_cache_path` must be placed in nginx's `http` context, that doesn't mean it has to go in the nginx.conf file; it can be placed at the top of your vhost file. In fact, this is probably a better place if you're setting up multiple fastcgi_cache_paths for different php apps across multiple domains. Just be sure they each have unique zone names and also unique paths if you want to manage them separately. Example here: nginx - Is it possible to have a fastcgi_cache_path for each website / virtual host? - Server Fault

    - in the example in the docs, the cache gets disabled for mobile user-agents. This is only relevant if different content is served to mobile than to desktop, for example using a plugin like wp-touch that tries to modify the content for mobile. These days most themes are responsive using media-queries, so they're shipping the same content to mobile as to desktop, so this isn't applicable to most wordpress installs. Might as well keep the cache on for as many users as possible--and in fact the Wordpress Codex has this commented out by default.

    - wpsecure.php has a bunch of random crap in it that isn't related to securing wordpress at all. For example,
    Code:
    #disallow
    location ~* (roundcube|webdav|smtp|http\:|soap|w00tw00t) { return 444; } 
    
    Since this is being included within the wordpress location block, it will only match if the parent block matched, and could provide user with a false sense of security if they think their roundcube/webdav install is now protected. As long as you're aware of this, it doesn't really hurt to include it I guess, but for me it was just confusing cruft as I was trying to piece this together.

    - Several of the regex's don't properly escape the periods. I know in Nginx you don't have to escape forward slashes, but fairly sure you do need to escape periods. For example:
    Code:
    location ~ /(\.|wp-config.php|readme.html|license.txt) { deny all; }
    The periods are treated as a wildcard unless escaped, so `wp-config.php` should be `wp-config\.php`. I found several instances of this--worth checking the regexes against something like regexr.com or regex101.com
    Another example:
    Code:
    if ($request_uri ~* "(memcache.php|/wp-admin|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(_index)?.xml|[a-z0-9_-]+-sitemap([0-9]+)?.xml)") {
    
    This looks like it's a problem in the Wordpress Codex as well.

    - the format of the access log for cached requests is _slightly_ different from the normal centminmod nginx access log format. I recommend taking the normal access log format and just tacking on the line:
    Code:
    ' nocache:$no_cache $upstream_cache_status';
    as that way if you ever want to ship your logs out to a log-analysis tool you'll have a consistent format.

    @eva2000 - you ever consider swapping your wiki website into a format that we can submit github pull requests against? Something like ReadTheDocs format would make it so the rest of us can submit cleanup changes and then you accept/reject... make for better docs + less work for you.
     
  6. eva2000

    eva2000 Administrator Staff Member

    55,197
    12,251
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,829
    Local Time:
    5:48 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    wiki you mean Nginx Wordpress Configuration Examples for rewrite rules & FastCGI Caching ? they'll just pure html pages right now hehe. So would have to come up with something

    but thanks for the rest of the info and insights (y) will need time to chew on them :)
     
  7. jeffwidman

    jeffwidman Active Member

    152
    27
    28
    Dec 3, 2014
    Ratings:
    +51
    Local Time:
    12:48 AM
    If they're just flat files, then just stick them on github under a docs folder. Easy to submit pull requests.

    But it can get a bit janky over time trying to manage flat files; hence why many open-source projects move to ReadTheDocs/Sphinx or similar tool that builds pretty-looking docs from markdown source files... pretty results in standards-compliant markup while still being easily edited as plain text files/markdown.

    Also allows for nice things like partials/fragments that can get re-used multiple places.

    Totally up to you, just it's the natural transition many projects go through as they grow larger...
     
  8. eva2000

    eva2000 Administrator Staff Member

    55,197
    12,251
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,829
    Local Time:
    5:48 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    yeah been looking at document/manual builders for a while - just haven't found one i like :)
     
  9. jeffwidman

    jeffwidman Active Member

    152
    27
    28
    Dec 3, 2014
    Ratings:
    +51
    Local Time:
    12:48 AM
    Another item:

    If Wordpress is run from a subdirectory, they all need to have the wordpress subdirectory added to the front of the regex:
    Code:
    if ($request_uri ~* "/wordpress_subdirectory/(memcache\.php|apc\.php|wp-admin/.*|xmlrpc\.php|wp-(app|cron|login|register|mail)\.php|wp-.*\.php|feed/|index\.php|wp-comments-popup\.php|wp-links-opml\.php|wp-locations\.php|sitemap(_index)?\.xml|[a-z0-9_-]+-sitemap([0-9]+)?\.xml)") {
    
    1) I also fixed the regex to catch all wp-admin/.* values... theoretically it shouldn't be needed since those urls should be only hit by logged in users, but just in case someone's running some kind of weird cron that hits a wp-admin url or something.
    2) Even if Wordpress is placed in the root, the regex should still start with a single forward-slash outside of the inner parens. Currently it's mixed up because some of the filenames start with a forward-slash and some don't.... it's got to be either one way or the other, and simplest to place the forward slash outside the "(filenames)" so that it's applied to everything.
    3) The periods that were meant as periods and not wildcards need to be escaped with a backslash.

    @eva2000 it looks like you use this regex all over the place--WP-Supercache instructions, WP-FFPC instructions, etc.
     
    Last edited: Jun 7, 2015
  10. eva2000

    eva2000 Administrator Staff Member

    55,197
    12,251
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,829
    Local Time:
    5:48 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    thanks @jeffwidman really appreciate you taking the time to correct my oversights :D should have some of these corrected on the site by end of the day :)

    you sure ? not finding it on WP-FFPC page at Nginx Wordpress + WP-FFPC Plugin Setup + ngx_pagespeed ?
     
  11. jeffwidman

    jeffwidman Active Member

    152
    27
    28
    Dec 3, 2014
    Ratings:
    +51
    Local Time:
    12:48 AM
    No, thank you--you're the one who put in the hard work to set this up. Maybe set up that doc builder and I can do a pull request plus you can turn this into a fragment so it only has to be updated one place.

    My bad, you didn't use it there. Although possibly should to make sure stuff doesn't get cached that shouldn't... I've got an open issue on WP-FFPC as I'm a bit confused on whether it includes an object-cache alongside the full-page cache. If it includes the object cache, then it's okay AFAIK to cache logged-in user stuff, else probably best not to.
     
  12. jeffwidman

    jeffwidman Active Member

    152
    27
    28
    Dec 3, 2014
    Ratings:
    +51
    Local Time:
    12:48 AM
    Actually, I think I just found a much bigger bug:

    Basically I have two location contexts:

    location /wordpress/ { #outer location for pretty permalinks stuff
    if logged-in-cookie, set no-cache
    if url_args, set no cache
    location *.php { #inner location with php regex
    } #end inner location
    } # end outer location

    I manually set no-cache to always be 1 in the outer location block, but kept getting stuff in my cache. Looking at the logs, it looks like when a request was made to a url that ended in .php, it was getting generated in the cache. I suspect the problem is that Nginx is just skipping the outer location block and jumping straight to the inner one, so it never does the logged-in-cookie or url_args checks.

    It's quite late here, so I'm heading to bed, but this needs more testing. Aside from being inconvenient, it's a possible security hole because an admin could generate a cached result from admin.php which then gets served to a non-logged-in-user--the php itself won't be executed, but information might leak out that the admin wants to keep private.

    I've got a busy day tomorrow, but when I can I'll test it more. Easiest way is turn off the fastcgi cache completely, and just set a variable as if I were planning to use it for caching, and then check the value of that variable via logs or an additional header.

    The way I discovered it was while trying to implement FastCGI caching for Xenforo... I had an "if" statement that said "if $query_string != "" set $no_cache 1;" but then in my logs the url http://domainname.com/forum/css.php...read_posts_count&style=1&dir=LTR&d=1433655125 was generating a "MISS" when it should be generating a "BYPASS". I also had an "if $remote_addr != {{my_ip}} set $no_cache 1;" and even when I wasn't browsing the site new results where getting cached, which shouldn't be happening if the outer location "if" statements were working correctly. Pretty URLs work as expected, it's just when urls had *.php that I'm seeing funky behavior.
     
  13. eva2000

    eva2000 Administrator Staff Member

    55,197
    12,251
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,829
    Local Time:
    5:48 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    might need a fuller example so I can follow properly :)

    might be this from Nginx Wordpress Configuration Examples for rewrite rules & FastCGI Caching
     
    Last edited: Jun 8, 2015
  14. eva2000

    eva2000 Administrator Staff Member

    55,197
    12,251
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,829
    Local Time:
    5:48 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    yeah will look into that :)