Join the community today
Become a Member

The out of the box cache rules needs to be updated for WordPress running any ecommerce solution

Discussion in 'Install & Upgrades or Pre-Install Questions' started by Saumya Majumder, Aug 18, 2016.

  1. Saumya Majumder

    Saumya Majumder Member

    60
    3
    8
    Mar 16, 2016
    Ratings:
    +12
    Local Time:
    2:35 PM
    1.9.12
    10.0.24
    Hi @eva2000 ,
    on my VPS (I run a wordpress site) I used to be using ngx_pagespeed for months and recently I've turned off pagespeed and enabnled redis nxinx level caching + redis object caching.

    Now after the setup, I saw some problem happening due to these caching and my sites are not performing well.

    What are the problems you have seen?
    • As I use Easy Digital Download as my eCommerce solution on my WP site, I saw that even if I add something in the cart; when I go to homepage, blog article or any other page, the cart gets to 0 instead of 1 (as I said I've added a test product in cart).

      So, when I digged more about it I find out that EDD uses a cookie named

      Code:
      edd_items_in_cart
      which is not in the whitelist list of out of the box setting centminmod setting. Neither for redis nor wpcache or any other cache settings - which is creating this entire mess.

      I'm pretty sure the same problem will happen with woocommerce sites, but I personally don't use woocommerce.

    • The next problem I've faced is that any cache system of centminmod (as all use almost same caching rules) is caching the ecommerce related pages like checkout page, transaction complete, transaction failed etc. is getting cached as they aren't whitelisted in the config. EDD use the following slugs (by default)

      /checkout/ => for checkout page
      /checkout/purchase-confirmation/
      /checkout/purchase-history/
      /checkout/transaction-failed/
      /customer-dashboard/

      Now technically these pages are not supposed to be cached as it is different for all users.
    Have you whitelisted them in the config?
    No I haven't because what I looked into the rediscache_mysite.com.conf file, I saw the following config:

    Code:
    set $skip_cache 0;
    
    # POST requests and urls with a query string should always go to PHP
    if ($request_method = POST) {
      set $skip_cache 1;
    }
    
    if ($query_string != "") {
      set $skip_cache 1;
    }
    
    # Don't cache uris containing the following segments
    if ($request_uri ~* "/wp-admin/|/xmlrpc.php|wp-.*.php|/feed/|index.php|sitemap(_index)?.xml") {
      set $skip_cache 1;
    }
    
    # Don't use the cache for logged in users or recent commenters
    if ($http_cookie ~* "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in") {
      set $skip_cache 1;
    }
    
    location /redis-fetch {
      internal  ;
      set  $redis_key $args;
      redis_pass  redisbackend;
      redis_connect_timeout 60000;
      redis_read_timeout 60000;
      redis_send_timeout 60000;
    }
    
    location /redis-store {
      internal  ;
      set_unescape_uri $key $arg_key ;
      redis2_query set $key $echo_request_body;
      redis2_query expire $key 6h;
      redis2_pass  redisbackend;
      redis2_connect_timeout 60s;
      redis2_read_timeout 60s;
      redis2_send_timeout 60s;
    }
    
    Now here under the if block of cookie check I can easily add edd_items_in_cart within the check. But what I'm not sure of is how to add check for the above mentioned ecommerce pages?

    should it be like this:
    Code:
    if ($request_uri ~* "/wp-admin/|/xmlrpc.php|wp-.*.php|/feed/|index.php|sitemap(_index)?.xml|/checkout/|/checkout/purchase-confirmation/|/checkout/purchase-history/|/checkout/transaction-failed/|/customer-dashboard/") {
      set $skip_cache 1;
    }
    If you can take a look and let me know it will be really help.

    Also after I made the changes, should I make the same changes to any other file? And at last after the changes are made should I run just the following commands:

    Code:
    redis-cli flushall
    service redis restart
    or any other command along with it? Any help will be highly appriciated.
     
    Last edited: Aug 18, 2016
  2. Saumya Majumder

    Saumya Majumder Member

    60
    3
    8
    Mar 16, 2016
    Ratings:
    +12
    Local Time:
    2:35 PM
    1.9.12
    10.0.24
    Hey @eva2000, I'm graciously looking forward to your reply on this matter.
     
  3. eva2000

    eva2000 Administrator Staff Member

    30,161
    6,785
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    7:05 PM
    Nginx 1.13.x
    MariaDB 5.5
    which version of centmin mod ? 123.09beta01's wordpress auto installer via centmin.sh menu option 22 accounts for woocommerce in it's rules but not every ecommerce plugin so others need to be added by end user/you or like now you post a thread and notify me of the need to update. But seems the rediscache_vhostname.conf include was missed so will need to update that too.

    Will check it out and update 123.09beta01's wordpress auto install routines :)

    But you only need to do for rediscache_vhostname.conf include which will be updated version once i commit changes to 123.09beta01 branch. This ruleset accounts for both Easy Digital Downloads and WooCommerce wordpress plugins
    Code (Text):
    # Don't cache uris containing the following segments
    if ($request_uri ~* "\?add-to-cart=|/cart/|/my-account/|/checkout/|/shop/checkout/|/store/checkout/|/customer-dashboard/|/addons/|/wp-admin/|/xmlrpc.php|wp-.*.php|/feed/|index.php|sitemap(_index)?.xml") {
      set $skip_cache 1;
    }
    
     
    • Informative Informative x 1
  4. eva2000

    eva2000 Administrator Staff Member

    30,161
    6,785
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    7:05 PM
    Nginx 1.13.x
    MariaDB 5.5
    Just did a test Centmin Mod 123.09beta01 version of centmin.sh menu option 22 wordpress auto install for dummy domain = domain1.com to check the 3 variant include files for wp super cache, key cdn cache enabler and redis cache

    include files are placed in /usr/local/nginx/conf/wpincludes/domain1.com/
    Code (Text):
    ls -lah /usr/local/nginx/conf/wpincludes/domain1.com/
    total 24K
    drwxr-x--- 2 root root  146 Aug 18 15:13 .
    drwxr-x--- 3 root root   24 Aug 18 15:13 ..
    -rw-r--r-- 1 root root 1.2K Aug 18 15:13 rediscache_domain1.com.conf
    -rw-r--r-- 1 root root 1.4K Aug 18 15:13 wpcacheenabler_domain1.com.conf
    -rw-r--r-- 1 root root 8.4K Aug 18 15:13 wpsecure_domain1.com.conf
    -rw-r--r-- 1 root root  588 Aug 18 15:13 wpsupercache_domain1.com.conf

    updated /usr/local/nginx/conf/wpincludes/domain1.com/rediscache_domain1.com.conf
    Code (Text):
    set $skip_cache 0;
    
    # POST requests and urls with a query string should always go to PHP
    if ($request_method = POST) {
      set $skip_cache 1;
    }
    
    if ($query_string != "") {
      set $skip_cache 1;
    }
    
    # Don't cache uris containing the following segments
    if ($request_uri ~* "\?add-to-cart=|/cart/|/my-account/|/checkout/|/shop/checkout/|/store/checkout/|/customer-dashboard/|/addons/|/wp-admin/|/xmlrpc.php|wp-.*.php|/feed/|index.php|sitemap(_index)?.xml") {
      set $skip_cache 1;
    }
    
    # Don't use the cache for logged in users or recent commenters
    if ($http_cookie ~* "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in|edd_items_in_cart|woocommerce_items_in_cart") {
      set $skip_cache 1;
    }
    
    location /redis-fetch {
      internal  ;
      set  $redis_key $args;
      redis_pass  redisbackend;
      redis_connect_timeout 60000;
      redis_read_timeout 60000;
      redis_send_timeout 60000;
    }
    
    location /redis-store {
      internal  ;
      set_unescape_uri $key $arg_key ;
      redis2_query set $key $echo_request_body;
      redis2_query expire $key 6h;
      redis2_pass  redisbackend;
      redis2_connect_timeout 60s;
      redis2_read_timeout 60s;
      redis2_send_timeout 60s;
    }
    

    Updated keycdn cache enabler include /usr/local/nginx/conf/wpincludes/domain1.com/wpcacheenabler_domain1.com.conf
    Code (Text):
        set $cache_uri $request_uri;
    
        # bypass cache if POST requests or URLs with a query string
        if ($request_method = POST) {
            set $cache_uri 'nullcache';
        }
        if ($query_string != "") {
            set $cache_uri 'nullcache';
        }
    
        # bypass cache if URLs containing the following strings
        if ($request_uri ~* "(\?add-to-cart=|/cart/|/my-account/|/checkout/|/shop/checkout/|/store/checkout/|/customer-dashboard/|/addons/|/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(_index)?.xml|[a-z0-9_-]+-sitemap([0-9]+)?.xml)") {
            set $cache_uri 'nullcache';
        }
    
        # bypass cache if the cookies containing the following strings
        if ($http_cookie ~* "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_logged_in|edd_items_in_cart|woocommerce_items_in_cart") {
            set $cache_uri 'nullcache';
        }
    
        # custom sub directory e.g. /blog
        set $custom_subdir '';
    
        # default html file
        set $cache_enabler_uri '${custom_subdir}/wp-content/cache/cache-enabler/${http_host}${cache_uri}index.html';
    
        # webp html file
        if ($http_accept ~* "image/webp") {
            set $cache_enabler_uri '${custom_subdir}/wp-content/cache/cache-enabler/${http_host}${cache_uri}index-webp.html';
        }
    
     
    • Informative Informative x 1
  5. Saumya Majumder

    Saumya Majumder Member

    60
    3
    8
    Mar 16, 2016
    Ratings:
    +12
    Local Time:
    2:35 PM
    1.9.12
    10.0.24
    Hi @eva2000,
    Thanks a lot for your reply. I'm using v0.8 stable as it is a production site. I think you missed one thing in the above config, i.e. the sub checkout pages which will be like /checkout/* if I'm not wrong.

    I'm talking about these pages:
    /checkout/purchase-confirmation/
    /checkout/purchase-history/
    /checkout/transaction-failed/

    So in the end the final config will look like this:

    Code:
    set $skip_cache 0;
    
    # POST requests and urls with a query string should always go to PHP
    if ($request_method = POST) {
      set $skip_cache 1;
    }
    
    if ($query_string != "") {
      set $skip_cache 1;
    }
    
    # Don't cache uris containing the following segments
    if ($request_uri ~* "\?add-to-cart=|/cart/|/my-account/|/checkout/|/checkout/*|/shop/checkout/|/store/checkout/|/customer-dashboard/|/addons/|/wp-admin/|/xmlrpc.php|wp-.*.php|/feed/|index.php|sitemap(_index)?.xml") {
      set $skip_cache 1;
    }
    
    # Don't use the cache for logged in users or recent commenters
    if ($http_cookie ~* "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in|edd_items_in_cart|woocommerce_items_in_cart") {
      set $skip_cache 1;
    }
    
    location /redis-fetch {
      internal  ;
      set  $redis_key $args;
      redis_pass  redisbackend;
      redis_connect_timeout 60000;
      redis_read_timeout 60000;
      redis_send_timeout 60000;
    }
    
    location /redis-store {
      internal  ;
      set_unescape_uri $key $arg_key ;
      redis2_query set $key $echo_request_body;
      redis2_query expire $key 6h;
      redis2_pass  redisbackend;
      redis2_connect_timeout 60s;
      redis2_read_timeout 60s;
      redis2_send_timeout 60s;
    }
    
    Let me know if I made any mistakes above.

    Also thanks for the cache enabler config, but I guess I can't use it as I'm using both redis nginx level caching + redis object caching. So cache enabler will not work. Also I read on other threads here that using redis at negix level is far superior than cache enabler.

    Also after I made these changes in my config, should I need to run any other commends besides the following:

    Code:
    npreload
    redis-cli flushall
    service redis restart
    
    Please let me know. Looking forward to your reply.
     
  6. eva2000

    eva2000 Administrator Staff Member

    30,161
    6,785
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    7:05 PM
    Nginx 1.13.x
    MariaDB 5.5
    the $request_uri ~* part just for /checkout/ should match for all /checkout/subdirectories uris i believe, try it and see

    the keycdn cache enabler post was for benefit for others reading this as it is the default config for centmin mod centmin.sh menu option 22 in 123.09beta01
     
    • Informative Informative x 1
  7. eva2000

    eva2000 Administrator Staff Member

    30,161
    6,785
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    7:05 PM
    Nginx 1.13.x
    MariaDB 5.5
    quick test to confirm only one /checkout/ needed
    Code (Text):
    if ($request_uri ~* "/checkout") {
      set $checkout 1;
    }
    
    location ~* /checkout {
      echo $checkout;
    }
    

    Code (Text):
    curl localhost/checkout
    1
    
    curl localhost/checkout/subdir
    1
     
    • Like Like x 1
  8. eva2000

    eva2000 Administrator Staff Member

    30,161
    6,785
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    7:05 PM
    Nginx 1.13.x
    MariaDB 5.5
    Just these 2 should be enough
     
    • Like Like x 1
  9. Saumya Majumder

    Saumya Majumder Member

    60
    3
    8
    Mar 16, 2016
    Ratings:
    +12
    Local Time:
    2:35 PM
    1.9.12
    10.0.24
    Thank you very much. I'm doing this changes right now. Also as you mentioned above that you have added the cache enabler thing for everyone's help. Then I think you should also change the config code for wpcache, wpsupercache and fastcgi configs too. Just a thought.
     
  10. eva2000

    eva2000 Administrator Staff Member

    30,161
    6,785
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    7:05 PM
    Nginx 1.13.x
    MariaDB 5.5
  11. Saumya Majumder

    Saumya Majumder Member

    60
    3
    8
    Mar 16, 2016
    Ratings:
    +12
    Local Time:
    2:35 PM
    1.9.12
    10.0.24
    @eva2000 Hey, I need another help.

    I want all search bots to see non cache copy, so what should I put in the redis config? Can you please help?
     
  12. eva2000

    eva2000 Administrator Staff Member

    30,161
    6,785
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    7:05 PM
    Nginx 1.13.x
    MariaDB 5.5
  13. Saumya Majumder

    Saumya Majumder Member

    60
    3
    8
    Mar 16, 2016
    Ratings:
    +12
    Local Time:
    2:35 PM
    1.9.12
    10.0.24
    @eva2000 you mean like this:

    Code:
    map $http_user_agent $redis_device {
        default                               'desktop';
        ~*bots                                        'bots';
    }
    
    and then do this:

    Code:
    # exclude mobile devices from redis caching
    if ($redis_device = bots) {
      set $skip_cache 1;
    }
    
    Please let me know
     
  14. eva2000

    eva2000 Administrator Staff Member

    30,161
    6,785
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    7:05 PM
    Nginx 1.13.x
    MariaDB 5.5
    Yup just change $redis_device variable instances to unique variable name
     
  15. Saumya Majumder

    Saumya Majumder Member

    60
    3
    8
    Mar 16, 2016
    Ratings:
    +12
    Local Time:
    2:35 PM
    1.9.12
    10.0.24
    like redis_bots ? My above code is right? WOW! didn't expected that. :p
     
  16. eva2000

    eva2000 Administrator Staff Member

    30,161
    6,785
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    7:05 PM
    Nginx 1.13.x
    MariaDB 5.5
    Variable name can be anything really
     
  17. eva2000

    eva2000 Administrator Staff Member

    30,161
    6,785
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    7:05 PM
    Nginx 1.13.x
    MariaDB 5.5
    Oh the map default value can change from desktop to just 1 or something meaningful as you can di a reverse if match i.e. nonbot value instead if desktop
     
  18. Saumya Majumder

    Saumya Majumder Member

    60
    3
    8
    Mar 16, 2016
    Ratings:
    +12
    Local Time:
    2:35 PM
    1.9.12
    10.0.24
    Made the changes and restarted redis. Thank you for your help.
     
  19. Saumya Majumder

    Saumya Majumder Member

    60
    3
    8
    Mar 16, 2016
    Ratings:
    +12
    Local Time:
    2:35 PM
    1.9.12
    10.0.24
    I kept it as desktop as it was. Hope that's not gonna create any issue.
     
  20. eva2000

    eva2000 Administrator Staff Member

    30,161
    6,785
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,136
    Local Time:
    7:05 PM
    Nginx 1.13.x
    MariaDB 5.5
    yeah it should matter if you are using a unique variable map name and no referencing it anywhere

    i.e.

    Code (Text):
    map $http_user_agent $redis_bots {
        default                               '1';
        ~*bots                               'bots';
    }
    


    Code (Text):
    # exclude mobile devices from redis caching
    if ($redis_bots = bots) {
      set $skip_cache 1;
    }
    

    a reverse match
    Code (Text):
    # reverse match
    if ($redis_bots = 1) {
      .... do something....
    }
    
     
    • Like Like x 1