Join the community today
Become a Member

Xenforo memcached & mod_pagespeed: Is force caching a good idea? Or is there such thing?

Discussion in 'Forum software usage' started by frm, Feb 11, 2020.

  1. frm

    frm Member

    34
    4
    8
    Dec 17, 2018
    Ratings:
    +6
    Local Time:
    4:01 AM
    1.15.7
    10.3.11
    When I make a major update to the forum, I need to clear the cache and mod_pagespeed sometimes or people are served with an older version and I'm unclear on when exactly it's updated to the latest version (sometimes a new comment, reply, etc. makes it recache; other than that, it's up in the air on when it actually does).

    So, I thought to myself: Why don't I "recache" it by visiting as a guest?

    Obviously I can't visit every page myself to recache a mod_pagespeed modified version of what's being served up...

    ...but I can have another VPS crawl the entire forum again with the code below:
    Code:
    wget \
         --recursive \
         --no-clobber \
         --html-extension \
         --convert-links \
         --restrict-file-names=windows \
         --domains website.com \
             www.website.com
    I let it run for about 15 minutes and download whatever was given to the other VPS, as those are probably the freshest pages anyhow.


    Bandwidth aside, I saw the database server spike to 10% and the file server to 50% CPU usage, so I know something was happening.

    I could leave it untouched and just let new pages cache again in due time.

    However, my thinking behind crawling and downloading on my own was SEO. In particular, the page speed and rules SEO look for (collapsed whitespace, etc. with mod_pagespeed) in order to give it that extra boost for speed and maybe other such stuff like forcing .webp images to be redone as the cache was cleared.

    After a major renovation, where I could still have the page speed as before, would crawling the site be advantageous or did I just waste time?

    I think this applies to all sites, but a simple blog with 50 pages would be easier as opposed to 1000s on a forum that could potentially harm SEO if speed and other elements are in fact a ranking factor to return the page to "normal" and serve up the smallest site possible as opposed to Google or another engine being the first to access the page and rendering it slower than the 2nd time they come around.
     
  2. frm

    frm Member

    34
    4
    8
    Dec 17, 2018
    Ratings:
    +6
    Local Time:
    4:01 AM
    1.15.7
    10.3.11
    Does anybody have an opinion on crawling to cache?
     
  3. eva2000

    eva2000 Administrator Staff Member

    55,400
    12,255
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,835
    Local Time:
    5:01 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    If you can crawl and preload and the you have measured actual benefits, then by all means do it. I do it with my Centmin Mod powered Wordpress blog at https://servermanager.guide/ I have 16+ VPS servers in 16 geographic locations that already run centminmod.com site, so I added a cronjob to each server to prewarm the Wordpress blog's Cloudflare respective region datacenter edge caches + PHP-FPM fastcgi guest full page HTML caches.