Learn about Centmin Mod LEMP Stack today
Become a Member

Install Upgrade Nginx PageSpeed optimise-images WebP with optimise-images.sh

Discussion in 'Install & Upgrades or Pre-Install Questions' started by EckyBrazzz, May 10, 2019.

  1. EckyBrazzz

    EckyBrazzz Active Member

    230
    40
    28
    Mar 28, 2018
    Brazil
    Ratings:
    +78
    Local Time:
    1:38 PM
    1.15.x
    10.3.x
    As I'm say "Goodbye ngx_pagespeed" because it limits me a little and get's in the way of the functionality of some nice things that CMM has. I need to find some other solutions and found the, for me almost perfect solution, namely the optimise-images.sh script. Have 2 issues. I'm very glad it works :)

    Got it working following this post https://community.centminmod.com/th...tespeed-vs-centmin-mod-lemp.15211/#post-65241 but noticed one big disappointment namely item 1
    The second item is the cronjob, it's not clear to me how to set it up
    1. My score at www.webpagetest.org Cache static content went from A to F optimise-images1.png
    Code (Text):
    wp_upload_dir='/home/nginx/domains/domain.com/public/wp-content/uploads-test'
    
    find $wp_upload_dir -maxdepth 2 -mindepth 2 -type d | grep -E '[0-9]{4}' | grep "$(date +"%Y\/%m")" | sort | while read d; do if [[ "$(ls -Al $d | head -n1 | grep -o 'total 0')" != 'total 0' ]]; then echo "optimise directory: $d"; echo "/root/tools/optimise-images/optimise-images.sh optimise-cron $d"; /root/tools/optimise-images/optimise-images.sh optimise-cron $d; chown -R nginx:nginx $d; fi; done
    


    How to set up crontab -e so that when uploading a new image it gets almost instantly convertert into WebP? For example that it runs every 10 or 20 minutes.

    It's not clearly described on how to set up the cronjob, need a little helping hand on these 2 issues.
     
  2. eva2000

    eva2000 Administrator Staff Member

    39,758
    8,769
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +13,494
    Local Time:
    2:38 AM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    you can't do that via a cronjob for instant conversion, you may need to use something like EWWW Image Optimizer Wordpress plugin

    Also @ShawnH modified my optimise-images.sh script for Wordpress usage and outlined his process at https://community.centminmod.com/threads/optimise-images-modification-and-plugin.16851/ - haven't tried his version yet though, so try and see for yourself.

    Official optimise-images.sh github repo has more cronjob examples for Unattended Subdirectory Runs in general sense. I also snuck in a not yet mentioned wp cron script too optimise-images-wpcron.sh which utilises the optimise-cron & optimise-cron-age mode for use in cronjob scheduled runs which removes the post-optimisation profile routine run to save time. The optimise-cron-age is based on outlined optimise-age mode which allows you to filter and limit profiling and optimisation of images by image timestamp age.

    Snuck in as I ain't providing support for how to use it etc so just left it there for folks to fine and use if they know what to do from instructions/shell script code within the script and from Unattended Subdirectory Runs outlined example options.
    .
     
  3. EckyBrazzz

    EckyBrazzz Active Member

    230
    40
    28
    Mar 28, 2018
    Brazil
    Ratings:
    +78
    Local Time:
    1:38 PM
    1.15.x
    10.3.x
    Well, came acrous the plugin WebP Express but it needs the Gmagick extension.
    WebP Express doesn't work out of the box with nginx, but there is a little manual on how to get it working.

    Gave Install Graphics Magick on CentOS 7 a try, but could not get the Gmagick extension to work on our instalation.

    How to enable the Gmagick php extension?

     
  4. eva2000

    eva2000 Administrator Staff Member

    39,758
    8,769
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +13,494
    Local Time:
    2:38 AM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    maybe a topic for a separate thread in system admin forums https://community.centminmod.com/forums/system-administration.39/
     
  5. EckyBrazzz

    EckyBrazzz Active Member

    230
    40
    28
    Mar 28, 2018
    Brazil
    Ratings:
    +78
    Local Time:
    1:38 PM
    1.15.x
    10.3.x
    Well, Installed Gmagick php extension but noticed that I could not use it together with imagick, my mistake even opened a bug report for it (marked it as solved). But go and give the plugin a try, because I guess it will resolve the cache issue and is on the fly.
     
    • Informative Informative x 1
  6. eva2000

    eva2000 Administrator Staff Member

    39,758
    8,769
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +13,494
    Local Time:
    2:38 AM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    WebP Express interesting indeed but seems to use .htaccess apache rewrite rules for conditional browser webp supported rewrites which nginx doesn't support .htaccess. But Centmin Mod Nginx in 123.09beta01 native can rewrite with instructions at https://centminmod.com/webp/. You'd have to used mingled destination folder mode though for Centmin Mod Nginx's webp to pick up the converted webp version

    upload_2019-5-11_2-8-48.png

    also centmin.sh menu option 22 installed wordpress wpsecure whitelisting is needed as php files can't be run in WebP Express doing conversion test = 403 no permission as per https://community.centminmod.com/threads/wordpress-403-permission-denied-errors.11215/

    upload_2019-5-11_2-11-40.png

    added whitelist exception in my test domain's (funky.domain.com) wpsecure include file /usr/local/nginx/conf/wpincludes/funky.domain.com/wpsecure_funky.domain.com.confright after the last whitelist exception location context

    replace all references to funky.domain.com with your domain name :)
    Code (Text):
    # Whitelist Exception for webp-express
    location ~ ^/wp-content/plugins/webp-express/ {
      include /usr/local/nginx/conf/php.conf;
      include /usr/local/nginx/conf/staticfiles.conf;
      # below include file needs to be manually created at that path and to be uncommented
      # by removing the hash # in front of below line to take effect. This wpwhitelist_common.conf
      # allows you to add commonly shared settings to all wp plugin location matches which
      # whitelist php processing access at /usr/local/nginx/conf/wpincludes/funky.domain.com/wpsecure_funky.domain.com.conf
      #include /usr/local/nginx/conf/wpincludes/funky.domain.com/wpwhitelist_common.conf;
    }
    

    then restart php-fpm service and now test conversions work. This wpsecure whitelisting allows you finer grain control over which wordpress plugins can run php code from /wp-content/plugins/yourpluginname directory.

    upload_2019-5-11_2-18-43.png

    test media image upload to see if it converts to webp file

    upload_2019-5-11_2-33-59.png


    nice
    Code (Text):
    find /home/nginx/domains/funky.domain.com/public/wp-content/uploads -type f -name "*.webp"
    /home/nginx/domains/funky.domain.com/public/wp-content/uploads/2019/05/epyc2-cray-8180m-01-100x100.png.webp
    /home/nginx/domains/funky.domain.com/public/wp-content/uploads/2019/05/epyc2-cray-8180m-01-300x114.png.webp
    /home/nginx/domains/funky.domain.com/public/wp-content/uploads/2019/05/epyc2-cray-8180m-01-150x150.png.webp
    /home/nginx/domains/funky.domain.com/public/wp-content/uploads/2019/05/epyc2-cray-8180m-01-324x304.png.webp
    /home/nginx/domains/funky.domain.com/public/wp-content/uploads/2019/05/epyc2-cray-8180m-01-768x292.png.webp
    /home/nginx/domains/funky.domain.com/public/wp-content/uploads/2019/05/epyc2-cray-8180m-01-416x158.png.webp
    /home/nginx/domains/funky.domain.com/public/wp-content/uploads/2019/05/epyc2-cray-8180m-01.png.webp
    


    all that is left is setup Centmin Mod Nginx's conditional .webp extension rewrite for browsers that support webp as per https://centminmod.com/webp/. Centmin Mod 123.09beta01 already has setup instructed /usr/local/nginx/conf/webp.conf include file in nginx.conf.

    you can do that by editing wpsecure for your wordpress site i.e. for my funky.domain.com wordpress site would be at
    /usr/local/nginx/conf/wpincludes/funky.domain.com/wpsecure_funky.domain.com.conf
    and adding right after nginx-helper log blocking location context the following
    Code (Text):
    # webp extension support if you are converting /uploads images to webp
    location ~ ^/wp-content/uploads/ {
      #pagespeed off;
      #pagespeed unplugged;
      #autoindex on;
      #add_header X-Robots-Tag "noindex, nofollow";
      location ~* ^/wp-content/uploads/(.+/)?(.+)\.(png|jpe?g)$ {
       expires 30d;
       add_header Vary "Accept-Encoding";
       add_header Cache-Control "public, no-transform";
       try_files $uri$webp_extension $uri =404;
      }
    }
    

    so in wpsecure include file right after block nginx-helper log context
    Code (Text):
    # Block nginx-help log from public viewing
    location ~* /wp-content/uploads/nginx-helper/ { deny all; }
    
    # webp extension support if you are converting /uploads images to webp
    location ~ ^/wp-content/uploads/ {
      #pagespeed off;
      #pagespeed unplugged;
      #autoindex on;
      #add_header X-Robots-Tag "noindex, nofollow";
      location ~* ^/wp-content/uploads/(.+/)?(.+)\.(png|jpe?g)$ {
        expires 30d;
        add_header Vary "Accept-Encoding";
        add_header Cache-Control "public, no-transform";
        try_files $uri$webp_extension $uri =404;
      }
    }
    

    then restart nginx + php-fpm
    Code (Text):
    nprestart
    

    inspecting Chrome devtools network tab

    webp-express-working-centminmod-nginx-01.png
    webp-express-working-centminmod-nginx-02.png
    FYI, seems with Guttenberg editor in wordpress Centmin Mod Nginx's editing posts gives 403 permission denied for /wp-json requests, need to figure that out so for now installed and reverted to installing Classic Editor wordpress plugin
     
    Last edited: May 11, 2019
    • Winner Winner x 1
  7. EckyBrazzz

    EckyBrazzz Active Member

    230
    40
    28
    Mar 28, 2018
    Brazil
    Ratings:
    +78
    Local Time:
    1:38 PM
    1.15.x
    10.3.x
    Works with imagick or with gmagick, its or / or situation, but when installing the plugin it's a bit confusing because it tells that both are required when hoovering the extension. Again, RTFM.
     
    • Agree Agree x 1
  8. eva2000

    eva2000 Administrator Staff Member

    39,758
    8,769
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +13,494
    Local Time:
    2:38 AM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    updated my previous post with the centmin.sh menu option 22 wordpress auto installer's wpsecure whitelisting syntax to allow WebP Express to work - probably add it to default whitelisted plugins in next 123.09beta01 update so folks don't need to whitelist it manually :)
     
  9. eva2000

    eva2000 Administrator Staff Member

    39,758
    8,769
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +13,494
    Local Time:
    2:38 AM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    • Like Like x 1
  10. EckyBrazzz

    EckyBrazzz Active Member

    230
    40
    28
    Mar 28, 2018
    Brazil
    Ratings:
    +78
    Local Time:
    1:38 PM
    1.15.x
    10.3.x
    Perfect, now others can take the benefits of it! I there was a posseble to muliple "like" I would give you a 1000 of them at once!
     
    • Like Like x 1
  11. fabianski

    fabianski Member

    41
    2
    8
    Feb 20, 2019
    Ratings:
    +8
    Local Time:
    1:38 PM
    So only new installations already have this enabled by default?
    Will this plugin only convert the new images to WebP or also those already in the gallery?
     
  12. eva2000

    eva2000 Administrator Staff Member

    39,758
    8,769
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +13,494
    Local Time:
    2:38 AM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    no WebP Express isn't being installed by centmin.sh menu option 22, you have to manually install it still as outlined at https://community.centminmod.com/th...-wordpress-plugin-on-centmin-mod-nginx.17469/

    WebP Express only converts new images uploaded to Wordpress.
     
    • Like Like x 1
  13. EckyBrazzz

    EckyBrazzz Active Member

    230
    40
    28
    Mar 28, 2018
    Brazil
    Ratings:
    +78
    Local Time:
    1:38 PM
    1.15.x
    10.3.x
    You can use the BULK option as well. Figured that out when I told it was magic or it was eva2000.
     
    • Informative Informative x 1
..