Join the community today
Register Now

WebPerf Optimize Images

Discussion in 'Web Development & Web Performance' started by Fernando, Aug 17, 2017.

  1. Fernando

    Fernando New Member

    17
    4
    3
    Jul 21, 2017
    Ratings:
    +5
    Local Time:
    3:49 PM
    1.13.3
    10.1.25
    Hi,

    I'm using optimise-images.sh for my website, thank you for this great tool! I'm planning to replace my plugins with optimise-images.sh :)

    I just have a question or maybe if you have any suggestion.
    I have set:
    Code:
    IMAGICK_WEBP='y'
    IMAGICK_WEBPLOSSLESS='n'
    And I'm planning to add:
    Code:
    UNATTENDED_OPTIMISE='y'
    So I can do this automatically.

    I can see the webp images properly in my website.
    The question that I have is my website allows customers to upload images, so at the end of the day I can have let's say 100 new images in uploads/2017/08 just as an example. I can even have more images per day in the folder.

    Is there a way that I can skip already optimized images? Or do you have any suggestion so I don't re-optimize a lot of images every time when I run the script?

    Thank you again for this great tool!
    Regards
     
  2. eva2000

    eva2000 Administrator Staff Member

    30,631
    6,862
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,308
    Local Time:
    7:49 AM
    Nginx 1.13.x
    MariaDB 5.5
    Is this wordpress site ? You should combine it with EWWW Image Optimizer + Imsanity i.e. Wordpress - Playing with EWWW Image Optimizer Wordpress Plugin. This allows automatic upload resize/optimisation in lossless format first in free EWWW plugin. Lossless optimisation usually only saves 2-5% overall for already ok optimized images and up to 5-20% for others.

    Then set optimise-images.sh via cronjob to work on /upload directory full path i.e. /home/nginx/domains/domain.com/public/wp-content/upload to do lossy optimisation on resulting EWWW images. The lossy optimisation boosts the image size reductions to between 20-85%.

    Currently, no way in optimise-images.sh script but if you look carefully in the optimise command verbose output you will see if you re-ran optimise mode on already optimised images, you don't do much at all

    example of re-running optimise-images.sh optimise mode on already optimised directory
    Code (Text):
    ------------------------------------------------------------------------------
    image optimisation start
    ------------------------------------------------------------------------------
    ### dslr_nikon_d7200_1.jpg (jpg) ###
    /bin/nice -n 10 /usr/bin/ionice -c2 -n7 /usr/bin/convert -define registry:temporary-path=/home/imagicktmp dslr_nikon_d7200_1.jpg -define jpeg:size=4096x4096 -filter triangle -define filter:support=2 -define jpeg:fancy-upsampling=off -unsharp 0.25x0.08+8.3+0.045 -interlace none -strip             -write mpr:dslr_nikon_d7200_1 -resize 2048x2048\> -write dslr_nikon_d7200_1.jpg +delete             mpr:dslr_nikon_d7200_1 -define webp:thread-level=1 -define webp:method=4 -define webp:alpha-quality=100 -define webp:lossless=false -quality 75 -resize 2048x2048\> dslr_nikon_d7200_1.jpg.webp
    /bin/nice -n 10 /usr/bin/ionice -c2 -n7 jpegoptim -p --max=82 dslr_nikon_d7200_1.jpg
    dslr_nikon_d7200_1.jpg 2048x1365 24bit N JFIF  [OK] 374587 --> 374561 bytes (0.01%), optimized.
    

    2nd run
    Code (Text):
    ------------------------------------------------------------------------------
    Original or Existing Images:
    ------------------------------------------------------------------------------
    | Avg width | Avg height | Avg quality | Avg size   | Total size (Bytes) | Total size (KB) |
    | --------- | ---------- | ----------- | --------   | ------------------ | --------------- |
    | 1166      | 945        | 87          | 438435     | 3507478            | 3425            |
    
    ------------------------------------------------------------------------------
    Optimised WebP Images:
    ------------------------------------------------------------------------------
    | Avg width | Avg height | Avg quality | Avg size   | Total size (Bytes) | Total size (KB) |
    | --------- | ---------- | ----------- | --------   | ------------------ | --------------- |
    | 1166      | 945        | 92          | 89689      | 717514             | 701             |
    

    3rd run
    Code (Text):
    ------------------------------------------------------------------------------
    Original or Existing Images:
    ------------------------------------------------------------------------------
    | Avg width | Avg height | Avg quality | Avg size   | Total size (Bytes) | Total size (KB) |
    | --------- | ---------- | ----------- | --------   | ------------------ | --------------- |
    | 1166      | 945        | 87          | 438415     | 3507317            | 3425            |
    
    ------------------------------------------------------------------------------
    Optimised WebP Images:
    ------------------------------------------------------------------------------
    | Avg width | Avg height | Avg quality | Avg size   | Total size (Bytes) | Total size (KB) |
    | --------- | ---------- | ----------- | --------   | ------------------ | --------------- |
    | 1166      | 945        | 92          | 89684      | 717476             | 701             |
    

    But I can see the problem you're describing. I could probably could add another mode to only look for images created/new in last XX hours i.e. 24hrs to process to limit the duplication of processing.
     
    • Like Like x 1
  3. Fernando

    Fernando New Member

    17
    4
    3
    Jul 21, 2017
    Ratings:
    +5
    Local Time:
    3:49 PM
    1.13.3
    10.1.25
    Hi :)

    This would be great!: I could probably could add another mode to only look for images created/new in last XX hours i.e. 24hrs to process to limit the duplication of processing.

    If I understand this: Then set optimise-images.sh via cronjob to work on /upload directory full path i.e. /home/nginx/domains/domain.com/public/wp-content/upload to do lossy optimisation on resulting EWWW images. The lossy optimisation boosts the image size reductions to between 20-85%.

    You suggest to use:
    Code:
    IMAGICK_WEBPLOSSLESS='y'
    However, in the post it says:
    IMAGICK_WEBPLOSSLESS='n' (kept at lossless as lossy 'y' results in large WebP images for some reason)

    So I want to double check what option should I set in regarsd to IMAGICK_WEBPLOSSLESS for better optimization.

    Once again thank you!
    Regards
     
  4. eva2000

    eva2000 Administrator Staff Member

    30,631
    6,862
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,308
    Local Time:
    7:49 AM
    Nginx 1.13.x
    MariaDB 5.5
    the optimise-images.sh lossy optimisation is for JPG/PNG only while keeping WebP IMAGICK_WEBPLOSSLESS='n' disabled
     
  5. Fernando

    Fernando New Member

    17
    4
    3
    Jul 21, 2017
    Ratings:
    +5
    Local Time:
    3:49 PM
    1.13.3
    10.1.25
    Hi,

    Thank you for your help! Now it's clear.
    Best Regards
     
  6. eva2000

    eva2000 Administrator Staff Member

    30,631
    6,862
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,308
    Local Time:
    7:49 AM
    Nginx 1.13.x
    MariaDB 5.5
    Updated optimise-images.sh with new profile-age & optimise-age Mode which allow you to filter and limit profiling and optimisation of images by image timestamp age :)
     
    • Winner Winner x 1
  7. eva2000

    eva2000 Administrator Staff Member

    30,631
    6,862
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,308
    Local Time:
    7:49 AM
    Nginx 1.13.x
    MariaDB 5.5
    slight correction, optimise-image.sh only operates 1 level deep so need to do some scripting as outlined at GitHub - centminmod/optimise-images: Batch image resizer, optimiser and profiler using ImageMagick convert, OptiPNG, JpegOptim and optional ZopfliPNG, Guetzli and MozJPEG. so to process directories at /home/nginx/domains/domain.com/public/wp-content/upload/2017/XX where XX is month

    i.e. new optimise-cron-age mode to filter by image timestamp age + specifically made for use as a cronjob as it saves processing time by removing the post-optimisation profile run. Set UNATTENDED_OPTIMISE='y' to disable the backup directory prompt so you can run unattended first. Edit optimise-cron-age command for command you want to use.
    Code (Text):
    
    find /home/nginx/domains/domain.com/public/wp-content/upload/2017/ -type d | sort | while read d; do echo "optimise directory: $d"; echo "/root/tools/optimise-images.sh optimise-cron-age $d"; /root/tools/optimise-images.sh optimise-cron-age $d; done
    chown -R nginx:nginx /home/nginx/domains/domain.com/public/wp-content/upload/2017/
    

    add a chown command to change file user/group permissions too
     
  8. Fernando

    Fernando New Member

    17
    4
    3
    Jul 21, 2017
    Ratings:
    +5
    Local Time:
    3:49 PM
    1.13.3
    10.1.25
    Hi,

    This is great! Thank you for your help :)
    Best Regards
     
    • Like Like x 1
  9. Fernando

    Fernando New Member

    17
    4
    3
    Jul 21, 2017
    Ratings:
    +5
    Local Time:
    3:49 PM
    1.13.3
    10.1.25
    Hi :)

    I was checking and testing. I noticed that the new filters will continue to optimize old images instead of new images. For example, if you set: FIND_IMGAGE=1440 and run the job today then everything from 8/17/2017 and older will be optimized, if you run the job again on 8/19/2017 then everything from 8/18/2017 and older will be optimized so the optimization will be performed twice, I believe the correct behaviour should be as you described:

    I could probably could add another mode to only look for images created/new in last XX hours i.e. 24hrs to process to limit the duplication of processing.

    With the current filter it just ignores X amount of time and optimize everything in the past :(

    Is this correct?
     
  10. eva2000

    eva2000 Administrator Staff Member

    30,631
    6,862
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,308
    Local Time:
    7:49 AM
    Nginx 1.13.x
    MariaDB 5.5
    whoops yeah now you explain, yes that is what FIND_IMGAGE does optimises stuff older than heh :oops:

    will fix it
     
  11. eva2000

    eva2000 Administrator Staff Member

    30,631
    6,862
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,308
    Local Time:
    7:49 AM
    Nginx 1.13.x
    MariaDB 5.5
  12. eva2000

    eva2000 Administrator Staff Member

    30,631
    6,862
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,308
    Local Time:
    7:49 AM
    Nginx 1.13.x
    MariaDB 5.5
    working on a new option add ADD_COMMENT='n' option · centminmod/optimise-images@ab1c7d3 · GitHub
    haven't done the skipping routine yet just putting the marker comment option in place first :)
     
  13. eva2000

    eva2000 Administrator Staff Member

    30,631
    6,862
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,308
    Local Time:
    7:49 AM
    Nginx 1.13.x
    MariaDB 5.5
    @Fernando @Matt will like this :)
    Code (Text):
    ./optimise-images.sh optimise /home/optimise-benchmarks
    
    ------------------------------------------------------------------------------
    image optimisation start
    ------------------------------------------------------------------------------
    ### dslr_nikon_d7200_1.jpg (jpg) skip already optimised ###
    ### webp-study-source-firebreathing.png (png) skip already optimised ###
    ### png24-interlaced-image1.png (png) skip already optimised ###
    ### dslr_nikon_d7200_2.jpg (jpg) skip already optimised ###
    ### dslr_canon_eos_m6_1.jpg (jpg) skip already optimised ###
    ### png24-image1.png (png) skip already optimised ###
    ### samsung_s7_mobile_1.jpg (jpg) skip already optimised ###
    ### bees.png (png) skip already optimised ###
    ------------------------------------------------------------------------------
    Completion Time: 2.55 seconds
    ------------------------------------------------------------------------------
     
    • Informative Informative x 1
  14. Fernando

    Fernando New Member

    17
    4
    3
    Jul 21, 2017
    Ratings:
    +5
    Local Time:
    3:49 PM
    1.13.3
    10.1.25
    Amazing! :) just tested and it works perfectly
    Thank you so much for everything!
     
  15. eva2000

    eva2000 Administrator Staff Member

    30,631
    6,862
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,308
    Local Time:
    7:49 AM
    Nginx 1.13.x
    MariaDB 5.5
    thanks for testing, bug reporting and feedback :D