Learn about Centmin Mod LEMP Stack today
Become a Member

Xenforo Xenforo users what's your image proxy cache size ?

Discussion in 'Forum software usage' started by eva2000, May 2, 2018.

  1. eva2000

    eva2000 Administrator Staff Member

    50,922
    11,805
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,263
    Local Time:
    8:26 AM
    Nginx 1.25.x
    MariaDB 10.x
    Came across this thread on Xenforo support forums regarding the size of image proxy cache What would be a reasonable server disk size for a big forum with SSL on. When Xenforo is used with HTTPS, you want to enable the image proxy so 3rd party references images in posts via IMG tag are proxied via Xenforo image proxy so there is no mixed content HTTPS errors. Xenforo image proxy caches these proxied images for a certain time which means increased disk usage before Xenforo image proxy cache TTL expires the cached entries.

    The thread linked to above shows how you can calculate your Xenforo image proxy cache size. So curious what other folks numbers look like. For this forum it's where domain.com is my domain name.
    Code (Text):
    cd /home/nginx/domains/domain.com/public/internal_data
    find /home/nginx/domains/domain.com/public/internal_data/image_cache/  | wc -l ;  du -hs ./image_cache/
    

    which comes to just 613 image proxy cached images taking up 68MB of disk space
    Code (Text):
    613
    68M     ./image_cache/
    

    Or with a bit of extra manipulation use
    Code (Text):
    cd /home/nginx/domains/domain.com/public/internal_data
    (find /home/nginx/domains/domain.com/public/internal_data/image_cache/  | wc -l ;  du -s ./image_cache/) 2>&1 | xargs | while read n s d; do echo "total images: $n total size: $s KB avg size/image: $(($s/$n)) KB"; done
    

    will return
    Code (Text):
    total images: 612 total size: 69240 KB avg size/image: 113 KB
    


     
  2. rdan

    rdan Well-Known Member

    5,426
    1,389
    113
    May 25, 2014
    Ratings:
    +2,172
    Local Time:
    6:26 AM
    Mainline
    10.2
    Code:
    total images: 24626 total size: 2902756 KB avg size/image: 117 KB
     
  3. Jon Snow

    Jon Snow Active Member

    727
    150
    43
    Jun 30, 2017
    Ratings:
    +215
    Local Time:
    6:26 PM
    Nginx 1.13.9
    MariaDB 10.1.31
    Code:
    total images: 3031 total size: 773752 KB avg size/image: 255 KB
     
  4. Andy

    Andy Active Member

    519
    84
    28
    Aug 6, 2014
    Ratings:
    +121
    Local Time:
    5:26 PM
    Apparently, my site is not images heavy
    total images: 260 total size: 14076 KB avg size/image: 54 KB
     
  5. eva2000

    eva2000 Administrator Staff Member

    50,922
    11,805
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,263
    Local Time:
    8:26 AM
    Nginx 1.25.x
    MariaDB 10.x
    Interesting numbers guys.. thanks for sharing. Some of you have larger forums too ! I can't imagine how that linked Xenforo user had to triple their disk capacity to 150GB and still run out of disk space due to Xenforo image proxy local disk caching ! That's some heavy 3rd party image posting activity !
     
  6. Sunka

    Sunka Well-Known Member

    1,150
    325
    83
    Oct 31, 2015
    Pula, Croatia
    Ratings:
    +525
    Local Time:
    11:26 PM
    Nginx 1.17.9
    MariaDB 10.3.22
    Code (Text):
    total images: 1242 total size: 66928 KB avg size/image: 53 KB
     
  7. rdan

    rdan Well-Known Member

    5,426
    1,389
    113
    May 25, 2014
    Ratings:
    +2,172
    Local Time:
    6:26 AM
    Mainline
    10.2
    Mine is even lower now.
    Because I only cached 2mb max images and Ignore https images.

    upload_2018-5-3_2-28-55.png
     
  8. eva2000

    eva2000 Administrator Staff Member

    50,922
    11,805
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,263
    Local Time:
    8:26 AM
    Nginx 1.25.x
    MariaDB 10.x
    @RoldanLT you using an XF Addon for ignore https images ?
     
  9. rdan

    rdan Well-Known Member

    5,426
    1,389
    113
    May 25, 2014
    Ratings:
    +2,172
    Local Time:
    6:26 AM
    Mainline
    10.2
  10. eva2000

    eva2000 Administrator Staff Member

    50,922
    11,805
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,263
    Local Time:
    8:26 AM
    Nginx 1.25.x
    MariaDB 10.x
    Ah looked familiar :)
     
  11. dcg

    dcg Member

    62
    23
    8
    Oct 17, 2015
    Florida, USA
    Ratings:
    +47
    Local Time:
    5:26 PM
    1.21.x
    10.3.x
    Code:
    753
    158M    ./image_cache/
    
    But I use convertimage addon, so externally linked images are switched to attachments. I had an issue with 3rd party images going to 404, which destroys continuity of threads.

    So my attachments are:
    Code:
    456425
    253G    ./attachments/
    
     
  12. eva2000

    eva2000 Administrator Staff Member

    50,922
    11,805
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,263
    Local Time:
    8:26 AM
    Nginx 1.25.x
    MariaDB 10.x
    253GB :eek:
     
  13. dcg

    dcg Member

    62
    23
    8
    Oct 17, 2015
    Florida, USA
    Ratings:
    +47
    Local Time:
    5:26 PM
    1.21.x
    10.3.x
    Yup :D, content rich site. Lots of photos posted!!
     
  14. eva2000

    eva2000 Administrator Staff Member

    50,922
    11,805
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,263
    Local Time:
    8:26 AM
    Nginx 1.25.x
    MariaDB 10.x
    There's a reason why i restrict attachments to smaller max sizes - preventing attachment disk usage blow outs :)
     
  15. deltahf

    deltahf Premium Member Premium Member

    565
    251
    63
    Jun 8, 2014
    Ratings:
    +463
    Local Time:
    5:26 PM
    Image Cache: 612k+ files, 125GB
    Code (Text):
    total images: 612628 total size: 125855236 KB avg size/image: 205 KB
    


    Attachments: 631k+ files, 232GB

    I also have 32GB+ of images uploaded to WordPress /wp-content.
     
  16. Derek

    Derek Member

    47
    9
    8
    Aug 5, 2016
    Ratings:
    +22
    Local Time:
    5:26 PM
    Code:
    total images: 6587 total size: 656516 KB avg size/image: 99 KB
    
     
  17. eva2000

    eva2000 Administrator Staff Member

    50,922
    11,805
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,263
    Local Time:
    8:26 AM
    Nginx 1.25.x
    MariaDB 10.x

    Thanks for sharing :)

    Original poster updated their xenforo thread with this addon installed their proxy cache disk size dropped from 120GB to 77GB !
     
  18. rdan

    rdan Well-Known Member

    5,426
    1,389
    113
    May 25, 2014
    Ratings:
    +2,172
    Local Time:
    6:26 AM
    Mainline
    10.2
    Seen it, and that's almost 50% savings :love:.
    Addon is really worth on big boards.
     
  19. Silv3er

    Silv3er New Member

    9
    1
    3
    Feb 7, 2018
    Ratings:
    +8
    Local Time:
    5:26 PM
    Nginx 1.18.x
    MariaDB 10.x
    On my forum I use this atmos/camo (github use it) with a subdomain in full caching via cloudflare in one month [​IMG] :D