Join the community today
Become a Member

Xenforo Xenforo users what's your image proxy cache size ?

Discussion in 'Forum software usage' started by eva2000, May 2, 2018.

  1. eva2000

    eva2000 Administrator Staff Member

    36,408
    7,992
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,306
    Local Time:
    7:00 AM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    Came across this thread on Xenforo support forums regarding the size of image proxy cache What would be a reasonable server disk size for a big forum with SSL on. When Xenforo is used with HTTPS, you want to enable the image proxy so 3rd party references images in posts via IMG tag are proxied via Xenforo image proxy so there is no mixed content HTTPS errors. Xenforo image proxy caches these proxied images for a certain time which means increased disk usage before Xenforo image proxy cache TTL expires the cached entries.

    The thread linked to above shows how you can calculate your Xenforo image proxy cache size. So curious what other folks numbers look like. For this forum it's where domain.com is my domain name.
    Code (Text):
    cd /home/nginx/domains/domain.com/public/internal_data
    find /home/nginx/domains/domain.com/public/internal_data/image_cache/  | wc -l ;  du -hs ./image_cache/
    

    which comes to just 613 image proxy cached images taking up 68MB of disk space
    Code (Text):
    613
    68M     ./image_cache/
    

    Or with a bit of extra manipulation use
    Code (Text):
    cd /home/nginx/domains/domain.com/public/internal_data
    (find /home/nginx/domains/domain.com/public/internal_data/image_cache/  | wc -l ;  du -s ./image_cache/) 2>&1 | xargs | while read n s d; do echo "total images: $n total size: $s KB avg size/image: $(($s/$n)) KB"; done
    

    will return
    Code (Text):
    total images: 612 total size: 69240 KB avg size/image: 113 KB
    
     
  2. rdan

    rdan Premium Member Premium Member

    4,308
    1,044
    113
    May 25, 2014
    Ratings:
    +1,504
    Local Time:
    5:00 AM
    Mainline
    10.2
    Code:
    total images: 24626 total size: 2902756 KB avg size/image: 117 KB
     
    • Like Like x 1
  3. Jon Snow

    Jon Snow Active Member

    377
    60
    28
    Jun 30, 2017
    Ratings:
    +91
    Local Time:
    6:00 PM
    Nginx 1.13.9
    MariaDB 10.1.31
    Code:
    total images: 3031 total size: 773752 KB avg size/image: 255 KB
     
    • Like Like x 1
  4. Andy

    Andy Premium Member Premium Member

    420
    60
    28
    Aug 6, 2014
    Ratings:
    +84
    Local Time:
    5:00 PM
    Apparently, my site is not images heavy
    total images: 260 total size: 14076 KB avg size/image: 54 KB
     
    • Like Like x 1
  5. eva2000

    eva2000 Administrator Staff Member

    36,408
    7,992
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,306
    Local Time:
    7:00 AM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    Interesting numbers guys.. thanks for sharing. Some of you have larger forums too ! I can't imagine how that linked Xenforo user had to triple their disk capacity to 150GB and still run out of disk space due to Xenforo image proxy local disk caching ! That's some heavy 3rd party image posting activity !
     
  6. Sunka

    Sunka Well-Known Member

    1,021
    281
    83
    Oct 31, 2015
    Rijeka, Croatia
    Ratings:
    +459
    Local Time:
    11:00 PM
    Nginx 1.15.0
    MariaDB 10.2.15
    Code (Text):
    total images: 1242 total size: 66928 KB avg size/image: 53 KB
     
    • Like Like x 1
  7. rdan

    rdan Premium Member Premium Member

    4,308
    1,044
    113
    May 25, 2014
    Ratings:
    +1,504
    Local Time:
    5:00 AM
    Mainline
    10.2
    Mine is even lower now.
    Because I only cached 2mb max images and Ignore https images.

    upload_2018-5-3_2-28-55.png
     
    • Like Like x 1
  8. eva2000

    eva2000 Administrator Staff Member

    36,408
    7,992
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,306
    Local Time:
    7:00 AM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    @RoldanLT you using an XF Addon for ignore https images ?
     
  9. rdan

    rdan Premium Member Premium Member

    4,308
    1,044
    113
    May 25, 2014
    Ratings:
    +1,504
    Local Time:
    5:00 AM
    Mainline
    10.2
    • Informative Informative x 1
  10. eva2000

    eva2000 Administrator Staff Member

    36,408
    7,992
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,306
    Local Time:
    7:00 AM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    Ah looked familiar :)
     
  11. dcg

    dcg Member

    53
    18
    8
    Oct 17, 2015
    Florida, USA
    Ratings:
    +34
    Local Time:
    5:00 PM
    1.15.x
    10.2.x
    Code:
    753
    158M    ./image_cache/
    
    But I use convertimage addon, so externally linked images are switched to attachments. I had an issue with 3rd party images going to 404, which destroys continuity of threads.

    So my attachments are:
    Code:
    456425
    253G    ./attachments/
    
     
    • Informative Informative x 1
  12. eva2000

    eva2000 Administrator Staff Member

    36,408
    7,992
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,306
    Local Time:
    7:00 AM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    253GB :eek:
     
    • Funny Funny x 1
  13. dcg

    dcg Member

    53
    18
    8
    Oct 17, 2015
    Florida, USA
    Ratings:
    +34
    Local Time:
    5:00 PM
    1.15.x
    10.2.x
    Yup :D, content rich site. Lots of photos posted!!
     
  14. eva2000

    eva2000 Administrator Staff Member

    36,408
    7,992
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,306
    Local Time:
    7:00 AM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    There's a reason why i restrict attachments to smaller max sizes - preventing attachment disk usage blow outs :)
     
  15. deltahf

    deltahf Premium Member Premium Member

    293
    130
    43
    Jun 8, 2014
    Ratings:
    +209
    Local Time:
    5:00 PM
    Image Cache: 612k+ files, 125GB
    Code (Text):
    total images: 612628 total size: 125855236 KB avg size/image: 205 KB
    


    Attachments: 631k+ files, 232GB

    I also have 32GB+ of images uploaded to WordPress /wp-content.
     
  16. Derek

    Derek Member

    43
    9
    8
    Aug 5, 2016
    Ratings:
    +21
    Local Time:
    5:00 PM
    Code:
    total images: 6587 total size: 656516 KB avg size/image: 99 KB
    
     
    • Informative Informative x 1
  17. eva2000

    eva2000 Administrator Staff Member

    36,408
    7,992
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +12,306
    Local Time:
    7:00 AM
    Nginx 1.15.x
    MariaDB 5.5/10.x

    Thanks for sharing :)

    Original poster updated their xenforo thread with this addon installed their proxy cache disk size dropped from 120GB to 77GB !
     
    • Like Like x 2
  18. rdan

    rdan Premium Member Premium Member

    4,308
    1,044
    113
    May 25, 2014
    Ratings:
    +1,504
    Local Time:
    5:00 AM
    Mainline
    10.2
    Seen it, and that's almost 50% savings :love:.
    Addon is really worth on big boards.
     
    • Agree Agree x 2
  19. Silv3er

    Silv3er New Member

    9
    1
    3
    Feb 7, 2018
    Ratings:
    +8
    Local Time:
    5:00 PM
    Nginx 1.13.x
    MariaDB 10.x
    On my forum I use this atmos/camo (github use it) with a subdomain in full caching via cloudflare in one month [​IMG] :D
     
    • Informative Informative x 1
..