Discover Centmin Mod today
Register Now

Xenforo Xenforo users what's your image proxy cache size ?

Discussion in 'Forum software usage' started by eva2000, May 2, 2018.

  1. eva2000

    eva2000 Administrator Staff Member

    34,242
    7,576
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,645
    Local Time:
    10:07 AM
    Nginx 1.13.x
    MariaDB 5.5
    Came across this thread on Xenforo support forums regarding the size of image proxy cache What would be a reasonable server disk size for a big forum with SSL on. When Xenforo is used with HTTPS, you want to enable the image proxy so 3rd party references images in posts via IMG tag are proxied via Xenforo image proxy so there is no mixed content HTTPS errors. Xenforo image proxy caches these proxied images for a certain time which means increased disk usage before Xenforo image proxy cache TTL expires the cached entries.

    The thread linked to above shows how you can calculate your Xenforo image proxy cache size. So curious what other folks numbers look like. For this forum it's where domain.com is my domain name.
    Code (Text):
    cd /home/nginx/domains/domain.com/public/internal_data
    find /home/nginx/domains/domain.com/public/internal_data/image_cache/  | wc -l ;  du -hs ./image_cache/
    

    which comes to just 613 image proxy cached images taking up 68MB of disk space
    Code (Text):
    613
    68M     ./image_cache/
    

    Or with a bit of extra manipulation use
    Code (Text):
    cd /home/nginx/domains/domain.com/public/internal_data
    (find /home/nginx/domains/domain.com/public/internal_data/image_cache/  | wc -l ;  du -s ./image_cache/) 2>&1 | xargs | while read n s d; do echo "total images: $n total size: $s KB avg size/image: $(($s/$n)) KB"; done
    

    will return
    Code (Text):
    total images: 612 total size: 69240 KB avg size/image: 113 KB
    

     
  2. RoldanLT

    RoldanLT Premium Member Premium Member

    4,182
    1,013
    113
    May 25, 2014
    Phillipines
    Ratings:
    +1,435
    Local Time:
    8:07 AM
    1.11
    10.2
    Code:
    total images: 24626 total size: 2902756 KB avg size/image: 117 KB
     
    • Like Like x 1
  3. Jon Snow

    Jon Snow Active Member

    361
    58
    28
    Jun 30, 2017
    Ratings:
    +86
    Local Time:
    9:07 PM
    Nginx 1.13.9
    MariaDB 10.1.31
    Code:
    total images: 3031 total size: 773752 KB avg size/image: 255 KB
     
    • Like Like x 1
  4. Andy

    Andy Premium Member Premium Member

    391
    57
    28
    Aug 6, 2014
    Ratings:
    +68
    Local Time:
    8:07 PM
    Apparently, my site is not images heavy
    total images: 260 total size: 14076 KB avg size/image: 54 KB
     
    • Like Like x 1
  5. eva2000

    eva2000 Administrator Staff Member

    34,242
    7,576
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,645
    Local Time:
    10:07 AM
    Nginx 1.13.x
    MariaDB 5.5
    Interesting numbers guys.. thanks for sharing. Some of you have larger forums too ! I can't imagine how that linked Xenforo user had to triple their disk capacity to 150GB and still run out of disk space due to Xenforo image proxy local disk caching ! That's some heavy 3rd party image posting activity !
     
  6. Sunka

    Sunka Well-Known Member

    983
    263
    63
    Oct 31, 2015
    Rijeka, Croatia
    Ratings:
    +434
    Local Time:
    2:07 AM
    Nginx 1.13.3
    MariaDB 10.1.24
    Code (Text):
    total images: 1242 total size: 66928 KB avg size/image: 53 KB
     
    • Like Like x 1
  7. RoldanLT

    RoldanLT Premium Member Premium Member

    4,182
    1,013
    113
    May 25, 2014
    Phillipines
    Ratings:
    +1,435
    Local Time:
    8:07 AM
    1.11
    10.2
    Mine is even lower now.
    Because I only cached 2mb max images and Ignore https images.

    upload_2018-5-3_2-28-55.png
     
    • Like Like x 1
  8. eva2000

    eva2000 Administrator Staff Member

    34,242
    7,576
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,645
    Local Time:
    10:07 AM
    Nginx 1.13.x
    MariaDB 5.5
    @RoldanLT you using an XF Addon for ignore https images ?
     
  9. RoldanLT

    RoldanLT Premium Member Premium Member

    4,182
    1,013
    113
    May 25, 2014
    Phillipines
    Ratings:
    +1,435
    Local Time:
    8:07 AM
    1.11
    10.2
    • Informative Informative x 1
  10. eva2000

    eva2000 Administrator Staff Member

    34,242
    7,576
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,645
    Local Time:
    10:07 AM
    Nginx 1.13.x
    MariaDB 5.5
    Ah looked familiar :)
     
  11. dcg

    dcg Premium Member Premium Member

    48
    17
    8
    Oct 17, 2015
    Florida, USA
    Ratings:
    +32
    Local Time:
    8:07 PM
    1.13.5
    10.1.28
    Code:
    753
    158M    ./image_cache/
    
    But I use convertimage addon, so externally linked images are switched to attachments. I had an issue with 3rd party images going to 404, which destroys continuity of threads.

    So my attachments are:
    Code:
    456425
    253G    ./attachments/
    
     
    • Informative Informative x 1
  12. eva2000

    eva2000 Administrator Staff Member

    34,242
    7,576
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,645
    Local Time:
    10:07 AM
    Nginx 1.13.x
    MariaDB 5.5
    253GB :eek:
     
    • Funny Funny x 1
  13. dcg

    dcg Premium Member Premium Member

    48
    17
    8
    Oct 17, 2015
    Florida, USA
    Ratings:
    +32
    Local Time:
    8:07 PM
    1.13.5
    10.1.28
    Yup :D, content rich site. Lots of photos posted!!
     
  14. eva2000

    eva2000 Administrator Staff Member

    34,242
    7,576
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,645
    Local Time:
    10:07 AM
    Nginx 1.13.x
    MariaDB 5.5
    There's a reason why i restrict attachments to smaller max sizes - preventing attachment disk usage blow outs :)
     
  15. deltahf

    deltahf Premium Member Premium Member

    242
    113
    43
    Jun 8, 2014
    Ratings:
    +179
    Local Time:
    8:07 PM
    Image Cache: 612k+ files, 125GB
    Code (Text):
    total images: 612628 total size: 125855236 KB avg size/image: 205 KB
    


    Attachments: 631k+ files, 232GB

    I also have 32GB+ of images uploaded to WordPress /wp-content.
     
  16. Derek

    Derek Member

    43
    9
    8
    Aug 5, 2016
    Ratings:
    +21
    Local Time:
    8:07 PM
    Code:
    total images: 6587 total size: 656516 KB avg size/image: 99 KB
    
     
    • Informative Informative x 1
  17. eva2000

    eva2000 Administrator Staff Member

    34,242
    7,576
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,645
    Local Time:
    10:07 AM
    Nginx 1.13.x
    MariaDB 5.5

    Thanks for sharing :)

    Original poster updated their xenforo thread with this addon installed their proxy cache disk size dropped from 120GB to 77GB !
     
    • Like Like x 2
  18. RoldanLT

    RoldanLT Premium Member Premium Member

    4,182
    1,013
    113
    May 25, 2014
    Phillipines
    Ratings:
    +1,435
    Local Time:
    8:07 AM
    1.11
    10.2
    Seen it, and that's almost 50% savings :love:.
    Addon is really worth on big boards.
     
    • Agree Agree x 2
  19. Silv3er

    Silv3er New Member

    7
    1
    3
    Feb 7, 2018
    Ratings:
    +8
    Local Time:
    8:07 PM
    Nginx 1.13.x
    MariaDB 10.x
    On my forum I use this atmos/camo (github use it) with a subdomain in full caching via cloudflare in one month [​IMG] :D
     
    • Informative Informative x 1
..