/ Register

Learn about Centmin Mod LEMP Stack today
Become a Member

Images on cloud/cdn ?

Discussion in 'System Administration' started by pamamolf, Sep 8, 2015.

Tags:
Previous Thread Next Thread
Loading...
Page 1 of 3
  1. Sep 8, 2015 #1
    pamamolf

    pamamolf Premium Member Premium Member

    4,086
    428
    83
    May 31, 2014
    Ratings:
    +834
    Local Time:
    9:07 AM
    Nginx-1.25.x
    MariaDB 10.3.x
    Hi

    Is the best solution for performance most to move all images to a cloud or CDN when there is no disk space and can't get bigger ssd's disks?

    Can you please recommend me a nice and fast cloud or cdn service?

    Thank you
     
  2. Sep 8, 2015 #2
    eva2000

    eva2000 Administrator Staff Member

    54,892
    12,240
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,811
    Local Time:
    5:07 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Depends 2 types of CDN methods - pull based or push based. Most CDNs are predominantly pull based, so you need the origin - local server to serve the original image/static files anyway. So that doesn't suit your needs if your concern is running out of local disk space.

    The push based CDN is where you upload images/static files to the CDN cloud storage i.e for Amazon Cloudfront you'd upload images to an Amazon S3 bucket and setup Amazon Cloudfront to point to that S3 bucket which acts as the origin. Still you'd need to have the original source images and static files at one time or another to be locally stored before being uploaded to cloud /CDN storage like an S3 bucket. So either way you cut it, you need ample disk space on local server especially if you take into account growing disk log storage and the fact you need reserve disk free space for backups/restores of your data.

    Both @RoldanLT and myself have used KeyCDN but only used for pull based although support push based CDN usage as well. However, my experience using it on this forum has been mixed.. always get a slight but very noticeable lag on initial page loads/navigation compared to non-CDN. Then there's Amazon Cloudfront haven't used it this forum, but I do use it on my Wordpress 7 test site :) KeyCDN easier to setup your own SSL certificate for https based CDN usage. Amazon Cloudfront is alot tricker for SSL CDN usage as they don't have a web interface for it they use their AWS CLI command line tools AFAIK so haven't tried it. There's also MaxCDN I use, expensive for SSL https based CDN usage at US$250/month for std SSL and US$600/month for wildcard SSL and most expensive with monthly contracts but performance probably the best or on par with Amazon Cloudfront for non-https at least.

    Also I use CDN77 as well :)

    I'm also developing my own private SPDY/HTTP/2 based CDN/proxy for centminmod.com usage as well using Centmin Mod LEMP stack's Nginx to power it so I have full control :) CentminMod.com deployment in geodns cluster is the first part of it :)
     
    Last edited: Sep 8, 2015
  3. Sep 8, 2015 #3
    rdan

    rdan Well-Known Member

    5,447
    1,408
    113
    May 25, 2014
    Ratings:
    +2,201
    Local Time:
    3:07 PM
    Mainline
    10.2
    If your script/web app is capable of uploading your static files on different server, then that would be easy.
    I mean attachment uploaded by users, avatars.
     
  4. Sep 8, 2015 #4
    pamamolf

    pamamolf Premium Member Premium Member

    4,086
    428
    83
    May 31, 2014
    Ratings:
    +834
    Local Time:
    9:07 AM
    Nginx-1.25.x
    MariaDB 10.3.x
    Yes my script is able to use direct cdn uploads from users so when they upload an avatar it can store straight to the cdn.....

    At the moment i do not use https but i will next year so it must be easy to configure it on CDN later....

    So a push based cdn is what i need if i read correct your info :)

    I will try the Amazon Cloudfront and the KeyCDN :)
     
  5. Sep 8, 2015 #5
    rdan

    rdan Well-Known Member

    5,447
    1,408
    113
    May 25, 2014
    Ratings:
    +2,201
    Local Time:
    3:07 PM
    Mainline
    10.2
    For cheaper way, you can use separate VPS on SSD with only Nginx on it.
    yum install nginx -y
     
  6. Sep 8, 2015 #6
    pamamolf

    pamamolf Premium Member Premium Member

    4,086
    428
    83
    May 31, 2014
    Ratings:
    +834
    Local Time:
    9:07 AM
    Nginx-1.25.x
    MariaDB 10.3.x
    Creating my own cdn?

    Don't know how to configure it :(
     
  7. Sep 8, 2015 #7
    rdan

    rdan Well-Known Member

    5,447
    1,408
    113
    May 25, 2014
    Ratings:
    +2,201
    Local Time:
    3:07 PM
    Mainline
    10.2
    Not exactly your own CDN, but plain server only use for serving static files.
     
  8. Sep 8, 2015 #8
    pamamolf

    pamamolf Premium Member Premium Member

    4,086
    428
    83
    May 31, 2014
    Ratings:
    +834
    Local Time:
    9:07 AM
    Nginx-1.25.x
    MariaDB 10.3.x
    But maybe i can set it up from my script as i can do it with the above cdn's by just changing the path for remote images :)
     
  9. Sep 8, 2015 #9
    rdan

    rdan Well-Known Member

    5,447
    1,408
    113
    May 25, 2014
    Ratings:
    +2,201
    Local Time:
    3:07 PM
    Mainline
    10.2
    But costly.
     
  10. Sep 8, 2015 #10
    pamamolf

    pamamolf Premium Member Premium Member

    4,086
    428
    83
    May 31, 2014
    Ratings:
    +834
    Local Time:
    9:07 AM
    Nginx-1.25.x
    MariaDB 10.3.x
    George do you use for your cluster MariaDB galera cluster and GlusterFS ?
     
  11. Sep 8, 2015 #11
    eva2000

    eva2000 Administrator Staff Member

    54,892
    12,240
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,811
    Local Time:
    5:07 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    GlusterFS is only one I have experience with out of the 2. For centminmod.com it's just a Git backed repository which allows easier update/syncing for files. For CDN it will be pull/proxy based so no syncing needed.
     
  12. Sep 8, 2015 #12
    pamamolf

    pamamolf Premium Member Premium Member

    4,086
    428
    83
    May 31, 2014
    Ratings:
    +834
    Local Time:
    9:07 AM
    Nginx-1.25.x
    MariaDB 10.3.x
    Not so clear answer on what you use for your cluster for syncing files and database clustering :)
     
  13. Sep 8, 2015 #13
    rdan

    rdan Well-Known Member

    5,447
    1,408
    113
    May 25, 2014
    Ratings:
    +2,201
    Local Time:
    3:07 PM
    Mainline
    10.2
    WoW, so I just know some technology today:D
    That GIT can be use for Cluster Server :D
    Didn't think of that is possible :D
    Thanks Eva!
     
  14. Sep 8, 2015 #14
    eva2000

    eva2000 Administrator Staff Member

    54,892
    12,240
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,811
    Local Time:
    5:07 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    GlusterFS/NFS and also csync2 and lsyncd and MySQL replication or Tungsten Replicator Replication
    Well it's easy for files only which aren't user generated like centminmod.com - use gitlab.com and not github.com has they have large file size support and gitlab.com offers free private and public git repos GitLab.com | GitLab
     
    Last edited: Sep 8, 2015
  15. Sep 8, 2015 #15
    pamamolf

    pamamolf Premium Member Premium Member

    4,086
    428
    83
    May 31, 2014
    Ratings:
    +834
    Local Time:
    9:07 AM
    Nginx-1.25.x
    MariaDB 10.3.x
    Add the power to Centminmod lol :)

    Is it very important to use ssd's on the remote images server as they will be limited by the Network speed transfer?
     
  16. Sep 8, 2015 #16
    eva2000

    eva2000 Administrator Staff Member

    54,892
    12,240
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,811
    Local Time:
    5:07 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    SSDs help in terms of disk latency access times. But you'd be limited by both network speed and number of cpu cores/threads you have available if you're hosting it yourself for CDN. Reason why I am looking into 10Gbps network connections too :D
     
  17. Sep 8, 2015 #17
    pamamolf

    pamamolf Premium Member Premium Member

    4,086
    428
    83
    May 31, 2014
    Ratings:
    +834
    Local Time:
    9:07 AM
    Nginx-1.25.x
    MariaDB 10.3.x
    I am thinking the scenario that RoldanLT recommend me to get a dedicated server and just host there the images....

    Ssd's are more expensive and i am not so sure if they really help for such scenario....
     
  18. Sep 8, 2015 #18
    eva2000

    eva2000 Administrator Staff Member

    54,892
    12,240
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,811
    Local Time:
    5:07 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Yeah or just get a better server :)
     
  19. Sep 8, 2015 #19
    pamamolf

    pamamolf Premium Member Premium Member

    4,086
    428
    83
    May 31, 2014
    Ratings:
    +834
    Local Time:
    9:07 AM
    Nginx-1.25.x
    MariaDB 10.3.x
    Getting a better server and pay more than 100 euro extra just for bigger ssd's as now the server is more than enough maybe is not the best solution.... :)
     
  20. Sep 8, 2015 #20
    rdan

    rdan Well-Known Member

    5,447
    1,408
    113
    May 25, 2014
    Ratings:
    +2,201
    Local Time:
    3:07 PM
    Mainline
    10.2
Previous Thread Next Thread
Loading...
Page 1 of 3

.