/ Register

Learn about Centmin Mod LEMP Stack today
Register Now

Sysadmin Help with a solution for video streaming :(

Discussion in 'System Administration' started by KinderOvO, Sep 1, 2018.

Previous Thread Next Thread
Loading...
Page 1 of 3
  1. Sep 1, 2018 #1
    KinderOvO

    KinderOvO Member

    92
    1
    8
    Jan 13, 2015
    Ratings:
    +1
    Local Time:
    4:12 PM
    1.4
    10
    Hello I have clients in which I provide server for streaming videos .mp4

    I was using "bunnycdn" but some configuration was limited to only 2gbps usage and 20tb day. this caused many problems of slow videos in the videos, for some it was fast and for others slow.

    - Then I migrated to datapacket, 5gbps plan.
    And it is consuming 5gbps full and I already received an email offering upgrade to 10gbps.

    In other words, it is already totally out of my budget.

    I have always used ovh, they have option of 3gbps upgrade, but the problem is that if I hire 3 servers, I have to do rsync of the videos on the 3 servers, they are 500gb mp4 and increasing every day. - I have to use ssd disks, because the read rate currently using iotop "265 mb/s" ie I can not use ordinary disks. Is there a strategy for using 3 x 3gbps, but the files on only 1 server? this would save a lot and would be a good solution, aiming that the price of the ovh would be better. I'm using the default centminmod config for videos .mp4
     
  2. Sep 1, 2018 #2
    eva2000

    eva2000 Administrator Staff Member

    53,142
    12,108
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,643
    Local Time:
    4:12 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Have you tried contacting Cloudflare for their Cloudflare Stream service Stream | Cloudflare ?
     
  3. Sep 1, 2018 #3
    KinderOvO

    KinderOvO Member

    92
    1
    8
    Jan 13, 2015
    Ratings:
    +1
    Local Time:
    4:12 PM
    1.4
    10
    So expensive 1, per 1 thousand minute.
     
  4. Sep 1, 2018 #4
    eva2000

    eva2000 Administrator Staff Member

    53,142
    12,108
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,643
    Local Time:
    4:12 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Do some small A/B testing of some videos served via Cloudflare Stream and see how it goes and pick CF support's brains ? i.e. select a few videos to serve over CF Stream

    Just rsync gigabytes of videos each day, is probably eating into your 3Gbps network pipe itself ? Or using OVH VPC and separate private network ?

    how much you paying at datapacket for 5Gbps servers ?
     
  5. Sep 1, 2018 #5
    eva2000

    eva2000 Administrator Staff Member

    53,142
    12,108
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,643
    Local Time:
    4:12 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
  6. Sep 1, 2018 #6
    KinderOvO

    KinderOvO Member

    92
    1
    8
    Jan 13, 2015
    Ratings:
    +1
    Local Time:
    4:12 PM
    1.4
    10
    $1250
     
  7. Sep 1, 2018 #7
    KinderOvO

    KinderOvO Member

    92
    1
    8
    Jan 13, 2015
    Ratings:
    +1
    Local Time:
    4:12 PM
    1.4
    10
    Thanks everyone for trying to help, I'm going to call on Monday because the support in "pt" is back next week.


    I would like to understand better.
    - If I have example "1 server with 3gbps"
    - he is using 100% the whole month.
    - being thus 3gbps = 3x330 = 990TB per month

    but the daily consumption is 20tb, totaling 30x20TB = 600TB per month

    So here comes my doubt, what happens to my visitors, when I'm using 3gbps and I have no more link available?

    the videos slow down? or does my 20tb day consumption increase until I reach the monthly limit of 990tb?
     
  8. Sep 1, 2018 #8
    buik

    buik “The best traveler is one without a camera.”

    1,990
    518
    113
    Apr 29, 2016
    Flanders
    Ratings:
    +1,647
    Local Time:
    8:12 AM
    Constantly 5gbps and 20+ TB a day is starting to become serious.
    You are no longer a small player.

    I have no knowledge of streaming at all but with these volumes it requires a customized solution. Enlist an stream-expert is my advice.

    About your budget, although I do not know the numbers, it seems like your customer is underpaying you. Streaming services seems to be cheap but are expensive as h*** as you start to notice now.

    If the customer in question grows, the budget for you also has to grow, otherwise you get stuck or you have to adjust yourself. Which of course is not the intention.
     
  9. Sep 1, 2018 #9
    eva2000

    eva2000 Administrator Staff Member

    53,142
    12,108
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,643
    Local Time:
    4:12 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    the way I see it you need a solution that will scale as what happens if you grow to 10Gbps, 15Gbps, 20Gbps, 30Gbps network bandwidth requirements ? Your server hosting costs will continue to rise and you need to figure out whether and/or at what point maybe Cloudflare Stream like service may make more sense ? Provided you work out revenue side of things as hosting videos.

    Indeed not much practical knowledge for streaming videos. But might as well try dabbling myself. So added to 123.09beta01 Nginx, optional support for Nginx Slice module Beta Branch - add optional support for Nginx Slice module in 123.09beta01 so can play with HTML5 video streaming and Nginx proxy cache and byte-range caching.
    @KinderOvO how are you serving videos via HTML5 video streaming ? video formats ? As you could try optimising video sizes so to reduce streaming bandwidth ?
     
  10. Sep 1, 2018 #10
    buik

    buik “The best traveler is one without a camera.”

    1,990
    518
    113
    Apr 29, 2016
    Flanders
    Ratings:
    +1,647
    Local Time:
    8:12 AM
    On 5GBPS, 20TB per day and counting.
    Not that it concerns and no offensive, but it makes me curious.
    Are you offering services to Youtube porn like sites?
     
  11. Sep 1, 2018 #11
    eva2000

    eva2000 Administrator Staff Member

    53,142
    12,108
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,643
    Local Time:
    4:12 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Well not the first time.. as I have found out lots of porn/adult sites are using Centmin Mod LEMP stack these days :D
     
  12. Sep 1, 2018 #12
    KinderOvO

    KinderOvO Member

    92
    1
    8
    Jan 13, 2015
    Ratings:
    +1
    Local Time:
    4:12 PM
    1.4
    10
    Yes adult sites, I know webmasters who have 10 times more visits than I do every day about 2 million, and traffic and half of my current, I can not understand.
     
  13. Sep 1, 2018 #13
    buik

    buik “The best traveler is one without a camera.”

    1,990
    518
    113
    Apr 29, 2016
    Flanders
    Ratings:
    +1,647
    Local Time:
    8:12 AM
    I do understand. Streaming is a complex market with stream compression, encoding etc etc to reduce bandwidth. As bandwidth is expensive like ...
    Reduce bandwidth and so costs is task nummero uno and there is a lot possible.

    Therefore once again enlist an stream-expert is my advice.
    It will hurt the costs for a while, but you will earn it later.

    In addition, pay attention to additional services that you are going to rent.
    Not all companies accept adult.
     
  14. Sep 2, 2018 #14
    eva2000

    eva2000 Administrator Staff Member

    53,142
    12,108
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,643
    Local Time:
    4:12 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    indeed @KinderOvO what video formats you serving ? details of how you have everything setup ? if you switched to prefer webm before mp4, you'd get much smaller video sizes

    Did some sliced byte-range caching tests on Centmin Mod Nginx with some old Centmin Mod videos - there's virtually no disk I/O load once videos' byte-range segments are cached by Nginx as videos are then served out of Nginx cache and not via the disk :)

    look at mp4 vs webm sizes
    Code (Text):
    -rw-r--r-- 1 root  nginx  35K Sep  1 08:18 cmm-add-nginx-vhost.jpg
-rw-r--r-- 1 root  nginx  34M Sep  1 05:32 cmm-add-nginx-vhost.mp4
-rw-r--r-- 1 root  nginx  22M Sep  1 07:57 cmm-add-nginx-vhost.webm
-rw-r--r-- 1 root  nginx  35K Sep  1 08:18 cmm-betainstall.jpg
-rw-r--r-- 1 root  nginx  38M Sep  1 05:28 cmm-betainstall.mp4
-rw-r--r-- 1 root  nginx  19M Sep  1 07:54 cmm-betainstall.webm
-rw-r--r-- 1 root  nginx  32K Sep  1 08:18 cmm-centmin.sh-menu.jpg
-rw-r--r-- 1 root  nginx 9.3M Sep  1 05:32 cmm-centmin.sh-menu.mp4
-rw-r--r-- 1 root  nginx 6.9M Sep  1 07:44 cmm-centmin.sh-menu.webm
     
  15. Sep 3, 2018 #15
    eva2000

    eva2000 Administrator Staff Member

    53,142
    12,108
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,643
    Local Time:
    4:12 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
  16. Sep 3, 2018 #16
    JJC84

    JJC84 Ad astra per aspera

    247
    109
    43
    Jan 31, 2018
    San Antonio, Texas
    Ratings:
    +169
    Local Time:
    1:12 AM
    1.15.x
    10.x.x
    my bad!
     
  17. Sep 3, 2018 #17
    eva2000

    eva2000 Administrator Staff Member

    53,142
    12,108
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,643
    Local Time:
    4:12 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Last edited: Sep 3, 2018
  18. Sep 3, 2018 #18
    eva2000

    eva2000 Administrator Staff Member

    53,142
    12,108
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,643
    Local Time:
    4:12 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Doing my research of video streaming led me to these video at At Scale 2017 meet that Facebook hosted - how video is encoded matters i.e. VP9 webm vs H.264. There's also consideration for new AV1 codec This technology will improve your streaming video by 30 percent
    https://www.facebook.com/video.php?v=1875063729433389


    https://www.facebook.com/video.php?v=1875054462767649
     
  19. Sep 3, 2018 #19
    gabel

    gabel Member

    68
    38
    18
    Jan 7, 2016
    Ratings:
    +64
    Local Time:
    7:12 AM
    1.9.9
    MariaDB 10
    @eva2000 yup i`m using kvs and i burn quite a bit of bandwidth , but still not as much as @KinderOvO does (close though). I have a few different servers from where i serve the content so the bandwidth consumption is split between all of them. Depending on what script he is using , he could try to limit the bitrate to try and save some bandwidth.

    @KinderOvO i found that using a cdn for video streaming gets expensive once you start to have serious traffic. What cms are you using?
     
  20. Sep 4, 2018 #20
    eva2000

    eva2000 Administrator Staff Member

    53,142
    12,108
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,643
    Local Time:
    4:12 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Indeed.. rate and/or connection limiting might help. So limit to 3-5 connections per IP and have a rate limit which maybe dynamically and gradually gets stricter as there are more concurrent connections from the same IP. So to prevent individual users from going over board when downloading concurrent streams ?
     
Previous Thread Next Thread
Loading...
Page 1 of 3

.