Welcome to Centmin Mod Community
Become a Member

DigitalOcean DO managed database -worth migrating to?

Discussion in 'Virtual Private Server (VPS) hosting' started by Colin, Sep 12, 2020.

  1. Colin

    Colin Premium Member Premium Member

    188
    57
    28
    Oct 7, 2015
    Sheffield UK
    Ratings:
    +149
    Local Time:
    3:03 AM
    1.19.#
    MariaDB 10.1.#
    Anyone using Digital Oceans managed SQL? How is it under real load?


    I have a test cluster up for a test site, it works...

    Right now I'm still doing the monolith all on one server and just upsizing as it gets tight...
    presently on a linode dedicated 32Gb ram with zero complaints. Everything is just about fitting in, but today triggered a 80% alarm on ram... traffic is picking up again.

    I switched from dynamic to static when I moved to the dedicated cpu, running 120 php procs. I should probabbly if I'm reading my queue length right need more.

    Anyhow, I've been toying with moving the db off. Although you get a higher chance of downtime with two nodes, that would ( touch wood ) still be quite low and bearable for a forum site.

    So I was toying with moving to DO for the managed mysql. Partly so I don't have to worry about it; sod the cost. Partly also for the 'real' private network, although linode is meant to be bringing that this year.

    However my mysql instance needs around 14gb or ram for the innodb tables/index, which is right on the max DO managed db size, unless I'm misunderstanding modern mysql innodb sizing. I also can't see anyway to 'tune' innodb sizing, buffer size etc. Do I trust they look in on this... I could possibly reduce this memory impact, if I get search offloaded to elastic. So 3 nodes ;)

    I know it's a heck of a price, but I'm not of the mind to do a mysql cluster. Because If I did that I'd be daft not to do a php a:b pair too. But thats overkil for what it is...

    Thoughts?

    ps I'm also a bit wary of DO's ddos reactions of null routing, but that might be bad internet blood.
     
  2. eva2000

    eva2000 Administrator Staff Member

    50,930
    11,809
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,268
    Local Time:
    1:03 PM
    Nginx 1.25.x
    MariaDB 10.x
    No idea on DO's managed database service. But if it's like their DO droplets, then performance might not be that good relative to other VPS providers. DO droplets for standard and dedicated cpu both consistently ranked at the bottom of comparison benchmarks I did at DigitalOcean - Linode - Vultr - Benchmarks - Upcloud - 13-Way VPS Server Benchmark Comparison Tests - Upcloud vs DigitalOcean vs Linode vs Vultr vs Hetzner.

    Best thing is to see what/if any caching can offload load from PHP itself. Some Centmin Mod commands might help dig deeper into system usage metrics via cminfo command
    Code (Text):
    cminfo top
    cminfo sar-cpu
    cminfo sar-mem
    cminfo phpstats
    cminfo phpmem
    

    up to you to interpret the output though.

    If you can use Elasticsearch to offload search related database size/buffer usage, then do it even if Elasticsearch is on same server it's not 1:1 for database size versus memory usage.
     
  3. Colin

    Colin Premium Member Premium Member

    188
    57
    28
    Oct 7, 2015
    Sheffield UK
    Ratings:
    +149
    Local Time:
    3:03 AM
    1.19.#
    MariaDB 10.1.#
    more commands and data to parse :)

    I'm now hovering around 75% of ram so won't push much more on to it.
    Code:
    Every 2.0s: ...  Wed Sep 16 14:50:08 2020
    
    Processes active: 18, idle: 82, Requests:
     2770091, slow: 0, Traffic: 5.9req/sec
    pool:                 www
    process manager:      static
    start time:           09/Sep/2020:22:36:2
    7 +0000
    start since:          576821
    accepted conn:        2770138
    listen queue:         0
    max listen queue:     51
    listen queue len:     511
    idle processes:       84
    active processes:     16
    total processes:      100
    max active processes: 207
    max children reached: 0
    slow requests:        0
    It's quite quiet at the moment. Kind of wish there was a way to capture the peak, nixstats quite handy for that in a way.

    I've used them, looked at the numbers, then went for a butt dyno.

    Setting up on upcloud. Would of gone vultr; possibly, but supply is nonexisteant and thats a concern if I wanted to experiment and to consolidate.

    There are more costs in upcloud $ for $, e.g. for a private network, they have a private utility for all vps's on account, to add another it's $5. A firewall is an addiitonal 5$, backups are a bit steeper, but choices like weekly, monthly or year retentions. There is also a scary zero balance termination bit in the terms. Hope the credit card never fails ;)

    So to the question of is a managed db worth it... Given I'd be almost doubling my hosting cost, just to bump the database to a standalone 'manged', maybe. It would be closer tripple with failover, which is what I was attracted to, too save setting/managing it. Thats quite a sum and certianly a pause for thought, it would easiy cover the cost of a 'on call' sys admin.

    Although this is my first multibox setup in a very long time; I've been spoilt by the likes of heroku and vertical box scaling... first under centmin too, so it's raised questions for me about csf config :) That and reminded me I'm not a sysadmin; still a dangerous one, a rookie; I can break things and not always know how to fix them :)

    Code:
    #csf.allow
    tcp|in|d=3306|s=10.x.x.x
    
    I was looking for a way to; for the db nodes, ignore all from eth0; public, allow from eth1; private, then allow port 3306 on eth1 etc rather than each ip on it's own... although being explicit isn't a problem.

    But then I see all the allow from cloudflare which will blow past any port restrictions from their ips.

    My TCP_IN rule for permit is empty on both web and sql sets of nodes. So kinf oa TCP_IN for each eth really, but can't see anything in the csf docs that leaps out.

    Fun days :)
     
  4. eva2000

    eva2000 Administrator Staff Member

    50,930
    11,809
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,268
    Local Time:
    1:03 PM
    Nginx 1.25.x
    MariaDB 10.x
    Just setup command to run via a cronjob - you can do it per second logging to a log file. Then you can use Linux grep/awk cmds to filter and inspect the logs and dig much deeper into specifics than charting would.

    FYI, recently added cminfo sar-json command too if you want to manipulate sar stats in json format Upgrade - Nginx - Insight Guide - cminfo command explained
    Upcloud operates on a credit added basis instead of credit card charge basis AFAIK.
    It shouldn't if db connection is via private plan and db server IP address / port isn't listening to a public IP but private IP.