Want more timely Centmin Mod News Updates?
Become a Member

DigitalOcean DO managed database -worth migrating to?

Discussion in 'Virtual Private Server (VPS) hosting' started by Colin, Sep 12, 2020.

  1. Colin

    Colin Premium Member Premium Member

    191
    59
    28
    Oct 7, 2015
    Sheffield UK
    Ratings:
    +153
    Local Time:
    1:00 AM
    1.19.#
    MariaDB 10.1.#
    Anyone using Digital Oceans managed SQL? How is it under real load?

    I have a test cluster up for a test site, it works...

    Right now I'm still doing the monolith all on one server and just upsizing as it gets tight...
    presently on a linode dedicated 32Gb ram with zero complaints. Everything is just about fitting in, but today triggered a 80% alarm on ram... traffic is picking up again.

    I switched from dynamic to static when I moved to the dedicated cpu, running 120 php procs. I should probabbly if I'm reading my queue length right need more.

    Anyhow, I've been toying with moving the db off. Although you get a higher chance of downtime with two nodes, that would ( touch wood ) still be quite low and bearable for a forum site.

    So I was toying with moving to DO for the managed mysql. Partly so I don't have to worry about it; sod the cost. Partly also for the 'real' private network, although linode is meant to be bringing that this year.

    However my mysql instance needs around 14gb or ram for the innodb tables/index, which is right on the max DO managed db size, unless I'm misunderstanding modern mysql innodb sizing. I also can't see anyway to 'tune' innodb sizing, buffer size etc. Do I trust they look in on this... I could possibly reduce this memory impact, if I get search offloaded to elastic. So 3 nodes ;)

    I know it's a heck of a price, but I'm not of the mind to do a mysql cluster. Because If I did that I'd be daft not to do a php a:b pair too. But thats overkil for what it is...


    Thoughts?

    ps I'm also a bit wary of DO's ddos reactions of null routing, but that might be bad internet blood.
     
  2. eva2000

    eva2000 Administrator Staff Member

    52,262
    12,005
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,508
    Local Time:
    10:00 AM
    Nginx 1.25.x
    MariaDB 10.x
    No idea on DO's managed database service. But if it's like their DO droplets, then performance might not be that good relative to other VPS providers. DO droplets for standard and dedicated cpu both consistently ranked at the bottom of comparison benchmarks I did at DigitalOcean - Linode - Vultr - Benchmarks - Upcloud - 13-Way VPS Server Benchmark Comparison Tests - Upcloud vs DigitalOcean vs Linode vs Vultr vs Hetzner.

    Best thing is to see what/if any caching can offload load from PHP itself. Some Centmin Mod commands might help dig deeper into system usage metrics via cminfo command
    Code (Text):
    cminfo top
    cminfo sar-cpu
    cminfo sar-mem
    cminfo phpstats
    cminfo phpmem
    

    up to you to interpret the output though.

    If you can use Elasticsearch to offload search related database size/buffer usage, then do it even if Elasticsearch is on same server it's not 1:1 for database size versus memory usage.
     
  3. Colin

    Colin Premium Member Premium Member

    191
    59
    28
    Oct 7, 2015
    Sheffield UK
    Ratings:
    +153
    Local Time:
    1:00 AM
    1.19.#
    MariaDB 10.1.#
    more commands and data to parse :)

    I'm now hovering around 75% of ram so won't push much more on to it.
    Code:
    Every 2.0s: ...  Wed Sep 16 14:50:08 2020
    
    Processes active: 18, idle: 82, Requests:
     2770091, slow: 0, Traffic: 5.9req/sec
    pool:                 www
    process manager:      static
    start time:           09/Sep/2020:22:36:2
    7 +0000
    start since:          576821
    accepted conn:        2770138
    listen queue:         0
    max listen queue:     51
    listen queue len:     511
    idle processes:       84
    active processes:     16
    total processes:      100
    max active processes: 207
    max children reached: 0
    slow requests:        0
    It's quite quiet at the moment. Kind of wish there was a way to capture the peak, nixstats quite handy for that in a way.

    I've used them, looked at the numbers, then went for a butt dyno.

    Setting up on upcloud. Would of gone vultr; possibly, but supply is nonexisteant and thats a concern if I wanted to experiment and to consolidate.

    There are more costs in upcloud $ for $, e.g. for a private network, they have a private utility for all vps's on account, to add another it's $5. A firewall is an addiitonal 5$, backups are a bit steeper, but choices like weekly, monthly or year retentions. There is also a scary zero balance termination bit in the terms. Hope the credit card never fails ;)

    So to the question of is a managed db worth it... Given I'd be almost doubling my hosting cost, just to bump the database to a standalone 'manged', maybe. It would be closer tripple with failover, which is what I was attracted to, too save setting/managing it. Thats quite a sum and certianly a pause for thought, it would easiy cover the cost of a 'on call' sys admin.

    Although this is my first multibox setup in a very long time; I've been spoilt by the likes of heroku and vertical box scaling... first under centmin too, so it's raised questions for me about csf config :) That and reminded me I'm not a sysadmin; still a dangerous one, a rookie; I can break things and not always know how to fix them :)

    Code:
    #csf.allow
    tcp|in|d=3306|s=10.x.x.x
    
    I was looking for a way to; for the db nodes, ignore all from eth0; public, allow from eth1; private, then allow port 3306 on eth1 etc rather than each ip on it's own... although being explicit isn't a problem.

    But then I see all the allow from cloudflare which will blow past any port restrictions from their ips.

    My TCP_IN rule for permit is empty on both web and sql sets of nodes. So kinf oa TCP_IN for each eth really, but can't see anything in the csf docs that leaps out.

    Fun days :)
     
  4. eva2000

    eva2000 Administrator Staff Member

    52,262
    12,005
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,508
    Local Time:
    10:00 AM
    Nginx 1.25.x
    MariaDB 10.x
    Just setup command to run via a cronjob - you can do it per second logging to a log file. Then you can use Linux grep/awk cmds to filter and inspect the logs and dig much deeper into specifics than charting would.

    FYI, recently added cminfo sar-json command too if you want to manipulate sar stats in json format Upgrade - Nginx - Insight Guide - cminfo command explained
    Upcloud operates on a credit added basis instead of credit card charge basis AFAIK.
    It shouldn't if db connection is via private plan and db server IP address / port isn't listening to a public IP but private IP.
     
  5. Genos

    Genos New Member

    2
    0
    1
    Aug 11, 2021
    Ratings:
    +0
    Local Time:
    3:00 AM
    1.5
    MariaDB 10
    If your MySQL instance requires around 14GB of RAM, that's close to the maximum offered by Digital Ocean's managed database service, you may want to explore other cloud providers that offer higher RAM options or consider optimizing your database to reduce its memory footprint. While some managed database services abstract away certain configurations for simplicity, others may allow you to tweak certain settings. It's important to check Digital Ocean's documentation or support channels to confirm if and how you can adjust settings like InnoDB buffer size. If not, you may need to consider whether the managed service meets your specific performance tuning requirements. If you're concerned about potential downtime during migrations or upgrades, consider a distributed architecture. This could involve having multiple database nodes, a load balancer, and potentially offloading certain functionalities like search to a separate system. However, this does increase complexity.
     
  6. buik

    buik “The best traveler is one without a camera.”

    1,974
    515
    113
    Apr 29, 2016
    Flanders
    Ratings:
    +1,641
    Local Time:
    2:00 AM
    Don't. By the way, you almost answer your own question already, which is a good sign.
    Although I have no professional experience with this service nor the company. Because I am not allowed to host data professionally at a company with an American owner, and therefore U.S. law.

    You are already at the DO limits and that right from the start, your current system is running fine and could be even upgraded if needed (Linode is up to 512GB per unit if needed), and you expect growth. Given these starting factors, I hadn't even considered DO.

    P.s. by the way, I only now see that this is an old topic from 2020. Apologies. It was tagged in my recent post list as of today. Hopefully it is still relevant to the topic starter or another interested person.
     
    Last edited: Feb 4, 2024