Learn about Centmin Mod LEMP Stack today
Register Now

Featured DigitalOcean Linode Vultr Benchmarks DigitalOcean vs Linode vs Vultr [48GB 16 CPUS] centminmodbench.sh results

Discussion in 'Virtual Private Server (VPS) hosting' started by eva2000, Sep 13, 2014.

  1. eva2000

    eva2000 Administrator Staff Member

    29,743
    6,719
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,031
    Local Time:
    9:32 AM
    Nginx 1.13.x
    MariaDB 5.5

    Compare DigitalOcean vs Linode vs Vultr



    I ran comparison centminmodbench.sh benchmarks for 48GB memory 16 CPU processor core VPSes from 3 cloud / VPS hosting providers DigitalOcean, Linode and Vultr. Below are the tabulated results. May 6th, 2015 update - just a note that Vultr has increased their prices and changed their plans.

    Referral links:

    If you like this comparison review, please consider using my referral links to sign up with the below providers or put the word out and share this benchmark review with your friends etc :D :
    • DigitalOcean - sign up via this link and you also get $10 credit to your account as well :)
    • Linode - similar offer, use promo code LINODE10 for $10 credit to your account too :)
    • Vultr - Vultr already offers free $5 credit for new sign ups :)
    VPS servers:

    For Vultr, New Jersey only location offering 16 cpu VPS
    • DigitalOcean KVM VPS - San Francisco: 48GB, 16 CPUS, 480GB SSD 8TB (1Gbps network in/out) US$480/month
    • Linode XEN VPS - Freemont California: 48GB, 16 CPUS, 1152GB SSD 20TB (40Gbps in & 6Gbps network out) US$480/month
    • Vultr KVM VPS - New Jersey: 48GB, 16 CPUS, 600GB SSD 8TB (1Gbps network in/out) US$429.95/month

    Testing Methodology



    I basically created each respective VPS server using CentOS 6.5 64bit OS. Then ran the new installnbench.sh file at centminmod/centminmodbench ยท GitHub which I wrote specifically for this testing. It basically does 2 tasks when run installnbench.sh via curl command line on a fresh CentOS 6.5 or 7.0 OS server:
    1. install latest Centmin Mod .08 beta01 LEMP stack ~ 15-30 mins time then
    2. installs and runs centminmodbench.sh (UnixBench enabled) ~ 30-60 mins time
    Code:
    curl -sL https://github.com/centminmod/centminmodbench/raw/master/installnbench.sh | bash
     
    Last edited: May 6, 2015
  2. eva2000

    eva2000 Administrator Staff Member

    29,743
    6,719
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,031
    Local Time:
    9:32 AM
    Nginx 1.13.x
    MariaDB 5.5

    Centmin Mod .08 beta01 Install time



    Centmin Mod .08 beta01 LEMP web stack install times are highly variable as part of the time is downloading source tarballs or YUM package downloads and installs so depends on the speed of the respective YUM mirrors the system picks up to download from as well as location of the VPS server geographically to the downloads. Downloads account for between 15-33% of the total install time. The rest is source compilation for Nginx, PHP-FPM, PHP extensions, Memcached server, CSF Firewall and other installed items via the Centmin Mod auto installer. Ultimately, comes down to cpu processor, disk performance and cpu clock speed for source compilation parts.

    For installnbench.sh users, you'll find the install time on the very last line in the /root/centminlogs/centminmod_1.2.3-eva2000.08_***-***_install.log install log.

    digitalocean_linode_vultr_48gb_16cpus_centminmod_install_time.png
     
    Last edited: Sep 14, 2014
  3. eva2000

    eva2000 Administrator Staff Member

    29,743
    6,719
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,031
    Local Time:
    9:32 AM
    Nginx 1.13.x
    MariaDB 5.5

    DigitalOcean vs Linode vs Vultr - ioping disk



    Ioping measures random disk IO performance.
    • For the default ioping test seems DigitalOcean > Vultr > Linode.
    • For disk seek performance, Linode > DigitalOcean > Vultr
    • For sequential disk ioping performance, DigitalOcean > Linode > Vultr
    • For cached ioping disk performance, Vultr > DigitalOcean > Linode

    digitalocean_linode_vultr_48gb_16cpus_ioping.png
     
    Last edited: Sep 13, 2014
  4. eva2000

    eva2000 Administrator Staff Member

    29,743
    6,719
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,031
    Local Time:
    9:32 AM
    Nginx 1.13.x
    MariaDB 5.5

    DigitalOcean vs Linode vs Vultr - disk dd



    Linode's SSD disk configuration is the winner here for disk dd benchmarks, followed by DigitalOcean and Vultr.

    digitalocean_linode_vultr_48gb_16cpus_disk_dd.png
     
    Last edited: Sep 13, 2014
  5. eva2000

    eva2000 Administrator Staff Member

    29,743
    6,719
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,031
    Local Time:
    9:32 AM
    Nginx 1.13.x
    MariaDB 5.5

    DigitalOcean vs Linode vs Vultr - disk fio



    Vultr seems to have lower yet more uniform disk performance for random reads and random writes being roughly equal with 48K iops compared to DigitalOcean at 88K reads and 16K writes and Linode's 86K reads and 28K writes. However, both DigitalOcean and Linode have 83% and 79% respectively more random read IOPs than Vultr but lower random write IOPs.

    digitalocean_linode_vultr_48gb_16cpus_disk_fio.png
     
    Last edited: Sep 13, 2014
  6. eva2000

    eva2000 Administrator Staff Member

    29,743
    6,719
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,031
    Local Time:
    9:32 AM
    Nginx 1.13.x
    MariaDB 5.5

    DigitalOcean vs Linode vs Vultr - bandwidth tests


    • Linode clearly has a better network in terms of bandwidth tests given that it's 48GB 16CPU VPS plan allows for respectively 40x and 6x times higher bandwidth network throughput with 40Gbps in and 6Gbps out compared to DigitalOcean and Vultr at 1Gbps in and out.
    • Linode also has better performance for Asian test locations and just edging out DigitalOcean for European bandwidth locations.
    • But note, Vultr is located on East coast USA in New Jersey, while DigitalOcean and Linode VPS are located on West coast USA.
    • My personal preference due to visitor traffic for my sites is for West Coast USA as it has better latency and speeds to Asian/Australian locations.
    Update: Some of the bandwidth speeds reported might be lower due to Centmin Mod .08 beta01 disabling tcp_sack. Re-enabling would allow most locations to gain in speed. So will be redoing comparison benchmarks for a part 2 when Vultr multi-core VPSes become available.

    digitalocean_linode_vultr_48gb_16cpus_bandwidth.png
     
    Last edited: Sep 17, 2014
  7. eva2000

    eva2000 Administrator Staff Member

    29,743
    6,719
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,031
    Local Time:
    9:32 AM
    Nginx 1.13.x
    MariaDB 5.5

    DigitalOcean vs Linode vs Vultr - ping benchmarks


    • Ping tests are all geographically relative to the VPS server location, so West coast USA would fair better with same location ping test sites and vice versa for East coast USA based Vultr VPS.
    digitalocean_linode_vultr_48gb_16cpus_ping.png
     
    Last edited: Sep 14, 2014
  8. eva2000

    eva2000 Administrator Staff Member

    29,743
    6,719
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,031
    Local Time:
    9:32 AM
    Nginx 1.13.x
    MariaDB 5.5

    DigitalOcean vs Linode vs Vultr - OpenSSL System Benchmarks


    • OpenSSL system benchmarks test against system default OpenSSL 1.0.1e for number of ciphers and configurations would be good indicators for https SSL performance given Google now includes https in their ranking signal search engine algorithms. Relevant tests include rsa 2048bit, rsa 4096bit and 256bit ecdsa key algorithms. You can read why the newer ECC SSL certificates using 256 bit ecdsa key algorithms are better performing than the more commonly used rsa 2048 bit keys here. As well as test aes128/256 cbc non cpu hardware assisted performance and AES-NI cpu hardware assisted performance via the evp aes128/256 numbers.
    • For DigitalOcean and Linode at least both are using Intel Xeon E5-2600v2 based processors based on Ivy Bridge EP platform which both support AES-NI (Advanced Encryption Standard New Instructions). So naturally you would see alot faster evp aes128/aes256 numbers compared to cpu processors not using AES-NI.
    • Vultr just edges out Linode to take the top position for rsa 2048 bit, rsa 4096 bit and 256 bit ecdsa sign/s and verify/s. However, Vultr just falls behind Linode for AES-NI assisted evp aes256 and evp aes128 performance which may suggest Vultr might be using a early generation processor - maybe Intel Xeon E5-2600v1 Sandy Bridge EP based which still supports AES-NI but can't match the minor improvements in AES-NI performance that the newer Intel Xeon E5-2600v2 Ivy Bridge EP platform has. It could be Vultr using an Intel Xeon E5-2600v1 which is higher clocked compared to Linode's Intel Xeon E5-2680v2, which would explain the higher numbers for non-AES-NI based OpenSSL numbers for rsa 2048 bit, rsa 4096 bit and 256 bit ecdsa sign/s and verify/s. However, lscpu output shows Vultr has a lower base clock, so it could be the cpu used has a higher Turbo Boost clock speed in play ?
    digitalocean_linode_vultr_48gb_16cpus_openssl_system.png
     
  9. eva2000

    eva2000 Administrator Staff Member

    29,743
    6,719
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,031
    Local Time:
    9:32 AM
    Nginx 1.13.x
    MariaDB 5.5

    DigitalOcean vs Linode vs Vultr - Centmin Mod Nginx static OpenSSL Benchmarks



    Centmin Mod LEMP web stack's Nginx install is compiled statically against a more up to date OpenSSL version 1.0.1i right now. This allows more flexibility in choosing what versions of OpenSSL to use for Nginx and Nginx SSL / SPDY SSL including using OpenSSL 1.0.2 with chacha20-poly1305 cipher support.

    digitalocean_linode_vultr_48gb_16cpus_openssl_nginx_static.png
     
    Last edited: Sep 13, 2014
  10. eva2000

    eva2000 Administrator Staff Member

    29,743
    6,719
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,031
    Local Time:
    9:32 AM
    Nginx 1.13.x
    MariaDB 5.5

    DigitalOcean vs Linode vs Vultr - mysqlslap MySQL Benchmarks



    mysqlslap tool is included with all MySQL installs including Oracle MySQL, MariaDB MySQL and Percona MySQL. The custom mysqlslap parameters I used are just arbitrarily in nature and only test MyISAM performamce across 64 clients.

    Actual test command used is:

    MariaDB 10.0.13 MySQL is the default installed MySQL version in Centmin Mod .08 beta01 LEMP auto installer. Vultr comes out on top. Processor performance and clock speed plays a factor and probably Vultr leads because of this as well as maybe the more uniform and performant random write disk IOPs numbers shown above for disk fio benchmarks.

    digitalocean_linode_vultr_48gb_16cpus_mysqlslap.png
     
  11. eva2000

    eva2000 Administrator Staff Member

    29,743
    6,719
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,031
    Local Time:
    9:32 AM
    Nginx 1.13.x
    MariaDB 5.5

    DigitalOcean vs Linode vs Vultr - PHP Benchmarks



    • PHP benchmarks test the Centmin Mod .08 beta01 installed PHP 5.4.32 (php-fpm) binary (which defaults include enabling APC Cache 3.1.13 with igbinary support) and tests against the PHP 5.4.32 source included Zend/bench.php and Zend/micro_bench.php files.
    • You can see the PHP 5.4.32 default compiled extensions here.
    • 3 runs are made and average taken for time it takes to run the benchmarks.
    • This test is single threaded in nature so doesn't scale with multiple cpu cores/threads and the underlying performance is based partly on cpu processor performance and clock speed. Which the final results confirm as Vultr is the winner, then Linode and DigitalOcean with the lower clock speeds comes out last.

    digitalocean_linode_vultr_48gb_16cpus_php.png

    PHP 5.4.32 default configuration

    PHP 5.4.32 default loaded modules

    Code:
    [PHP Modules]
    apc
    bcmath
    bz2
    calendar
    Core
    ctype
    curl
    date
    dom
    enchant
    ereg
    exif
    filter
    ftp
    gd
    gettext
    gmp
    hash
    iconv
    igbinary
    imagick
    imap
    intl
    json
    libxml
    mbstring
    mcrypt
    memcache
    memcached
    mhash
    mysql
    mysqli
    mysqlnd
    openssl
    pcntl
    pcre
    PDO
    pdo_mysql
    pdo_sqlite
    Phar
    posix
    pspell
    readline
    Reflection
    session
    shmop
    SimpleXML
    soap
    sockets
    SPL
    sqlite3
    standard
    sysvmsg
    sysvsem
    sysvshm
    tidy
    tokenizer
    xml
    xmlreader
    xmlrpc
    xmlwriter
    zip
    zlib
     
    Last edited: Sep 13, 2014
  12. eva2000

    eva2000 Administrator Staff Member

    29,743
    6,719
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,031
    Local Time:
    9:32 AM
    Nginx 1.13.x
    MariaDB 5.5

    DigitalOcean vs Linode vs Vultr - UnixBench Benchmarks



    As per below notes, you can only validly compare UnixBench system index scores within the same branch of Linux kernels 2.6.x or 3.x. Comparing 2.6.x vs 3.x kernels won't be directly comparable due to differences in subtest numbers that the index score is derived from.

    Vultr and DigitalOcean use same 2.6.x linux kernel, so you can see definitely that Vultr is a faster system than DigitalOcean. To compare with Linode, you can probably only rely on looking at the Dhrystone and Whetstone numbers to gauge cpu processor performance.
    • DigitalOcean uses Xeon E5-2630L processors with 6 cores / 12 threads clocked at 2.4Ghz with Turbo Boost to 2.8Ghz. All cores Turbo Boosts to 2.6Ghz and single cpu core Turbo Boosts to 2.8Ghz
    • Linode uses Xeon E5-2680v2 processors with 10 cores / 20 threads clocked at 2.8Ghz with Turbo Boost to 3.6Ghz. All cores Turbo Boosts to 3.1Ghz and single cpu core Turbo Boosts to 3.6Ghz.
    • Vultr was not able to determine which processors used from benchmark numbers but it looks to be faster than Linode's Xeon E5-2680v2 given the UnixBench subtest numbers from Dhrystone, Whestone and above cpu hardware assisted AES-NI OpenSSL evp aes128/aes256 benchmark numbers. Could be that Vultr is using a higher clocked Xeon E5-2600v2 most likely ? Xeon E5-2690v2 maybe ? The lscpu numbers show Vultr at 2.6Ghz base clock so it could be Xeon E5-2650v2 or an Xeon E5-2670v1 ?
    • Looks like based on Dhrystone, Whestone and OpenSSL tests, Vultr > Linode > DigitalOcean.
    Intel ARK comparison of Xeon E5-2630L vs Xeon E5-2680v2

    digitalocean_linode_vultr_48gb_16cpus_unixbench.png

    Notes regarding UnixBench Index numbers



    Unixbench index numbers are much lower for Linux 3.x kernels as opposed to Linux 2.6.x kernels as some sub tests are influenced by the differing Linux Kernel versions used. From previous local tests I have done with CentOS, the 2.6.32 Kernels performed better in some UnixBench subtests than 3.xx Kernels which are used by Linode. Particularly, for Process Creation Index tests.

    Code:
    2.6.32-431 = 3287.4
    3.8.13-26.1 = 2417.4
    3.13.4-1 = 1341.0
    You can read the explanation at byte-unixbench - A Unix benchmark suite - Google Project Hosting which suggests subtests like pipe throughout has no real world bearing on performance so can be disregarded

     
    Last edited: Sep 14, 2014
  13. eva2000

    eva2000 Administrator Staff Member

    29,743
    6,719
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,031
    Local Time:
    9:32 AM
    Nginx 1.13.x
    MariaDB 5.5

    DigitalOcean vs Linode vs Vultr Summary



    The reason why I did the above comparison benchmarks is because I use all 3 VPS cloud hosting providers to constantly test my Centmin Mod LEMP menu based auto installer web stack as well as having this very forum hosted on Linode right now. So it's good to know which VPS cloud hosting provider has the best performance for my needs which includes Centmin Mod install times (I may install and re-install 100s of times per month). As well as general performance that has some relative real world applications for what a normal LEMP web stack would be for - Nginx SPDY/3.1 SSL, PHP-FPM performance and MariaDB MySQL performance.

    Given the monthly prices are pretty much the same with Vultr slightly cheaper, it seems Linode is the better value as it has more SSD disk space at 1152GB vs 480GB for DigitalOcean and 600GB for Vultr, more bandwidth allocated 20TB vs 8TB, and faster network in/out at 40Gbps/6Gbps vs 1Gbps.

    For Centmin Mod install times:
    For Disk Performance:
    • There's no clear leader. However, overall Linode seems to have the right mix of sequential and random disk IO read/write performance. Followed by DigitalOcean and the Vultr.
    For Bandwidth & Pings performance:
    • Linode has the best network configuration with 40Gbps in and 6Gbps out for their 48GB 16CPU VPS plan. It shows in some of the numbers. However, you do need to note that Vultr was tested on the East coast USA in New Jersey so better suited for Europe regions as well while the other two were tested on West Coast USA which would be better suited to Asia/Oceania regions.
    For OpenSSL performance:
    For mysqlslap MySQL performance:
    • Vultr > Linode > DigitalOcean. I suspect partly due to cpu clock speeds although unsure of what processor Vultr uses. But MySQL performance tested was for MyISAM engine and default my.cnf tuned by Centmin Mod installer. So doesn't really reflect real world usage and when you use InnoDB engine or when you properly tune your MySQL my.cnf settings for your work loads.
    For PHP performance
    • The PHP benchmarks were only single threaded so reflects more cpu processor performance and clock speed. Again Vultr > Linode > DigitalOcean.
    For UnixBench performance:
    • Again, Vultr > Linode > DigitalOcean due to probably cpu processor clock speed factors.
    • Below are the lscpu outputs for all 3 VPS servers. But Vultr based cpu clock is lower than Linode according to lscpu output ? So maybe Vultr cpu has higher Turbo Boost clock speeds than Linode when not all cpu threads are utilised and engaged ? This would show itself in tests which do not use all available cpu cores/threads. Or could be difference in overhead for Linode Xen based virtualization vs Vultr KVM based virtualization ?

    DigitalOcean

    Code:
    ----------------------------------------------
    Architecture:          x86_64
    CPU op-mode(s):        32-bit, 64-bit
    Byte Order:            Little Endian
    CPU(s):                16
    On-line CPU(s) list:   0-15
    Thread(s) per core:    1
    Core(s) per socket:    1
    Socket(s):             16
    NUMA node(s):          1
    Vendor ID:             GenuineIntel
    CPU family:            6
    Model:                 62
    Stepping:              4
    CPU MHz:               2399.998
    BogoMIPS:              4799.99
    Virtualization:        VT-x
    Hypervisor vendor:     KVM
    Virtualization type:   full
    L1d cache:             32K
    L1i cache:             32K
    L2 cache:              256K
    L3 cache:              15360K
    NUMA node0 CPU(s):     0-15
    Linode

    Code:
    ----------------------------------------------
    Architecture:          x86_64
    CPU op-mode(s):        32-bit, 64-bit
    Byte Order:            Little Endian
    CPU(s):                16
    On-line CPU(s) list:   0-15
    Thread(s) per core:    16
    Core(s) per socket:    1
    Socket(s):             1
    NUMA node(s):          1
    Vendor ID:             GenuineIntel
    CPU family:            6
    Model:                 62
    Stepping:              4
    CPU MHz:               2800.046
    BogoMIPS:              5602.42
    Hypervisor vendor:     Xen
    Virtualization type:   full
    L1d cache:             32K
    L1i cache:             32K
    L2 cache:              256K
    L3 cache:              25600K
    NUMA node0 CPU(s):     0-15
    Vultr

    Code:
    ----------------------------------------------
    Architecture:          x86_64
    CPU op-mode(s):        32-bit, 64-bit
    Byte Order:            Little Endian
    CPU(s):                16
    On-line CPU(s) list:   0-15
    Thread(s) per core:    1
    Core(s) per socket:    16
    Socket(s):             1
    NUMA node(s):          1
    Vendor ID:             GenuineIntel
    CPU family:            6
    Model:                 42
    Stepping:              1
    CPU MHz:               2599.998
    BogoMIPS:              5199.99
    Hypervisor vendor:     KVM
    Virtualization type:   full
    L1d cache:             32K
    L1i cache:             32K
    L2 cache:              4096K
    NUMA node0 CPU(s):     0-15
    For Vultr cpu, with SMP capable Intel Xeon cpus with a base clock of 2.6Ghz, narrowed it down to maybe of these according to Intel ARK database ? Still waiting on reply from Vultr as to specific model used.
    • Intel Xeon Processor E5-2670 (20M Cache, 2.60 GHz)
    • Intel Xeon Processor E5-4650L (20M Cache, 2.60 GHz)
    • Intel Xeon Processor E5-2630 v2 (15M Cache, 2.60 GHz)
    • Intel Xeon Processor E5-2650 v2 (20M Cache, 2.60 GHz)
    • Intel Xeon Processor E5-4607 v2 (15M Cache, 2.60 GHz)
    • Intel Xeon Processor E5-4620 v2 (20M Cache, 2.60 GHz)
    • Intel Xeon Processor E7-4860 v2 (30M Cache, 2.60 GHz)
     
    Last edited: Sep 15, 2014
    • Like Like x 3
  14. RoldanLT

    RoldanLT Well-Known Member

    3,883
    947
    113
    May 25, 2014
    Phillipines
    Ratings:
    +1,291
    Local Time:
    7:32 AM
    1.11
    10.2
    That takes a lot of time.
    Thanks a lot for this result @eva2000 (y)
     
    • Like Like x 1
  15. eva2000

    eva2000 Administrator Staff Member

    29,743
    6,719
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,031
    Local Time:
    9:32 AM
    Nginx 1.13.x
    MariaDB 5.5
    • Like Like x 1
  16. RoldanLT

    RoldanLT Well-Known Member

    3,883
    947
    113
    May 25, 2014
    Phillipines
    Ratings:
    +1,291
    Local Time:
    7:32 AM
    1.11
    10.2
    Versaweb vs OVH vs SoYouStart vs Online.Net? :D
    With mostly the same specs.
     
  17. eva2000

    eva2000 Administrator Staff Member

    29,743
    6,719
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,031
    Local Time:
    9:32 AM
    Nginx 1.13.x
    MariaDB 5.5
    haha don't have that kind of $$$$ hehe
     
  18. RoldanLT

    RoldanLT Well-Known Member

    3,883
    947
    113
    May 25, 2014
    Phillipines
    Ratings:
    +1,291
    Local Time:
    7:32 AM
    1.11
    10.2
    I know @Matt have one extra on SoYouStart :D
    I know you have also one on VersaWeb :)
     
  19. eva2000

    eva2000 Administrator Staff Member

    29,743
    6,719
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,031
    Local Time:
    9:32 AM
    Nginx 1.13.x
    MariaDB 5.5
    maybe start a new thread for centminmodbench.sh for dedicated servers in dedicated server forums :)
     
  20. RoldanLT

    RoldanLT Well-Known Member

    3,883
    947
    113
    May 25, 2014
    Phillipines
    Ratings:
    +1,291
    Local Time:
    7:32 AM
    1.11
    10.2
    Later :D
    I tried to order one on online.net, much cheaper than I currently have but with much powerful cpu and ECC RAM.
    But they don't accept Paypal, and I don't have credit's on my Credit Card. :(