/ Register

Learn about Centmin Mod LEMP Stack today
Register Now

Featured Benchmarks Hardware Packet.net bare metal cloud AMD EPYC 7401P review & benchmarks

Discussion in 'Dedicated server hosting' started by eva2000, Feb 26, 2018.

Tags:
Thread Status:
Not open for further replies.
Previous Thread Next Thread
Loading...
  1. Feb 26, 2018 #1
    eva2000

    eva2000 Administrator Staff Member

    53,280
    12,117
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,655
    Local Time:
    2:23 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    This thread is dedicated to review and benchmarks for Packet.net - a bare metal cloud provider that offers on demand hourly, reserved, spot and privately deployed bare metal servers. Packet.net offers bare metal servers and recently announced roll out AMD EPYC 7401P 24 core/48 thread servers as well with AMD EPYC 7401P details as well as Intel Xeon Gold 5120 Scalable processors too. If you like this benchmark review and find it useful, please consider a small donation to support Centmin Mod. This time of testing does cost $$$ :)

    Packet.net Bare Metal Server Benchmark Reviews



    Packet.net Plans & Pricing



    • Packet.net Pricing - with bandwidth out prices at US$0.05/GB pay as you go so prices are too high for continuous 24x7x30 day usage for me but they're catering to developers too so if you need high performance bare metal servers for a few hours, then just the perfect fit. Centmin Mod optimisation and testing will be fun now that they have AMD EPYC 7401P cpus on offer :) They have spot pricing, for example their TYPE 1E plan is for Intel Xeon E3-1578Lv5 Skylake based cpu with 32GB DDR4-2400 ECC and 240GB SSD with 10Gbps Intel X710 dual NIC in TLB config runs at US$0.40/hr on demand pricing but I managed to get Los Angeles based TYPE 1E on spot pricing for US$0.08-0.15/hr for testing.
    • Packet.net FAQ
    The plans

    packetnet-plans.png

    And Intel Xeon Gold 5120 Scalable cpus

    packetnet-plans2.png

    The bandwidth pricing is still better than other large cloud providers like Amazon AWS, Google Cloud, Softlayer, Rackspace and Microsoft Azure etc.

    packetnet-bandwidth-price.png
     
    Last edited: Feb 28, 2018
  2. Feb 26, 2018 #2
    eva2000

    eva2000 Administrator Staff Member

    53,280
    12,117
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,655
    Local Time:
    2:23 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    Packet.net TYPE 1E Intel Xeon E3-1578Lv5 Skylake



    I spun up a Packet.net Los Angeles based TYPE 1E Intel Xeon E3-1578Lv5 Skylake bare metal server using spot bidding pricing at US$0.15/hr on current market rate of US$0.08/hr (on demand pricing is US$0.40/hr) for Centmin Mod 123.09beta01 install testing and centminmodbench.sh testing.

    Intel® Xeon® Processor E3-1578L v5 (8M Cache, 2.00 GHz) Product Specifications

    type1e-hardware-ark.png

    Highlights would definitely be 10Gbps network speed i.e. Vultr Los Angeles download speed test managed to pull 442MB/s ! The weakness if you must compare to other providers is disk I/O speed with read/write under 200-230MB/s with 45k/65K random IOPs.

    System
    • 4 Physical Cores @ 2.0 GHz base / 3.40 GHz burst (1 × E3-1578L)
    • Integrated Intel® Iris™ Pro Graphics P580
    • 32GB DDR4-2400 ECC RAM
    • 240 GB of SSD (1 × 240 GB)
    • 10 Gbps Network (2 × Intel X710 NIC's in TLB)
    • CentOS 7.4 64bit
    • Los Angeles
    type1e-hardware-01.png

    Packet.net TYPE 1E Details



    Intel Xeon E3-1578Lv5 Skylake cpu frequencies 2.0Ghz base with 3.4Ghz Turbo for 1 core and 3.2Ghz for all 4 cores.

    Code (Text):
    True Frequency (without accounting Turbo) 1992 MHz
  CPU Multiplier 20x || Bus clock frequency (BCLK) 99.60 MHz

Socket [0] - [physical cores=4, logical cores=8, max online cores ever=4]
  TURBO ENABLED on 4 Cores, Hyper Threading ON
  Max Frequency without considering Turbo 2091.60 MHz (99.60 x [21])
  Max TURBO Multiplier (if Enabled) with 1/2/3/4 Cores is  34x/33x/32x/32x
  Real Current Frequency 3373.64 MHz [99.60 x 33.87] (Max of below)
        Core [core-id]  :Actual Freq (Mult.)      C0%   Halt(C1)%  C3 %   C6 %  Temp      VCore
        Core 1 [0]:       3373.64 (33.87x)         1    96.9       1    1.84    21      1.0911
        Core 2 [1]:       3356.55 (33.70x)         1    97.9       1       1    21      1.0885
        Core 3 [2]:       3356.99 (33.70x)         1    98.8       0    1.05    20      1.1085
        Core 4 [3]:       3354.06 (33.68x)         1    97.9       1       1    20      1.1085

C0 = Processor running without halting
C1 = Processor running with halts (States >C0 are power saver modes with cores idling)
C3 = Cores running with PLL turned off and core cache turned off
C6, C7 = Everything in C3 + core state saved to last level cache, C7 is deeper than C6
  Above values in table are in percentage over the last 1 sec

    Code (Text):
    lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                8
On-line CPU(s) list:   0-7
Thread(s) per core:    2
Core(s) per socket:    4
Socket(s):             1
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 94
Model name:            Intel(R) Xeon(R) CPU E3-1578L v5 @ 2.00GHz
Stepping:              3
CPU MHz:               3396.015
CPU max MHz:           3400.0000
CPU min MHz:           800.0000
BogoMIPS:              3984.00
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              8192K
NUMA node0 CPU(s):     0-7
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb invpcid_single intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp

    Memory
    Code (Text):
    free -m
              total        used        free      shared  buff/cache   available
Mem:          32084        1481       29794          33         808       30166
Swap:          1949           0        1949

    Disk
    Code (Text):
    df -hT
Filesystem     Type      Size  Used Avail Use% Mounted on
/dev/sda3      ext4      219G  5.6G  202G   3% /
devtmpfs       devtmpfs   16G  4.0K   16G   1% /dev
tmpfs          tmpfs      16G     0   16G   0% /dev/shm
tmpfs          tmpfs      16G   17M   16G   1% /run
tmpfs          tmpfs      16G     0   16G   0% /sys/fs/cgroup
tmpfs          tmpfs     3.2G     0  3.2G   0% /run/user/0
tmpfs          tmpfs      16G  336K   16G   1% /tmp

    Centmin Mod 123.09beta01 Install



    Used default curl installer command below
    Code (Text):
    yum -y update; curl -O https://centminmod.com/betainstaller.sh && chmod 0700 betainstaller.sh && bash betainstaller.sh

    Initial Install Time was very decent at ~721 seconds for Intel Xeon E3-1578Lv5 Skylake with 2.0Ghz based and 3.40Ghz Turbo. You can compare that to other times posted at Install - Post your Centmin Mod 123.09beta01 install time stats. Though at as Feb 24th, 2018, the 123.09beta01 installed changed some defaults to more performance at the expense of initial install time.
    Code (Text):
    ---------------------------------------------------------------------------
Total Curl Installer YUM or DNF Time: 91.2912 seconds
Total YUM Time: 64.039117358 seconds
Total YUM or DNF + Source Download Time: 87.3015
Total Nginx First Time Install Time: 135.2867
Total PHP First Time Install Time: 141.0516
Download Zip From Github Time: 1.8695
Total Time Other eg. source compiles: 264.2750
Total Centmin Mod Install Time: 627.9147
---------------------------------------------------------------------------
Total Install Time (curl yum + cm install + zip download): 721.0754 seconds
---------------------------------------------------------------------------

    Post install check list
    Code (Text):
    *************************************************
* Post-Install Check List....
*************************************************

--------------------------------------------------------
Check ccache Version:
--------------------------------------------------------
ccache version 3.4.1

Copyright (C) 2002-2007 Andrew Tridgell
Copyright (C) 2009-2018 Joel Rosdahl

This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation; either version 3 of the License, or (at your option) any later
version.

--------------------------------------------------------
Check Nginx Version:
--------------------------------------------------------
nginx version: nginx/1.13.9
built by gcc 7.2.1 20170829 (Red Hat 7.2.1-1) (GCC)
built with OpenSSL 1.1.0g  2 Nov 2017
TLS SNI support enabled
configure arguments: --with-ld-opt='-ljemalloc -Wl,-z,relro -Wl,-rpath,/usr/local/lib' --with-cc-opt='-m64 -march=native -DTCP_FASTOPEN=23 -g -O3 -fstack-protector-strong -flto -fuse-ld=gold --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wimplicit-fallthrough=0 -fcode-hoisting -Wp,-D_FORTIFY_SOURCE=2 -gsplit-dwarf' --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-compat --with-http_stub_status_module --with-http_secure_link_module --with-libatomic --with-http_gzip_static_module --with-http_sub_module --with-http_addition_module --with-http_image_filter_module=dynamic --with-http_geoip_module --with-stream_geoip_module --with-stream_realip_module --with-stream_ssl_preread_module --with-threads --with-stream=dynamic --with-stream_ssl_module --with-http_realip_module --add-dynamic-module=../ngx-fancyindex-0.4.2 --add-module=../ngx_cache_purge-2.4.2 --add-module=../ngx_devel_kit-0.3.0 --add-dynamic-module=../set-misc-nginx-module-0.31 --add-dynamic-module=../echo-nginx-module-0.61 --add-module=../redis2-nginx-module-0.14 --add-module=../ngx_http_redis-0.3.7 --add-module=../memc-nginx-module-0.18 --add-module=../srcache-nginx-module-0.31 --add-dynamic-module=../headers-more-nginx-module-0.33 --with-pcre=../pcre-8.41 --with-pcre-jit --with-zlib=../zlib-cloudflare-1.3.0 --with-http_ssl_module --with-http_v2_module --with-openssl=../openssl-1.1.0g --with-openssl-opt='enable-ec_nistp_64_gcc_128'

--------------------------------------------------------
Check PHP-FPM Version:
--------------------------------------------------------
PHP 5.6.33 (cli) (built: Feb 25 2018 19:39:56)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
    with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies

--------------------------------------------------------
Check MariaDB installed RPM Versions:
--------------------------------------------------------
MariaDB-devel-10.1.31-1.el7.centos.x86_64
MariaDB-client-10.1.31-1.el7.centos.x86_64
MariaDB-server-10.1.31-1.el7.centos.x86_64
MariaDB-shared-10.1.31-1.el7.centos.x86_64
MariaDB-common-10.1.31-1.el7.centos.x86_64
MariaDB-compat-10.1.31-1.el7.centos.x86_64

--------------------------------------------------------
Check Memcached Server Version:
--------------------------------------------------------
memcached 1.5.5

--------------------------------------------------------
Check CSF Firewall Version:
--------------------------------------------------------
csf: v11.07 (generic)

--------------------------------------------------------
Check Siege Benchmark Version:
--------------------------------------------------------
SIEGE 4.0.4

Copyright (C) 2017 by Jeffrey Fulmer, et al.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE.


--------------------------------------------------------
Check mysqlreport version:
--------------------------------------------------------
mysqlreport v3.5-maria11 Jul 4 2013

--------------------------------------------------------
Check pure-ftpd Version:
--------------------------------------------------------
pure-ftpd v1.0.42

--------------------------------------------------------
Check YUM Repo List
--------------------------------------------------------
Loaded plugins: fastestmirror, priorities, versionlock
Loading mirror speeds from cached hostfile
 * base: mirror.fileplanet.com
 * epel: mirrors.develooper.com
 * extras: mirror.fileplanet.com
 * rpmforge: mirror.hmc.edu
 * updates: mirror.fileplanet.com
228 packages excluded due to repository priority protections
repo id                        repo name                             status
base/7/x86_64                  CentOS-7 - Base                       7,420+2,171
centos-sclo-rh/x86_64          CentOS-7 - SCLo rh                       6,546+94
centos-sclo-sclo/x86_64        CentOS-7 - SCLo sclo                          500
epel/x86_64                    Extra Packages for Enterprise Linux 7  11,376+923
extras/7/x86_64                CentOS-7 - Extras                             388
mariadb                        MariaDB                                      15+2
rpmforge                       RHEL 7 - RPMforge.net - dag               139+106
updates/7/x86_64               CentOS-7 - Updates                      1,499+430
!varnishcache_varnish41/x86_64 varnishcache_varnish41                         46
!varnishcache_varnish41-source varnishcache_varnish41-source                   0
repolist: 27,929

--------------------------------------------------------
mytimes check:
/usr/bin/mytimes
--------------------------------------------------------
Sun Feb 25 19:42:44 UTC 2018    [UTC]
Mon Feb 26 05:42:44 AEST 2018   [Australia/Brisbane]
Sun Feb 25 11:42:44 PST 2018    [America/Los_Angeles]
Sun Feb 25 13:42:44 CST 2018    [America/Chicago]
Sun Feb 25 14:42:44 EST 2018    [America/New_York]
Sun Feb 25 14:42:44 EST 2018    [America/Montreal]
Sun Feb 25 19:42:44 GMT 2018    [Europe/London]
Sun Feb 25 20:42:44 CET 2018    [Europe/Berlin]
Mon Feb 26 02:42:44 +07 2018    [Asia/Bangkok]
Mon Feb 26 02:42:44 +07 2018    [Asia/Ho_Chi_Minh]
Mon Feb 26 02:42:44 WIB 2018    [Asia/Jakarta]
Mon Feb 26 03:42:44 +08 2018    [Asia/Kuala_Lumpur]
Mon Feb 26 03:42:44 +08 2018    [Asia/Singapore]
--------------------------------------------------------

    Intel Xeon E3-1578Lv5 Skylake cpus
    Code (Text):
    cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 94
model name      : Intel(R) Xeon(R) CPU E3-1578L v5 @ 2.00GHz
stepping        : 3
microcode       : 0xba
cpu MHz         : 3397.187
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 0
cpu cores       : 4
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 22
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb invpcid_single intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bogomips        : 3984.00
clflush size    : 64
cache_alignment : 64
address sizes   : 39 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 94
model name      : Intel(R) Xeon(R) CPU E3-1578L v5 @ 2.00GHz
stepping        : 3
microcode       : 0xba
cpu MHz         : 3400.468
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 1
cpu cores       : 4
apicid          : 2
initial apicid  : 2
fpu             : yes
fpu_exception   : yes
cpuid level     : 22
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb invpcid_single intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bogomips        : 3984.00
clflush size    : 64
cache_alignment : 64
address sizes   : 39 bits physical, 48 bits virtual
power management:

processor       : 2
vendor_id       : GenuineIntel
cpu family      : 6
model           : 94
model name      : Intel(R) Xeon(R) CPU E3-1578L v5 @ 2.00GHz
stepping        : 3
microcode       : 0xba
cpu MHz         : 3052.343
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 2
cpu cores       : 4
apicid          : 4
initial apicid  : 4
fpu             : yes
fpu_exception   : yes
cpuid level     : 22
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb invpcid_single intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bogomips        : 3984.00
clflush size    : 64
cache_alignment : 64
address sizes   : 39 bits physical, 48 bits virtual
power management:

processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 94
model name      : Intel(R) Xeon(R) CPU E3-1578L v5 @ 2.00GHz
stepping        : 3
microcode       : 0xba
cpu MHz         : 3206.718
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 3
cpu cores       : 4
apicid          : 6
initial apicid  : 6
fpu             : yes
fpu_exception   : yes
cpuid level     : 22
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb invpcid_single intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bogomips        : 3984.00
clflush size    : 64
cache_alignment : 64
address sizes   : 39 bits physical, 48 bits virtual
power management:

processor       : 4
vendor_id       : GenuineIntel
cpu family      : 6
model           : 94
model name      : Intel(R) Xeon(R) CPU E3-1578L v5 @ 2.00GHz
stepping        : 3
microcode       : 0xba
cpu MHz         : 3223.671
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 0
cpu cores       : 4
apicid          : 1
initial apicid  : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 22
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb invpcid_single intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bogomips        : 3984.00
clflush size    : 64
cache_alignment : 64
address sizes   : 39 bits physical, 48 bits virtual
power management:

processor       : 5
vendor_id       : GenuineIntel
cpu family      : 6
model           : 94
model name      : Intel(R) Xeon(R) CPU E3-1578L v5 @ 2.00GHz
stepping        : 3
microcode       : 0xba
cpu MHz         : 3266.484
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 1
cpu cores       : 4
apicid          : 3
initial apicid  : 3
fpu             : yes
fpu_exception   : yes
cpuid level     : 22
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb invpcid_single intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bogomips        : 3984.00
clflush size    : 64
cache_alignment : 64
address sizes   : 39 bits physical, 48 bits virtual
power management:

processor       : 6
vendor_id       : GenuineIntel
cpu family      : 6
model           : 94
model name      : Intel(R) Xeon(R) CPU E3-1578L v5 @ 2.00GHz
stepping        : 3
microcode       : 0xba
cpu MHz         : 3400.234
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 2
cpu cores       : 4
apicid          : 5
initial apicid  : 5
fpu             : yes
fpu_exception   : yes
cpuid level     : 22
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb invpcid_single intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bogomips        : 3984.00
clflush size    : 64
cache_alignment : 64
address sizes   : 39 bits physical, 48 bits virtual
power management:

processor       : 7
vendor_id       : GenuineIntel
cpu family      : 6
model           : 94
model name      : Intel(R) Xeon(R) CPU E3-1578L v5 @ 2.00GHz
stepping        : 3
microcode       : 0xba
cpu MHz         : 3173.593
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 3
cpu cores       : 4
apicid          : 7
initial apicid  : 7
fpu             : yes
fpu_exception   : yes
cpuid level     : 22
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb invpcid_single intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bogomips        : 3984.00
clflush size    : 64
cache_alignment : 64
address sizes   : 39 bits physical, 48 bits virtual
power management:


    centminmodbench.sh results



    All new VPS/Dedicated servers I get, I will run my centminmodbench.sh benchmark script at bench.centminmod.com to fully evaluate the performance of the server. The results are below.

    Highlights would definitely be 10Gbps network speed i.e. Vultr Los Angeles download speed test managed to pull 442MB/s ! The weakness if you must compare to other providers is disk I/O speed with read/write under 200-230MB/s with 45k/65K random IOPs.

    Code (Text):
    -------------------------------------------
centminmodbench.sh 0.9.5
http://bench.centminmod.com
written by: George Liu (eva2000)
http://centminmod.com
-------------------------------------------

-------------------------------------------
System Information
-------------------------------------------

3.10.0-693.17.1.el7.x86_64

CentOS Linux release 7.4.1708 (Core)

Centmin Mod
----------------------------------------------
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                8
On-line CPU(s) list:   0-7
Thread(s) per core:    2
Core(s) per socket:    4
Socket(s):             1
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 94
Model name:            Intel(R) Xeon(R) CPU E3-1578L v5 @ 2.00GHz
Stepping:              3
CPU MHz:               3212.265
CPU max MHz:           3400.0000
CPU min MHz:           800.0000
BogoMIPS:              3984.00
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              8192K
NUMA node0 CPU(s):     0-7
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb invpcid_single intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp

CPU Flags
 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb invpcid_single intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp

----------------------------------------------
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE MAXMHZ    MINMHZ
0   0    0      0    0:0:0:0       yes    3400.0000 800.0000
1   0    0      1    1:1:1:0       yes    3400.0000 800.0000
2   0    0      2    2:2:2:0       yes    3400.0000 800.0000
3   0    0      3    3:3:3:0       yes    3400.0000 800.0000
4   0    0      0    0:0:0:0       yes    3400.0000 800.0000
5   0    0      1    1:1:1:0       yes    3400.0000 800.0000
6   0    0      2    2:2:2:0       yes    3400.0000 800.0000
7   0    0      3    3:3:3:0       yes    3400.0000 800.0000

----------------------------------------------
              total        used        free      shared  buff/cache   available
Mem:          32084        1421       30201          33         460       30267
Low:          32084        1882       30201
High:             0           0           0
Swap:          1949           0        1949

----------------------------------------------
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda3       219G  5.5G  202G   3% /
devtmpfs         16G  4.0K   16G   1% /dev
tmpfs            16G     0   16G   0% /dev/shm
tmpfs            16G   17M   16G   1% /run
tmpfs            16G     0   16G   0% /sys/fs/cgroup
tmpfs           3.2G     0  3.2G   0% /run/user/0
tmpfs            16G  332K   16G   1% /tmp

    Code (Text):
    -------------------------------------------
disk ioping tests
-------------------------------------------

Running IOPing I/O benchmark...

IOPing I/O: ./ioping -c 10 .
4.0 KiB from . (ext4 /dev/sda3): request=1 time=196 us
4.0 KiB from . (ext4 /dev/sda3): request=2 time=216 us
4.0 KiB from . (ext4 /dev/sda3): request=3 time=226 us
4.0 KiB from . (ext4 /dev/sda3): request=4 time=207 us
4.0 KiB from . (ext4 /dev/sda3): request=5 time=210 us
4.0 KiB from . (ext4 /dev/sda3): request=6 time=226 us
4.0 KiB from . (ext4 /dev/sda3): request=7 time=233 us
4.0 KiB from . (ext4 /dev/sda3): request=8 time=199 us
4.0 KiB from . (ext4 /dev/sda3): request=9 time=246 us
4.0 KiB from . (ext4 /dev/sda3): request=10 time=212 us

--- . (ext4 /dev/sda3) ioping statistics ---
10 requests completed in 9.0 s, 4.6 k iops, 18.0 MiB/s
min/avg/max/mdev = 196 us / 217 us / 246 us / 14 us

IOPing seek rate: ./ioping -RD .

--- . (ext4 /dev/sda3) ioping statistics ---
17.6 k requests completed in 3.0 s, 5.9 k iops, 23.0 MiB/s
min/avg/max/mdev = 56 us / 169 us / 2.1 ms / 38 us

IOPing sequential: ./ioping -RL .

--- . (ext4 /dev/sda3) ioping statistics ---
5.0 k requests completed in 3.0 s, 1.7 k iops, 425.5 MiB/s
min/avg/max/mdev = 538 us / 587 us / 6.5 ms / 105 us

IOPing cached: ./ioping -RC .

--- . (ext4 /dev/sda3) ioping statistics ---
4.0 M requests completed in 3.0 s, 1.5 M iops, 5.5 GiB/s
min/avg/max/mdev = 0 us / 0 us / 25 us / 0 us

    Code (Text):
    -------------------------------------------
disk DD tests
-------------------------------------------

dd if=/dev/zero of=sb-io-test bs=1M count=1k conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 4.48637 s, 239 MB/s

dd if=/dev/zero of=sb-io-test bs=64k count=16k conv=fdatasync
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 4.5041 s, 238 MB/s

dd if=/dev/zero of=sb-io-test bs=1M count=1k oflag=dsync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 4.15684 s, 258 MB/s

dd if=/dev/zero of=sb-io-test bs=64k count=16k oflag=dsync
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 6.3402 s, 169 MB/s

    Code (Text):
    -------------------------------------------
disk FIO tests
-------------------------------------------

Running FIO benchmark...

FIO_VERSION = fio-2.0.9

FIO random reads:
randomreads: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.0.9
Starting 1 process
randomreads: Laying out IO file(s) (1 file(s) / 1024MB)

randomreads: (groupid=0, jobs=1): err= 0: pid=7628: Sun Feb 25 19:47:53 2018
  read : io=1024.3MB, bw=183522KB/s, iops=45880 , runt=  5715msec
  cpu          : usr=5.48%, sys=20.46%, ctx=245176, majf=0, minf=83
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=262207/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=1024.3MB, aggrb=183521KB/s, minb=183521KB/s, maxb=183521KB/s, mint=5715msec, maxt=5715msec

Disk stats (read/write):
  sda: ios=259116/2, merge=66/1, ticks=359697/24, in_queue=360025, util=98.31%

FIO random writes:
randomwrites: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.0.9
Starting 1 process

randomwrites: (groupid=0, jobs=1): err= 0: pid=7632: Sun Feb 25 19:47:58 2018
  write: io=1024.3MB, bw=260126KB/s, iops=65031 , runt=  4032msec
  cpu          : usr=6.52%, sys=29.87%, ctx=254620, majf=0, minf=18
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=0/w=262207/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=1024.3MB, aggrb=260125KB/s, minb=260125KB/s, maxb=260125KB/s, mint=4032msec, maxt=4032msec

Disk stats (read/write):
  sda: ios=0/253692, merge=0/66, ticks=0/248203, in_queue=248405, util=97.55%

    Code (Text):
    -------------------------------------------
ramdisk DD tests - memory bandwidth
-------------------------------------------

dd if=/dev/zero of=sb-io-test bs=128k count=1k conv=fdatasync
1024+0 records in
1024+0 records out
134217728 bytes (134 MB) copied, 0.0338726 s, 4.0 GB/s

dd if=/dev/zero of=sb-io-test bs=8k count=16k conv=fdatasync
16384+0 records in
16384+0 records out
134217728 bytes (134 MB) copied, 0.0412279 s, 3.3 GB/s

dd if=/dev/zero of=sb-io-test bs=128k count=1k oflag=dsync
1024+0 records in
1024+0 records out
134217728 bytes (134 MB) copied, 0.0333627 s, 4.0 GB/s

dd if=/dev/zero of=sb-io-test bs=8k count=16k oflag=dsync
16384+0 records in
16384+0 records out
134217728 bytes (134 MB) copied, 0.0406788 s, 3.3 GB/s

    Code (Text):
    -------------------------------------------
ramdisk ioping tests - memory bandwidth
-------------------------------------------

Running IOPing I/O ramdisk benchmark...

IOPing I/O: ./ioping -c 10 .
4.0 KiB from . (tmpfs tmpfs): request=1 time=17 us
4.0 KiB from . (tmpfs tmpfs): request=2 time=20 us
4.0 KiB from . (tmpfs tmpfs): request=3 time=3 us
4.0 KiB from . (tmpfs tmpfs): request=4 time=4 us
4.0 KiB from . (tmpfs tmpfs): request=5 time=4 us
4.0 KiB from . (tmpfs tmpfs): request=6 time=4 us
4.0 KiB from . (tmpfs tmpfs): request=7 time=4 us
4.0 KiB from . (tmpfs tmpfs): request=8 time=4 us
4.0 KiB from . (tmpfs tmpfs): request=9 time=3 us
4.0 KiB from . (tmpfs tmpfs): request=10 time=4 us

--- . (tmpfs tmpfs) ioping statistics ---
10 requests completed in 9.0 s, 149.3 k iops, 583.0 MiB/s
min/avg/max/mdev = 3 us / 6 us / 20 us / 5 us

IOPing seek rate: ./ioping -R .

--- . (tmpfs tmpfs) ioping statistics ---
1.8 M requests completed in 3.0 s, 1.4 M iops, 5.5 GiB/s
min/avg/max/mdev = 0 us / 0 us / 22 us / 0 us

IOPing sequential: ./ioping -RL .

--- . (tmpfs tmpfs) ioping statistics ---
73.8 k requests completed in 3.0 s, 40.6 k iops, 9.9 GiB/s
min/avg/max/mdev = 20 us / 24 us / 83 us / 2 us

IOPing cached: ./ioping -RC .

--- . (tmpfs tmpfs) ioping statistics ---
3.8 M requests completed in 3.0 s, 1.4 M iops, 5.2 GiB/s
min/avg/max/mdev = 0 us / 0 us / 60 us / 0 us

    Code (Text):
    
-------------------------------------------
Running bandwidth benchmark...
-------------------------------------------

----------------------------------------------
Download from Cachefly (http://cachefly.cachefly.net/100mb.test)
Download Cachefly: 99.8MB/s

-------------------------------------------
USA bandwidth tests...
-------------------------------------------

----------------------------------------------
Download from Linode, Atlanta, GA, USA (http://speedtest.atlanta.linode.com/100MB-atlanta.bin)
Download Linode, Atlanta, GA, USA: 43.5MB/s
----------------------------------------------
Download from Linode, Dallas, TX, USA (http://speedtest.dallas.linode.com/100MB-dallas.bin)
Download Linode, Dallas, TX, USA: 69.1MB/s
----------------------------------------------
Download from Leaseweb, Manassas, VA, USA (http://mirror.us.leaseweb.net/speedtest/100mb.bin)
Download Leaseweb, Manassas, VA, USA: 49.2MB/s
----------------------------------------------
Download from Softlayer, Seattle, WA, USA (http://speedtest.sea01.softlayer.com/downloads/test100.zip)
Download Softlayer, Seattle, WA, USA: 58.9MB/s
----------------------------------------------
Download from Softlayer, San Jose, CA, USA (http://speedtest.sjc01.softlayer.com/downloads/test100.zip)
Download Softlayer, San Jose, CA, USA: 91.6MB/s
----------------------------------------------
Download from Softlayer, Washington, DC, USA (http://speedtest.wdc01.softlayer.com/downloads/test100.zip)
Download Softlayer, Washington, DC, USA: 15.7MB/s
----------------------------------------------
Download from VersaWeb, Las Vegas, Nevada (http://104.143.15.254/100MB.test)
Download VersaWeb, Las Vegas, Nevada: 45.4MB/s
----------------------------------------------
Download from OVH, BHS, Canada (http://bhs.proof.ovh.net/files/100Mio.dat)
Download OVH, BHS, Canada: 14.2MB/s
----------------------------------------------
Download from Vultr, Los Angeles, California (http://lax-ca-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Los Angeles, California: 442MB/s
----------------------------------------------
Download from Vultr, Seattle, Washington (http://wa-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Seattle, Washington: 66.3MB/s
----------------------------------------------
Download from Vultr, Dallas, Texas (http://tx-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Dallas, Texas: 85.0MB/s
----------------------------------------------
Download from Vultr, Chicago, Illinois (http://il-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Chicago, Illinois: 38.5MB/s
----------------------------------------------
Download from Vultr, Atlanta, Georgia (http://ga-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Atlanta, Georgia: 46.7MB/s
----------------------------------------------
Download from Vultr, Miami, Florida (http://fl-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Miami, Florida: 37.6MB/s
----------------------------------------------
Download from Vultr, New York / New Jersey (http://nj-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, New York / New Jersey: 27.0MB/s

-------------------------------------------
Asia bandwidth tests...
-------------------------------------------

----------------------------------------------
Download from Linode, Tokyo, JP (http://speedtest.tokyo.linode.com/100MB-tokyo.bin)
Download Linode, Tokyo, JP: 20.4MB/s
----------------------------------------------
Download from Softlayer, Singapore (http://speedtest.sng01.softlayer.com/downloads/test100.zip)
Download Softlayer, Singapore: 10.2MB/s
----------------------------------------------
Download from Vultr, Tokyo, Japan (http://hnd-jp-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Tokyo, Japan: 20.6MB/s

-------------------------------------------
Europe bandwidth tests...
-------------------------------------------

----------------------------------------------
Download from Linode, London, UK (http://speedtest.london.linode.com/100MB-london.bin)
Download Linode, London, UK: 15.5MB/s
----------------------------------------------
Download from OVH, Paris, France (http://proof.ovh.net/files/100Mio.dat)
Download OVH, Paris, France: 13.4MB/s
----------------------------------------------
Download from SmartDC, Rotterdam, Netherlands (http://mirror.i3d.net/100mb.bin)
Download SmartDC, Rotterdam, Netherlands: 8.93MB/s
----------------------------------------------
Download from Vultr, Amsterdam, Netherlands (http://ams-nl-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Amsterdam, Netherlands: 13.0MB/s
----------------------------------------------
Download from Vultr, London, UK (http://lon-gb-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, London, UK: 15.6MB/s
----------------------------------------------
Download from Vultr, Paris, France (http://par-fr-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Paris, France: 15.5MB/s

-------------------------------------------
Australia bandwidth tests...
-------------------------------------------

----------------------------------------------
Download from Vultr, Sydney, Australia (http://syd-au-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Sydney, Australia: 14.0MB/s

    Code (Text):
    
-------------------------------------------
Running Axel multi-threaded bandwidth benchmark...
-------------------------------------------

----------------------------------------------
Axel Download from Cachefly (http://cachefly.cachefly.net/100mb.test)
Axel Download Cachefly: 184.47MB/s

-------------------------------------------
USA bandwidth tests...
-------------------------------------------

----------------------------------------------
Axel Download from Linode, Atlanta, GA, USA (http://speedtest.atlanta.linode.com/100MB-atlanta.bin)
Axel Download Linode, Atlanta, GA, USA: 55.35MB/s
----------------------------------------------
Axel Download from Linode, Dallas, TX, USA (http://speedtest.dallas.linode.com/100MB-dallas.bin)
Axel Download Linode, Dallas, TX, USA: 105.50MB/s
----------------------------------------------
Axel Download from Leaseweb, Manassas, VA, USA (http://mirror.us.leaseweb.net/speedtest/100mb.bin)
Axel Download Leaseweb, Manassas, VA, USA: 39.98MB/s
----------------------------------------------
Axel Download from Softlayer, Seattle, WA, USA (http://speedtest.sea01.softlayer.com/downloads/test100.zip)
Axel Download Softlayer, Seattle, WA, USA: 49.00MB/s
----------------------------------------------
Axel Download from Softlayer, San Jose, CA, USA (http://speedtest.sjc01.softlayer.com/downloads/test100.zip)
Axel Download Softlayer, San Jose, CA, USA: 78.00MB/s
----------------------------------------------
Axel Download from Softlayer, Washington, DC, USA (http://speedtest.wdc01.softlayer.com/downloads/test100.zip)
Axel Download Softlayer, Washington, DC, USA: 18.43MB/s
----------------------------------------------
Axel Download from VersaWeb, Las Vegas, Nevada (http://104.143.15.254/100MB.test)
Axel Download VersaWeb, Las Vegas, Nevada: 89.27MB/s
----------------------------------------------
Axel Download from OVH, BHS, Canada (http://bhs.proof.ovh.net/files/100Mio.dat)
Axel Download OVH, BHS, Canada: 5.51MB/s
----------------------------------------------
Axel Download from Vultr, Los Angeles, California (http://lax-ca-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Los Angeles, California: 243.04MB/s
----------------------------------------------
Axel Download from Vultr, Seattle, Washington (http://wa-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Seattle, Washington: 82.23MB/s
----------------------------------------------
Axel Download from Vultr, Dallas, Texas (http://tx-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Dallas, Texas: 148.27MB/s
----------------------------------------------
Axel Download from Vultr, Chicago, Illinois (http://il-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Chicago, Illinois: 74.14MB/s
----------------------------------------------
Axel Download from Vultr, Atlanta, Georgia (http://ga-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Atlanta, Georgia: 85.74MB/s
----------------------------------------------
Axel Download from Vultr, Miami, Florida (http://fl-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Miami, Florida: 66.54MB/s
----------------------------------------------
Axel Download from Vultr, New York / New Jersey (http://nj-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, New York / New Jersey: 50.70MB/s

-------------------------------------------
Asia bandwidth tests...
-------------------------------------------

----------------------------------------------
Axel Download from Linode, Tokyo, JP (http://speedtest.tokyo.linode.com/100MB-tokyo.bin)
Axel Download Linode, Tokyo, JP: 12.35MB/s
----------------------------------------------
Axel Download from Softlayer, Singapore (http://speedtest.sng01.softlayer.com/downloads/test100.zip)
Axel Download Softlayer, Singapore: 7.78MB/s
----------------------------------------------
Axel Download from Vultr, Tokyo, Japan (http://hnd-jp-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Tokyo, Japan: 31.69MB/s

-------------------------------------------
Europe bandwidth tests...
-------------------------------------------

----------------------------------------------
Axel Download from Linode, London, UK (http://speedtest.london.linode.com/100MB-london.bin)
Axel Download Linode, London, UK: 25.59MB/s
----------------------------------------------
Axel Download from OVH, Paris, France (http://proof.ovh.net/files/100Mio.dat)
Axel Download OVH, Paris, France: 3.83MB/s
----------------------------------------------
Axel Download from SmartDC, Rotterdam, Netherlands (http://mirror.i3d.net/100mb.bin)
Axel Download SmartDC, Rotterdam, Netherlands: 3.49MB/s
----------------------------------------------
Axel Download from Vultr, Amsterdam, Netherlands (http://ams-nl-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Amsterdam, Netherlands: 15.83MB/s
----------------------------------------------
Axel Download from Vultr, London, UK (http://lon-gb-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, London, UK: 23.96MB/s
----------------------------------------------
Axel Download from Vultr, Paris, France (http://par-fr-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Paris, France: 21.42MB/s

-------------------------------------------
Australia bandwidth tests...
-------------------------------------------

----------------------------------------------
Axel Download from Vultr, Sydney, Australia (http://syd-au-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Sydney, Australia: 20.79MB/s

    Code (Text):
    
-------------------------------------------
Running ping tests...
-------------------------------------------

----------------------------------------------
Pings (cachefly.cachefly.net):
PING vip1.g5.cachefly.net (205.234.175.175) 56(84) bytes of data.
64 bytes from vip1.G-anycast1.cachefly.net (205.234.175.175): icmp_seq=1 ttl=61 time=0.296 ms
64 bytes from vip1.G-anycast1.cachefly.net (205.234.175.175): icmp_seq=2 ttl=61 time=0.282 ms
64 bytes from vip1.G-anycast1.cachefly.net (205.234.175.175): icmp_seq=3 ttl=61 time=0.203 ms

--- vip1.g5.cachefly.net ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 0.203/0.260/0.296/0.043 ms

----------------------------------------------
Pings (syd-au-ping.vultr.com):
PING syd-au-ping.vultr.com (108.61.212.117) 56(84) bytes of data.
64 bytes from 108.61.212.117.vultr.com (108.61.212.117): icmp_seq=1 ttl=52 time=151 ms
64 bytes from 108.61.212.117.vultr.com (108.61.212.117): icmp_seq=2 ttl=52 time=150 ms
64 bytes from 108.61.212.117.vultr.com (108.61.212.117): icmp_seq=3 ttl=52 time=150 ms

--- syd-au-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 150.780/150.927/151.192/0.486 ms

----------------------------------------------
Pings (hnd-jp-ping.vultr.com):
PING hnd-jp-ping.vultr.com (108.61.201.151) 56(84) bytes of data.
64 bytes from 108.61.201.151.vultr.com (108.61.201.151): icmp_seq=1 ttl=53 time=107 ms
64 bytes from 108.61.201.151.vultr.com (108.61.201.151): icmp_seq=2 ttl=53 time=107 ms
64 bytes from 108.61.201.151.vultr.com (108.61.201.151): icmp_seq=3 ttl=53 time=107 ms

--- hnd-jp-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 107.782/107.843/107.943/0.071 ms

----------------------------------------------
Pings (lax-ca-us-ping.vultr.com):
PING lax-ca-us-ping.vultr.com (108.61.219.200) 56(84) bytes of data.
64 bytes from 108.61.219.200.vultr.com (108.61.219.200): icmp_seq=1 ttl=57 time=0.536 ms
64 bytes from 108.61.219.200.vultr.com (108.61.219.200): icmp_seq=2 ttl=57 time=0.569 ms
64 bytes from 108.61.219.200.vultr.com (108.61.219.200): icmp_seq=3 ttl=57 time=0.463 ms

--- lax-ca-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 0.463/0.522/0.569/0.051 ms

----------------------------------------------
Pings (wa-us-ping.vultr.com):
PING wa-us-ping.vultr.com (108.61.194.105) 56(84) bytes of data.
64 bytes from 108.61.194.105.vultr.com (108.61.194.105): icmp_seq=1 ttl=53 time=33.8 ms
64 bytes from 108.61.194.105.vultr.com (108.61.194.105): icmp_seq=2 ttl=53 time=33.6 ms
64 bytes from 108.61.194.105.vultr.com (108.61.194.105): icmp_seq=3 ttl=53 time=33.7 ms

--- wa-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 33.653/33.744/33.809/0.066 ms

----------------------------------------------
Pings (tx-us-ping.vultr.com):
PING tx-us-ping.vultr.com (108.61.224.175) 56(84) bytes of data.
64 bytes from 108.61.224.175.vultr.com (108.61.224.175): icmp_seq=1 ttl=54 time=27.5 ms
64 bytes from 108.61.224.175.vultr.com (108.61.224.175): icmp_seq=2 ttl=54 time=27.6 ms
64 bytes from 108.61.224.175.vultr.com (108.61.224.175): icmp_seq=3 ttl=54 time=27.8 ms

--- tx-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 27.597/27.679/27.817/0.098 ms

----------------------------------------------
Pings (il-us-ping.vultr.com):
PING il-us-ping.vultr.com (107.191.51.12) 56(84) bytes of data.
64 bytes from 107.191.51.12.vultr.com (107.191.51.12): icmp_seq=1 ttl=56 time=59.3 ms
64 bytes from 107.191.51.12.vultr.com (107.191.51.12): icmp_seq=2 ttl=56 time=59.3 ms
64 bytes from 107.191.51.12.vultr.com (107.191.51.12): icmp_seq=3 ttl=56 time=59.3 ms

--- il-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 59.307/59.321/59.344/0.199 ms

----------------------------------------------
Pings (ga-us-ping.vultr.com):
PING ga-us-ping.vultr.com (108.61.193.166) 56(84) bytes of data.
64 bytes from 108.61.193.166.vultr.com (108.61.193.166): icmp_seq=1 ttl=51 time=50.1 ms
64 bytes from 108.61.193.166.vultr.com (108.61.193.166): icmp_seq=2 ttl=51 time=50.2 ms
64 bytes from 108.61.193.166.vultr.com (108.61.193.166): icmp_seq=3 ttl=51 time=50.2 ms

--- ga-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 50.183/50.216/50.246/0.260 ms

----------------------------------------------
Pings (fl-us-ping.vultr.com):
PING fl-us-ping.vultr.com (104.156.244.232) 56(84) bytes of data.
64 bytes from 104.156.244.232 (104.156.244.232): icmp_seq=1 ttl=52 time=62.5 ms
64 bytes from 104.156.244.232 (104.156.244.232): icmp_seq=2 ttl=52 time=62.4 ms
64 bytes from 104.156.244.232 (104.156.244.232): icmp_seq=3 ttl=52 time=62.5 ms

--- fl-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 62.487/62.541/62.588/0.291 ms

----------------------------------------------
Pings (nj-us-ping.vultr.com):
PING nj-us-ping.vultr.com (108.61.149.182) 56(84) bytes of data.
64 bytes from 108.61.149.182.vultr.com (108.61.149.182): icmp_seq=1 ttl=52 time=84.5 ms
64 bytes from 108.61.149.182.vultr.com (108.61.149.182): icmp_seq=2 ttl=52 time=84.5 ms
64 bytes from 108.61.149.182.vultr.com (108.61.149.182): icmp_seq=3 ttl=52 time=84.7 ms

--- nj-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 84.539/84.612/84.713/0.073 ms

----------------------------------------------
Pings (fra-de-ping.vultr.com):
PING fra-de-ping.vultr.com (108.61.210.117) 56(84) bytes of data.
64 bytes from 108.61.210.117.vultr.com (108.61.210.117): icmp_seq=1 ttl=55 time=163 ms
64 bytes from 108.61.210.117.vultr.com (108.61.210.117): icmp_seq=2 ttl=55 time=163 ms
64 bytes from 108.61.210.117.vultr.com (108.61.210.117): icmp_seq=3 ttl=55 time=163 ms

--- fra-de-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 163.749/163.817/163.895/0.335 ms

----------------------------------------------
Pings (ams-nl-ping.vultr.com):
PING ams-nl-ping.vultr.com (108.61.198.102) 56(84) bytes of data.
64 bytes from 108.61.198.102.vultr.com (108.61.198.102): icmp_seq=1 ttl=51 time=169 ms
64 bytes from 108.61.198.102.vultr.com (108.61.198.102): icmp_seq=2 ttl=51 time=165 ms
64 bytes from 108.61.198.102.vultr.com (108.61.198.102): icmp_seq=3 ttl=51 time=170 ms

--- ams-nl-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 165.045/168.531/170.664/2.485 ms

----------------------------------------------
Pings (lon-gb-ping.vultr.com):
PING lon-gb-ping.vultr.com (108.61.196.101) 56(84) bytes of data.
64 bytes from 108.61.196.101.vultr.com (108.61.196.101): icmp_seq=1 ttl=51 time=133 ms
64 bytes from 108.61.196.101.vultr.com (108.61.196.101): icmp_seq=2 ttl=51 time=133 ms
64 bytes from 108.61.196.101.vultr.com (108.61.196.101): icmp_seq=3 ttl=51 time=133 ms

--- lon-gb-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 133.801/133.813/133.831/0.013 ms

----------------------------------------------
Pings (par-fr-ping.vultr.com):
PING par-fr-ping.vultr.com (108.61.209.127) 56(84) bytes of data.
64 bytes from 108.61.209.127.vultr.com (108.61.209.127): icmp_seq=1 ttl=51 time=163 ms
64 bytes from 108.61.209.127.vultr.com (108.61.209.127): icmp_seq=2 ttl=51 time=163 ms
64 bytes from 108.61.209.127.vultr.com (108.61.209.127): icmp_seq=3 ttl=51 time=163 ms

--- par-fr-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 163.630/163.670/163.708/0.031 ms

----------------------------------------------
Pings (VersaWeb Las Vegas):
PING 104.143.15.254 (104.143.15.254) 56(84) bytes of data.
64 bytes from 104.143.15.254: icmp_seq=1 ttl=54 time=7.97 ms
64 bytes from 104.143.15.254: icmp_seq=2 ttl=54 time=7.93 ms
64 bytes from 104.143.15.254: icmp_seq=3 ttl=54 time=7.93 ms

--- 104.143.15.254 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 7.938/7.951/7.977/0.018 ms

----------------------------------------------
Pings (VersaWeb Seattle):
PING 76.164.234.1 (76.164.234.1) 56(84) bytes of data.
64 bytes from 76.164.234.1: icmp_seq=1 ttl=57 time=38.2 ms
64 bytes from 76.164.234.1: icmp_seq=2 ttl=57 time=38.2 ms
64 bytes from 76.164.234.1: icmp_seq=3 ttl=57 time=38.1 ms

--- 76.164.234.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 38.156/38.212/38.276/0.167 ms

----------------------------------------------
Pings (OVH Canada):
PING bhs.proof.ovh.net (192.99.19.165) 56(84) bytes of data.
64 bytes from bhs.proof.ovh.net (192.99.19.165): icmp_seq=1 ttl=54 time=74.6 ms
64 bytes from bhs.proof.ovh.net (192.99.19.165): icmp_seq=2 ttl=54 time=74.6 ms
64 bytes from bhs.proof.ovh.net (192.99.19.165): icmp_seq=3 ttl=54 time=74.6 ms

--- bhs.proof.ovh.net ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 74.624/74.638/74.656/0.013 ms

----------------------------------------------
Pings (10gbps.io Frankfurt):
PING 195.181.174.20 (195.181.174.20) 56(84) bytes of data.
64 bytes from 195.181.174.20: icmp_seq=1 ttl=56 time=181 ms
64 bytes from 195.181.174.20: icmp_seq=2 ttl=56 time=180 ms
64 bytes from 195.181.174.20: icmp_seq=3 ttl=56 time=180 ms

--- 195.181.174.20 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 180.824/180.887/181.001/0.356 ms

----------------------------------------------
Pings (10gbps.io London):
PING 185.59.221.20 (185.59.221.20) 56(84) bytes of data.
64 bytes from 185.59.221.20: icmp_seq=1 ttl=56 time=182 ms
64 bytes from 185.59.221.20: icmp_seq=2 ttl=56 time=424 ms
64 bytes from 185.59.221.20: icmp_seq=3 ttl=56 time=182 ms

--- 185.59.221.20 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 182.211/262.941/424.251/114.064 ms

----------------------------------------------
Pings (10gbps.io Paris):
PING 185.93.2.193 (185.93.2.193) 56(84) bytes of data.
64 bytes from 185.93.2.193: icmp_seq=1 ttl=56 time=182 ms
64 bytes from 185.93.2.193: icmp_seq=2 ttl=56 time=182 ms
64 bytes from 185.93.2.193: icmp_seq=3 ttl=56 time=182 ms

--- 185.93.2.193 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 182.887/182.923/182.979/0.040 ms

----------------------------------------------
Pings (10gbps.io Singapore):
PING 203.174.85.38 (203.174.85.38) 56(84) bytes of data.
64 bytes from 203.174.85.38: icmp_seq=1 ttl=57 time=176 ms
64 bytes from 203.174.85.38: icmp_seq=2 ttl=57 time=176 ms
64 bytes from 203.174.85.38: icmp_seq=3 ttl=57 time=176 ms

--- 203.174.85.38 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 176.685/176.729/176.765/0.486 ms

----------------------------------------------
Pings (10gbps.io Stockholm):
PING 185.76.9.10 (185.76.9.10) 56(84) bytes of data.
64 bytes from 185.76.9.10: icmp_seq=1 ttl=56 time=191 ms
64 bytes from 185.76.9.10: icmp_seq=2 ttl=56 time=191 ms
64 bytes from 185.76.9.10: icmp_seq=3 ttl=56 time=191 ms

--- 185.76.9.10 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 191.238/191.268/191.325/0.506 ms

----------------------------------------------
Pings (10gbps.io Moscow):
PING 87.245.204.20 (87.245.204.20) 56(84) bytes of data.
64 bytes from 87.245.204.20: icmp_seq=1 ttl=56 time=210 ms
64 bytes from 87.245.204.20: icmp_seq=2 ttl=56 time=210 ms
64 bytes from 87.245.204.20: icmp_seq=3 ttl=56 time=210 ms

--- 87.245.204.20 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 210.037/210.122/210.280/0.111 ms

----------------------------------------------
Pings (10gbps.io Miami):
PING 195.181.162.195 (195.181.162.195) 56(84) bytes of data.
64 bytes from 195.181.162.195: icmp_seq=1 ttl=55 time=81.1 ms
64 bytes from 195.181.162.195: icmp_seq=2 ttl=55 time=81.3 ms
64 bytes from 195.181.162.195: icmp_seq=3 ttl=55 time=81.1 ms

--- 195.181.162.195 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 81.130/81.210/81.348/0.343 ms

    Code (Text):
    
-------------------------------------------
Running mtr tests...
-------------------------------------------

----------------------------------------------
mtr --report -w 200 --report-cycles=10 cachefly.cachefly.net
Start: Sun Feb 25 19:53:36 2018
HOST: hostname         Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                        0.0%    10   20.8  19.0  11.6  21.3   2.8
  2.|-- vl101.fr3.lax.llnw.net         0.0%    10    0.2   0.2   0.2   0.3   0.0
  3.|-- ae10-352.cr0-lax1.ip4.gtt.net  0.0%    10    0.3   0.4   0.3   0.8   0.0
  4.|-- ???                           100.0    10    0.0   0.0   0.0   0.0   0.0
  5.|-- vip1.G-anycast1.cachefly.net   0.0%    10    0.2   0.3   0.2   0.5   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.atlanta.linode.com
Start: Sun Feb 25 19:53:51 2018
HOST: hostname                             Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                            0.0%    10    9.9   8.5   2.6  14.3   3.5
  2.|-- 2607:f4e8:1:110::122                               0.0%    10    0.3   1.3   0.2   9.6   2.9
  3.|-- ix-ae-20-0.tcore1.EQL-Los-Angeles.ipv6.as6453.net  0.0%    10  140.2 140.4 140.2 140.6   0.0
  4.|-- if-ae-30-2.tcore1.LVW-Los-Angeles.ipv6.as6453.net  0.0%    10  142.1 141.9 137.4 147.5   3.4
  5.|-- if-ae-34-2.tcore2.DT8-Dallas.ipv6.as6453.net       0.0%    10  136.7 136.7 136.3 137.6   0.0
  6.|-- if-ae-2-2.tcore1.DT8-Dallas.ipv6.as6453.net        0.0%    10  139.4 142.3 139.4 167.3   8.8
  7.|-- if-ae-27-2.tcore1.A56-Atlanta.ipv6.as6453.net      0.0%    10  137.3 138.3 137.2 145.8   2.6
  8.|-- 2001:5a0:a00:500::96                               0.0%    10   66.6  66.7  66.6  67.0   0.0
  9.|-- 2600:3c02:4444:1::2                                0.0%    10   66.7  66.7  66.5  67.2   0.0
 10.|-- 2600:3c02::4b                                      0.0%    10   66.4  66.1  66.0  66.4   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.dallas.linode.com
Start: Sun Feb 25 19:54:09 2018
HOST: hostname        Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                       0.0%    10    2.7   6.4   1.2  11.1   3.9
  2.|-- 2607:f4e8:1:110::122          0.0%    10    0.3   0.3   0.3   0.8   0.0
  3.|-- ve6.fr4.lax.ipv6.llnw.net     0.0%    10    0.3   0.3   0.2   0.3   0.0
  4.|-- lag23.fr3.phx4.ipv6.llnw.net  0.0%    10    9.4   9.5   9.4   9.7   0.0
  5.|-- 2607:f4e8:121:4000::2         0.0%    10    9.6   9.7   9.5  11.4   0.6
  6.|-- lag38.fr4.dal.ipv6.llnw.net   0.0%    10   32.1  32.2  32.1  32.3   0.0
  7.|-- eqix-ix-da-ipv6.linode.com    0.0%    10   32.8  37.4  32.8  71.9  12.2
  8.|-- 2600:3c00:2222:11::2          0.0%    10   33.9  33.6  33.4  33.9   0.0
  9.|-- speedtest.dallas.linode.com   0.0%    10   33.1  33.0  32.8  33.3   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 mirror.us.leaseweb.net
Start: Sun Feb 25 19:54:24 2018
HOST: hostname                             Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                            0.0%    10   13.0   6.9   2.1  13.0   3.7
  2.|-- 2607:f4e8:1:110::122                               0.0%    10    0.3   0.3   0.2   0.3   0.0
  3.|-- ix-ae-20-0.tcore1.EQL-Los-Angeles.ipv6.as6453.net  0.0%    10  140.5 144.4 140.3 164.3   8.6
  4.|-- if-ae-6-2.tcore2.LVW-Los-Angeles.ipv6.as6453.net   0.0%    10  140.5 141.4 140.4 148.6   2.4
  5.|-- if-ae-36-2.tcore2.AEQ-Ashburn.ipv6.as6453.net      0.0%    10  139.8 140.0 139.8 140.3   0.0
  6.|-- if-ae-2-2.tcore1.AEQ-Ashburn.ipv6.as6453.net       0.0%    10  142.8 142.9 142.4 144.8   0.6
  7.|-- if-ae-32-2.tcore1.VN5-Manassas.ipv6.as6453.net     0.0%    10  147.5 140.2 138.3 147.5   3.7
  8.|-- 2001:5a0:600:400::b2                               0.0%    10   80.4  80.5  80.4  80.6   0.0
  9.|-- 2604:9a00::108:59:15:247                           0.0%    10   79.1  79.0  78.9  79.1   0.0
 10.|-- 2604:9a00:2010:a0b8::5                             0.0%    10   79.9  80.0  79.8  80.1   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.sea01.softlayer.com
Start: Sun Feb 25 19:54:42 2018
HOST: hostname                                             Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                                            0.0%    10    8.6   7.5   2.1  10.4   2.3
  2.|-- 2607:f4e8:1:110::122                                               0.0%    10    0.3   0.4   0.2   1.4   0.0
  3.|-- ve7.fr3.lax5.ipv6.llnw.net                                         0.0%    10    0.5   2.1   0.3   7.7   2.4
  4.|-- 2001:504:13::210:131                                               0.0%    10    0.5   0.9   0.4   2.4   0.3
  5.|-- 2607:f0d0:2:2::24c                                                 0.0%    10    1.4   2.0   1.2   7.7   1.9
  6.|-- 1620.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   10.2   9.8   9.4  10.7   0.3
  7.|-- 0520.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10    9.6   9.8   9.4  10.9   0.3
  8.|-- ce20.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   27.6  27.9  27.3  28.6   0.0
  9.|-- 1630.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   26.5  27.3  26.1  33.0   2.0
 10.|-- po3.bbr01.sr01.sea01.networklayer.com                              0.0%    10   27.3  26.4  26.1  27.3   0.0
 11.|-- po1.fcr01.sr01.sea01.networklayer.com                              0.0%    10   26.6  26.7  26.5  27.5   0.0
 12.|-- softlayer.com                                                      0.0%    10   26.2  26.2  26.1  26.4   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.sjc01.softlayer.com
Start: Sun Feb 25 19:55:03 2018
HOST: hostname                                             Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                                            0.0%    10    1.8   5.4   0.8   9.7   3.4
  2.|-- 2607:f4e8:1:110::122                                               0.0%    10    0.3   0.4   0.2   1.7   0.3
  3.|-- ve7.fr3.lax5.ipv6.llnw.net                                         0.0%    10    0.6   5.0   0.3  14.7   4.9
  4.|-- 2001:504:13::210:131                                               0.0%    10    0.6   0.9   0.4   2.2   0.5
  5.|-- 2607:f0d0:2:2::24c                                                 0.0%    10    1.6   1.6   1.1   4.4   0.9
  6.|-- 1620.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   11.8  12.1  11.4  13.6   0.5
  7.|-- 9520.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   10.9  13.8  10.9  36.9   8.1
  8.|-- 1b10.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   11.2  11.3  11.1  11.8   0.0
  9.|-- 2607:f0d0:2100:1::3                                               10.0%    10   11.3  18.4  11.3  51.1  13.4
 10.|-- softlayer.com                                                      0.0%    10   11.1  11.2  11.1  11.4   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.wdc01.softlayer.com
Start: Sun Feb 25 19:55:20 2018
HOST: hostname                                             Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                                            0.0%    10    6.8   6.0   1.0   9.8   3.1
  2.|-- 2607:f4e8:1:110::122                                               0.0%    10    0.2   0.2   0.2   0.3   0.0
  3.|-- ve7.fr3.lax5.ipv6.llnw.net                                         0.0%    10    6.3   2.3   0.4   9.5   3.2
  4.|-- 2001:504:13::210:131                                               0.0%    10    1.9   1.3   0.5   7.2   2.1
  5.|-- 2607:f0d0:2:2::246                                                 0.0%    10    1.3   4.0   1.2  26.0   7.7
  6.|-- 2607:f0d0:2:2::2f2                                                 0.0%    10   31.3  32.6  31.3  41.8   3.2
  7.|-- d030.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   32.2  33.7  31.5  49.4   5.6
  8.|-- e230.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   31.7  33.0  31.6  36.5   1.6
  9.|-- ba20.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   64.0  65.4  64.0  75.6   3.5
 10.|-- ca20.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   64.5  64.5  64.0  66.9   0.7
 11.|-- 2607:f0d0:2:1::6                                                   0.0%    10   64.0  64.0  63.9  64.4   0.0
 12.|-- po2.dar01.sr01.wdc01.networklayer.com                              0.0%    10   64.2  64.2  64.1  64.3   0.0
 13.|-- po1.fcr01.sr01.wdc01.networklayer.com                              0.0%    10   65.1  65.2  64.4  70.0   1.6
 14.|-- softlayer.com                                                      0.0%    10   64.1  64.3  64.1  64.4   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.tokyo.linode.com
Start: Sun Feb 25 19:55:49 2018
HOST: hostname         Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                        0.0%    10    7.2   5.8   1.2  10.8   3.2
  2.|-- 2607:f4e8:1:110::122           0.0%    10    0.3   0.4   0.3   1.0   0.0
  3.|-- ve7.fr3.lax5.ipv6.llnw.net     0.0%    10    0.4   2.1   0.3   8.0   2.8
  4.|-- tge1-5.fr3.mia1.ipv6.llnw.net  0.0%    10  106.7 106.7 106.6 106.7   0.0
  5.|-- 2001:268:fc02:34::1            0.0%    10  240.7 240.6 240.4 240.7   0.0
  6.|-- 2001:268:fa02:127::1           0.0%    10  245.1 244.3 243.4 245.1   0.3
  7.|-- 2001:268:fa00:27e::2           0.0%    10  242.5 243.9 242.5 252.0   3.0
  8.|-- 6cm-fcu204.int-gw.kddi.ne.jp   0.0%    10  244.4 247.9 243.7 256.8   4.3
  9.|-- 2001:268:f702:6d::2            0.0%    10  243.4 243.7 243.4 243.9   0.0
 10.|-- speedtest.tokyo.linode.com     0.0%    10  245.8 245.9 245.8 246.1   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.sng01.softlayer.com
Start: Sun Feb 25 19:56:06 2018
HOST: hostname                                             Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                                            0.0%    10    1.2  12.5   1.2  57.2  16.1
  2.|-- 2607:f4e8:1:110::122                                               0.0%    10    0.3   0.3   0.2   0.3   0.0
  3.|-- ve6.fr4.lax.ipv6.llnw.net                                          0.0%    10    0.3   0.7   0.3   4.1   1.1
  4.|-- lag16.fr4.nrt1.ipv6.llnw.net                                       0.0%    10  102.5  99.4  98.0 102.6   2.0
  5.|-- tge12-1.fr4.tyo3.ipv6.llnw.net                                     0.0%    10  102.4 103.4  98.4 110.5   4.4
  6.|-- 36351.tyo.equinix.com                                              0.0%    10  155.8 155.9 155.8 156.1   0.0
  7.|-- 8740.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10  158.8 158.3 156.6 159.9   0.8
  8.|-- 1940.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10  227.3 228.3 227.0 239.2   3.8
  9.|-- 9840.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10  227.2 227.3 227.0 228.5   0.3
 10.|-- 1c00.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10  227.1 227.2 226.9 228.5   0.3
 11.|-- 3.00.0000.ip4.static.sl-reverse.com                               10.0%    10  227.0 230.0 227.0 251.6   8.1
 12.|-- 2.00.0000.ip4.static.sl-reverse.com                                0.0%    10  227.0 227.0 226.9 227.2   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.london.linode.com
Start: Sun Feb 25 19:56:25 2018
HOST: hostname                             Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                            0.0%    10    7.2   7.7   3.3  11.4   2.5
  2.|-- 2607:f4e8:1:110::122                               0.0%    10   10.1  11.2   0.3  30.3  11.0
  3.|-- ix-ae-20-0.tcore1.EQL-Los-Angeles.ipv6.as6453.net  0.0%    10  140.1 146.3 140.1 178.1  13.0
  4.|-- if-ae-6-2.tcore2.LVW-Los-Angeles.ipv6.as6453.net   0.0%    10  140.4 146.8 140.4 202.6  19.6
  5.|-- if-ae-36-2.tcore2.AEQ-Ashburn.ipv6.as6453.net      0.0%    10  140.1 140.0 139.9 140.1   0.0
  6.|-- tcore4.NJY-Newark.ipv6.as6453.net                  0.0%    10  140.0 140.3 140.0 140.8   0.0
  7.|-- if-ae-4-3.thar2.NJY-Newark.ipv6.as6453.net         0.0%    10  142.4 142.6 142.3 144.2   0.5
  8.|-- if-ae-1-3.thar1.NJY-Newark.ipv6.as6453.net         0.0%    10  142.8 142.5 142.3 143.0   0.0
  9.|-- if-ae-3-3.tcore3.NJY-Newark.ipv6.as6453.net        0.0%    10  145.8 139.1 137.4 145.8   3.2
 10.|-- if-ae-9-2.tcore1.L78-London.ipv6.as6453.net        0.0%    10  141.7 142.0 141.7 143.4   0.5
 11.|-- if-ae-2-2.tcore2.L78-London.ipv6.as6453.net        0.0%    10  139.3 139.3 139.1 139.9   0.0
 12.|-- if-ae-15-2.tcore2.LDN-London.ipv6.as6453.net       0.0%    10  141.8 142.2 141.8 144.3   0.6
 13.|-- 2a01:3e0:10:2100::66                               0.0%    10  147.9 147.7 147.5 147.9   0.0
 14.|-- 2a01:7e00:7777:1::2                                0.0%    10  139.8 139.7 139.5 139.8   0.0
 15.|-- speedtest.london.linode.com                        0.0%    10  139.2 139.2 139.0 139.4   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 mirror.i3d.net
Start: Sun Feb 25 19:56:45 2018
HOST: hostname            Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                           0.0%    10   36.6   9.7   1.6  36.6  10.1
  2.|-- 2607:f4e8:1:110::122              0.0%    10    1.7   1.8   0.2  13.7   4.2
  3.|-- ve6.fr4.lax.ipv6.llnw.net         0.0%    10    0.3   0.4   0.2   0.9   0.0
  4.|-- lag45.fr4.sjc.ipv6.llnw.net       0.0%    10    6.4   7.0   6.4   8.3   0.6
  5.|-- lag55.fr4.ord.ipv6.llnw.net       0.0%    10   58.1  58.1  58.0  58.3   0.0
  6.|-- lag23.fr4.fra1.ipv6.llnw.net      0.0%    10  173.0 173.2 173.0 173.4   0.0
  7.|-- i3d.ipv6.plix.pl                  0.0%    10  200.6 200.8 199.9 204.8   1.4
  8.|-- 10ge.fr0-wr0.inxn-1.fra.i3d.net  10.0%    10  199.4 200.0 199.4 200.4   0.0
  9.|-- 20ge.cr0-fr0.smartdc.rtd.i3d.net  0.0%    10  212.9 213.2 212.6 214.0   0.0
 10.|-- 80ge.br2-cr0.smartdc.rtd.i3d.net  0.0%    10  212.9 213.3 212.5 214.4   0.3
 11.|-- hosted-by.i3d.net                10.0%    10  212.4 212.9 212.4 213.4   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 syd-au-ping.vultr.com
Start: Sun Feb 25 19:57:04 2018
HOST: hostname                        Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                       0.0%    10   17.3  16.6  12.3  22.1   3.1
  2.|-- vl101.fr3.lax.llnw.net                        0.0%    10    0.3   0.6   0.3   2.7   0.7
  3.|-- ve7.fr3.lax5.llnw.net                         0.0%    10    7.3   3.2   0.3  10.2   3.6
  4.|-- vocus-connect-pty.as4826.any2ix.coresite.com  0.0%    10    1.0   0.8   0.7   1.0   0.0
  5.|-- bundle-150.cor01.lax01.ca.vocus.net           0.0%    10  146.1 145.8 145.8 146.1   0.0
  6.|-- 100g-0-1-0-0.cor01.sjc01.ca.vocus.net         0.0%    10  160.1 160.2 160.0 160.5   0.0
  7.|-- BE-200.cor02.syd04.nsw.VOCUS.net.au           0.0%    10  151.1 150.9 150.7 151.2   0.0
  8.|-- bundle-101.bdr01.syd03.nsw.vocus.net.au       0.0%    10  155.6 155.8 155.6 155.9   0.0
  9.|-- static-73.7.255.49.in-addr.VOCUS.net.au       0.0%    10  155.0 154.9 154.9 155.0   0.0
 10.|-- xe-1-0-1.gw102.sy3.ap.equinix.com             0.0%    10  154.9 155.0 154.9 155.2   0.0
 11.|-- 180.189.25.6                                  0.0%    10  155.1 156.2 155.0 160.5   1.8
 12.|-- vl901-ds1-b5-r1120.aus1.choopa.net            0.0%    10  152.1 165.0 152.0 281.4  40.9
 13.|-- ???                                          100.0    10    0.0   0.0   0.0   0.0   0.0
 14.|-- 108.61.212.117.vultr.com                      0.0%    10  150.9 150.9 150.8 151.3   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 hnd-jp-ping.vultr.com
Start: Sun Feb 25 19:57:21 2018
HOST: hostname                      Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                     0.0%    10   18.0  18.5  12.4  23.2   3.9
  2.|-- vl101.fr3.lax.llnw.net                      0.0%    10    0.3   0.3   0.2   0.3   0.0
  3.|-- ae10-352.cr0-lax1.ip4.gtt.net               0.0%    10    0.3   1.5   0.3  12.5   3.8
  4.|-- et-0-0-67.cr4-lax2.ip4.gtt.net              0.0%    10    0.3   1.8   0.3   5.6   2.0
  5.|-- as2914.cr4-lax2.ip4.gtt.net                 0.0%    10    0.8   0.8   0.6   1.0   0.0
  6.|-- ae-3.r23.lsanca07.us.bb.gin.ntt.net         0.0%    10    0.8   2.2   0.8   7.2   2.2
  7.|-- ae-12.r31.tokyjp05.jp.bb.gin.ntt.net       40.0%    10   99.8  98.8  98.0 101.0   1.1
  8.|-- ae-3.r02.tokyjp05.jp.bb.gin.ntt.net         0.0%    10  104.2 104.2 104.1 104.2   0.0
  9.|-- ce-0-14-0-1.r02.tokyjp05.jp.ce.gin.ntt.net  0.0%    10  105.0 108.3 104.8 120.7   5.9
 10.|-- ???                                        100.0    10    0.0   0.0   0.0   0.0   0.0
 11.|-- vl516-ds1-b5-r2605.tyo1.choopa.net          0.0%    10  105.3 105.3 105.3 105.5   0.0
 12.|-- ???                                        100.0    10    0.0   0.0   0.0   0.0   0.0
 13.|-- 108.61.201.151.vultr.com                    0.0%    10  108.0 108.0 107.9 108.1   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 lax-ca-us-ping.vultr.com
Start: Sun Feb 25 19:57:38 2018
HOST: hostname             Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                            0.0%    10   15.7  22.9  11.6  77.4  19.5
  2.|-- vl101.fr3.lax.llnw.net             0.0%    10    0.3   0.3   0.2   0.5   0.0
  3.|-- ae10-352.cr0-lax1.ip4.gtt.net      0.0%    10    0.3   0.7   0.3   2.6   0.7
  4.|-- xe-11-2-0.cr2-lax2.ip4.gtt.net     0.0%    10    0.4   0.4   0.3   0.4   0.0
  5.|-- as20473-gw.lax20.ip4.gtt.net       0.0%    10    0.5   1.6   0.4   6.7   2.0
  6.|-- vl901-ds1-b5-c705.lax3.choopa.net  0.0%    10    1.0   3.8   1.0  28.1   8.5
  7.|-- ???                               100.0    10    0.0   0.0   0.0   0.0   0.0
  8.|-- 108.61.219.200.vultr.com           0.0%    10    0.7   0.6   0.5   0.7   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 wa-us-ping.vultr.com
Start: Sun Feb 25 19:57:53 2018
HOST: hostname         Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                        0.0%    10   16.4  15.7  11.7  20.5   2.9
  2.|-- vl101.fr3.lax.llnw.net         0.0%    10    0.2   0.3   0.2   0.8   0.0
  3.|-- ae10-352.cr0-lax1.ip4.gtt.net  0.0%    10    0.3   0.5   0.3   2.4   0.6
  4.|-- et-2-3-0.cr3-sea2.ip4.gtt.net  0.0%    10   35.9  35.5  35.4  35.9   0.0
  5.|-- choopa-gw.ip4.gtt.net          0.0%    10   33.8  33.8  33.7  34.1   0.0
  6.|-- ???                           100.0    10    0.0   0.0   0.0   0.0   0.0
  7.|-- 108.61.194.105.vultr.com       0.0%    10   33.8  33.8  33.8  33.9   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 tx-us-ping.vultr.com
Start: Sun Feb 25 19:58:09 2018
HOST: hostname              Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10   20.7  18.3  15.9  22.4   1.9
  2.|-- vl101.fr3.lax.llnw.net              0.0%    10    0.3   0.3   0.2   0.3   0.0
  3.|-- ae10-352.cr0-lax1.ip4.gtt.net       0.0%    10    0.3   0.5   0.3   1.3   0.0
  4.|-- xe-2-0-1.cr0-dal2.ip4.gtt.net       0.0%    10   27.6  29.8  27.3  49.2   6.8
  5.|-- ip4.gtt.net                        30.0%    10   27.7  28.1  27.6  28.9   0.0
  6.|-- vl902-ds1-b5-02.04.dal4.choopa.net 70.0%    10  149.0 162.0 149.0 174.3  12.6
  7.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
  8.|-- 108.61.224.175.vultr.com            0.0%    10   27.6  27.9  27.6  29.6   0.6

----------------------------------------------
mtr --report -w 200 --report-cycles=10 il-us-ping.vultr.com
Start: Sun Feb 25 19:58:24 2018
HOST: hostname              Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10   20.7  17.4  12.5  21.2   2.8
  2.|-- vl101.fr3.lax.llnw.net              0.0%    10    0.2   0.4   0.2   1.5   0.3
  3.|-- ae10-352.cr0-lax1.ip4.gtt.net       0.0%    10    1.0   0.5   0.3   1.1   0.0
  4.|-- xe-0-3-0.cr8-chi1.ip4.gtt.net       0.0%    10   55.4  55.5  55.4  55.8   0.0
  5.|-- ip4.gtt.net                         0.0%    10   61.1  63.2  60.7  72.6   4.3
  6.|-- vl820-ds1-b5-gj178.chi2.choopa.net  0.0%    10   59.8  59.9  59.7  60.1   0.0
  7.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
  8.|-- 107.191.51.12.vultr.com             0.0%    10   59.5  59.5  59.4  59.6   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 ga-us-ping.vultr.com
Start: Sun Feb 25 19:58:39 2018
HOST: hostname               Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                              0.0%    10   15.1  22.8  12.6  65.5  15.2
  2.|-- vl101.fr3.lax.llnw.net               0.0%    10    0.2   0.3   0.2   0.3   0.0
  3.|-- las-b21-link.telia.net               0.0%    10    0.9   1.0   0.7   2.1   0.3
  4.|-- dls-b21-link.telia.net               0.0%    10   30.1  32.1  29.9  45.3   4.6
  5.|-- atl-b22-link.telia.net               0.0%    10   50.4  50.6  50.4  51.2   0.0
  6.|-- atl-b21-link.telia.net               0.0%    10   50.3  52.6  50.2  73.6   7.3
  7.|-- vultr-ic-319646-atl-b21.c.telia.net  0.0%    10   53.0  52.3  50.2  56.6   2.4
  8.|-- vl901-ds1-b5-r7.atl2.choopa.net      0.0%    10   50.8  50.8  50.8  51.0   0.0
  9.|-- ???                                 100.0    10    0.0   0.0   0.0   0.0   0.0
 10.|-- 108.61.193.166.vultr.com             0.0%    10   50.3  50.3  50.2  50.4   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 fl-us-ping.vultr.com
Start: Sun Feb 25 19:58:57 2018
HOST: hostname                           Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                          0.0%    10   19.4  17.0  11.5  27.7   4.3
  2.|-- vl101.fr3.lax.llnw.net                           0.0%    10    0.3   0.4   0.2   1.9   0.3
  3.|-- ix-ae-20-0.tcore1.EQL-Los-Angeles.as6453.net     0.0%    10   33.5  33.3  33.2  33.5   0.0
  4.|-- if-ae-6-2.tcore2.LVW-Los-Angeles.as6453.net      0.0%    10   34.4  38.3  33.7  51.6   6.5
  5.|-- te-0-5-0-8-pe01.losangeles.ca.ibone.comcast.net  0.0%    10    1.0   1.0   0.9   1.3   0.0
  6.|-- be-11599-cr02.losangeles.ca.ibone.comcast.net    0.0%    10    2.9   2.2   1.4   3.0   0.0
  7.|-- be-11524-cr02.dallas.tx.ibone.comcast.net        0.0%    10   34.4  33.9  32.7  34.7   0.3
  8.|-- be-12224-cr02.miami.fl.ibone.comcast.net         0.0%    10   62.8  62.7  62.1  63.5   0.0
  9.|-- be-12274-pe01.nota.fl.ibone.comcast.net          0.0%    10   61.4  61.2  61.0  61.5   0.0
 10.|-- 96-87-8-30-static.hfc.comcastbusiness.net        0.0%    10   63.2  61.9  61.5  63.2   0.3
 11.|-- ???                                             100.0    10    0.0   0.0   0.0   0.0   0.0
 12.|-- ???                                             100.0    10    0.0   0.0   0.0   0.0   0.0
 13.|-- 104.156.244.232                                  0.0%    10   62.7  62.7  62.5  63.4   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 nj-us-ping.vultr.com
Start: Sun Feb 25 19:59:13 2018
HOST: hostname              Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10   16.8  21.8  13.9  58.4  13.2
  2.|-- vl101.fr3.lax.llnw.net              0.0%    10    1.5   0.5   0.2   1.6   0.0
  3.|-- las-b21-link.telia.net              0.0%    10    0.9   0.8   0.7   1.0   0.0
  4.|-- nyk-bb4-link.telia.net              0.0%    10   84.3  84.4  84.3  84.8   0.0
  5.|-- nyk-b2-link.telia.net               0.0%    10   84.3  85.3  84.1  95.0   3.3
  6.|-- vultr-ic-318452-nyk-b2.c.telia.net  0.0%    10   84.5  86.7  84.5  92.0   2.6
  7.|-- vl39-br1.pnj1.choopa.net            0.0%    10   88.4  88.5  84.6  95.0   3.8
  8.|-- vl139-c4-3-b5-2.pnj1.choopa.net     0.0%    10   85.3  85.2  85.1  85.4   0.0
  9.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
 10.|-- 108.61.149.182.vultr.com            0.0%    10   84.7  84.6  84.6  84.7   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 fra-de-ping.vultr.com
Start: Sun Feb 25 19:59:29 2018
HOST: hostname         Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                        0.0%    10   22.4  19.9  12.4  22.9   3.5
  2.|-- vl101.fr3.lax.llnw.net         0.0%    10    0.2   0.3   0.2   0.3   0.0
  3.|-- ae10-352.cr0-lax1.ip4.gtt.net  0.0%    10    0.3   0.6   0.3   2.4   0.5
  4.|-- xe-8-0-2.cr1-fra2.ip4.gtt.net  0.0%    10  163.8 163.8 163.6 163.9   0.0
  5.|-- ip4.gtt.net                    0.0%    10  164.0 164.9 163.7 173.4   2.9
  6.|-- ???                           100.0    10    0.0   0.0   0.0   0.0   0.0
  7.|-- 108.61.210.117.vultr.com       0.0%    10  163.9 163.8 163.7 164.0   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 ams-nl-ping.vultr.com
Start: Sun Feb 25 19:59:44 2018
HOST: hostname              Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10   19.8  18.8  13.5  28.8   4.8
  2.|-- vl101.fr3.lax.llnw.net              0.0%    10    0.3   0.3   0.2   0.7   0.0
  3.|-- las-b21-link.telia.net              0.0%    10    0.9   0.8   0.7   0.9   0.0
  4.|-- nyk-bb3-link.telia.net              0.0%    10   80.6  80.5  80.2  80.6   0.0
  5.|-- ldn-bb3-link.telia.net              0.0%    10  156.0 150.9 149.6 156.0   1.9
  6.|-- adm-bb3-link.telia.net              0.0%    10  156.7 156.7 156.5 156.7   0.0
  7.|-- adm-b3-link.telia.net               0.0%    10  176.1 181.0 176.1 184.6   2.4
  8.|-- vultr-ic-313751-adm-b3.c.telia.net 10.0%    10  180.5 181.4 179.6 184.7   1.4
  9.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
 10.|-- 108.61.198.102.vultr.com            0.0%    10  170.0 167.6 161.4 170.9   3.2

----------------------------------------------
mtr --report -w 200 --report-cycles=10 lon-gb-ping.vultr.com
Start: Sun Feb 25 20:00:00 2018
HOST: hostname              Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10   14.8  16.4  11.9  21.3   3.5
  2.|-- vl101.fr3.lax.llnw.net              0.0%    10    0.3   1.0   0.2   7.4   2.3
  3.|-- las-b21-link.telia.net              0.0%    10    0.9   0.8   0.8   0.9   0.0
  4.|-- ash-bb3-link.telia.net              0.0%    10   60.7  61.2  60.5  64.2   0.9
  5.|-- ldn-bb3-link.telia.net              0.0%    10  142.0 143.2 141.8 150.7   2.7
  6.|-- ldn-b9-link.telia.net               0.0%    10  135.9 136.0 135.7 137.0   0.0
  7.|-- vultr-ic-312913-ldn-b9.c.telia.net  0.0%    10  146.8 138.8 136.4 146.8   3.7
  8.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
  9.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
 10.|-- 108.61.196.101.vultr.com            0.0%    10  133.9 134.0 133.9 134.1   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 par-fr-ping.vultr.com
Start: Sun Feb 25 20:00:16 2018
HOST: hostname              Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10   14.4  19.6  12.4  41.4   8.3
  2.|-- vl101.fr3.lax.llnw.net              0.0%    10    0.3   0.3   0.3   0.4   0.0
  3.|-- las-b21-link.telia.net              0.0%    10    0.8   0.9   0.8   1.2   0.0
  4.|-- nyk-bb3-link.telia.net              0.0%    10   83.9  83.9  83.6  84.8   0.0
  5.|-- prs-bb3-link.telia.net              0.0%    10  163.3 163.4 163.1 164.6   0.3
  6.|-- prs-b7-link.telia.net               0.0%    10  163.3 163.3 163.3 163.4   0.0
  7.|-- vultr-ic-312910-prs-b7.c.telia.net  0.0%    10  164.2 165.7 163.8 172.2   3.1
  8.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
  9.|-- 108.61.209.127.vultr.com            0.0%    10  163.7 163.7 163.6 163.9   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 VersaWeb Las Vegas
Start: Sun Feb 25 20:00:31 2018
HOST: hostname                  Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                 0.0%    10   12.0  16.7  12.0  21.3   3.0
  2.|-- vl101.fr3.lax.llnw.net                  0.0%    10    0.3   0.3   0.2   0.3   0.0
  3.|-- ve6.fr4.lax.llnw.net                    0.0%    10    0.5   0.3   0.3   0.5   0.0
  4.|-- 10gigabitethernet1-3.core1.lax1.he.net  0.0%    10    9.4   2.1   0.2   9.4   3.1
  5.|-- 100ge14-1.core1.lax2.he.net             0.0%    10    6.3   4.3   0.4  11.5   4.5
  6.|-- 10ge4-3.core1.las1.he.net               0.0%    10    7.8   9.6   7.7  15.7   2.8
  7.|-- fiberhub.10ge3-2.core1.las1.he.net      0.0%    10   10.3  12.5   8.1  19.0   3.8
  8.|-- 192.228.109.86                          0.0%    10    8.2   8.1   8.1   8.2   0.0
  9.|-- lg.las1.ultravps.net                    0.0%    10    8.1   8.1   8.0   8.2   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 VersaWeb Seattle
Start: Sun Feb 25 20:00:47 2018
HOST: hostname                 Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                0.0%    10   16.5  15.0  12.1  19.7   2.1
  2.|-- vl101.fr3.lax.llnw.net                 0.0%    10    0.3   0.4   0.2   1.3   0.0
  3.|-- be4695.ccr24.lax05.atlas.cogentco.com  0.0%    10    0.9   0.9   0.8   1.0   0.0
  4.|-- be2538.ccr41.lax01.atlas.cogentco.com  0.0%    10    0.7   0.8   0.6   1.1   0.0
  5.|-- be3176.ccr21.sjc01.atlas.cogentco.com  0.0%    10   11.9  11.8  11.6  12.1   0.0
  6.|-- be3178.ccr21.sfo01.atlas.cogentco.com  0.0%    10   13.0  12.9  12.8  13.1   0.0
  7.|-- be2075.ccr21.sea01.atlas.cogentco.com  0.0%    10   36.8  36.8  36.8  36.9   0.0
  8.|-- be2083.ccr21.sea02.atlas.cogentco.com  0.0%    10   30.2  30.2  30.0  30.8   0.0
  9.|-- 38.142.48.250                          0.0%    10   29.7  30.0  29.7  31.4   0.3
 10.|-- 76.164.234.1                           0.0%    10   38.4  38.3  38.3  38.4   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 OVH Canada
Start: Sun Feb 25 20:01:04 2018
HOST: hostname        Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                       0.0%    10    5.9  10.2   1.3  39.7  10.8
  2.|-- 2607:f4e8:1:110::122          0.0%    10    0.3   0.2   0.2   0.3   0.0
  3.|-- lag45.fr3.sjc.ipv6.llnw.net   0.0%    10    9.9  10.0   9.9  10.3   0.0
  4.|-- lag41.fr3.ord.ipv6.llnw.net   0.0%    10   64.3  64.3  64.2  64.7   0.0
  5.|-- ve6.fr4.ord.ipv6.llnw.net     0.0%    10   63.0  63.0  62.9  63.6   0.0
  6.|-- lag23.fr4.fra1.ipv6.llnw.net  0.0%    10  188.4 191.5 176.3 203.7   9.4
  7.|-- ???                          100.0    10    0.0   0.0   0.0   0.0   0.0
  8.|-- be100-1166.ams-1-a9.nl.eu     0.0%    10  188.6 189.0 188.5 189.3   0.0
  9.|-- mil-3-4m.it.eu               20.0%    10  190.2 197.8 189.3 242.5  18.4
 10.|-- po100.rbx-g1-a75.fr.eu        0.0%    10  188.9 188.7 188.1 189.1   0.0
 11.|-- vl165.rbx-g2-a75.fr.eu        0.0%    10  188.3 188.5 188.2 188.8   0.0
 12.|-- vl1305.bhs-g2-a75-lo2.qc.ca   0.0%    10  268.2 268.2 267.5 268.7   0.0
 13.|-- 2001:41d0:0:50::6:80f         0.0%    10  267.2 267.2 266.7 267.6   0.0
 14.|-- 2001:41d0:0:50::2:404d       10.0%    10  266.5 266.9 266.5 267.3   0.0
 15.|-- 2607:5300:60:44a5::1          0.0%    10  266.7 266.5 266.0 266.7   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io Frankfurt:
Start: Sun Feb 25 20:01:21 2018
HOST: hostname              Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10   17.2  21.7   6.5  73.0  18.5
  2.|-- vl101.fr3.lax.llnw.net              0.0%    10    0.3   0.4   0.2   1.5   0.3
  3.|-- lag45.fr3.sjc.llnw.net              0.0%    10   10.0  10.0   9.9  10.3   0.0
  4.|-- lag41.fr3.ord.llnw.net              0.0%    10   64.3  64.3  64.2  64.9   0.0
  5.|-- lag44.fr3.lga.llnw.net              0.0%    10   80.7  80.7  80.6  81.1   0.0
  6.|-- xe-1-2-0-0.nyk10.core-backbone.com  0.0%    10   80.5  80.5  80.4  80.6   0.0
  7.|-- ae2-2025.fra30.core-backbone.com    0.0%    10  168.6 168.7 168.5 168.9   0.0
  8.|-- core-backbone.cdn77.com             0.0%    10  181.0 180.9 180.9 181.1   0.0
  9.|-- frankfurt-20.cdn77.com              0.0%    10  180.9 181.0 180.9 181.1   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io London:
Start: Sun Feb 25 20:01:37 2018
HOST: hostname              Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10   17.0  18.1  14.4  21.9   2.9
  2.|-- vl101.fr3.lax.llnw.net              0.0%    10    8.1   2.5   0.2  10.4   3.8
  3.|-- lag45.fr3.sjc.llnw.net              0.0%    10   10.0  10.1   9.9  11.8   0.5
  4.|-- lag41.fr3.ord.llnw.net              0.0%    10   64.2  64.2  64.2  64.3   0.0
  5.|-- lag44.fr3.lga.llnw.net              0.0%    10   80.6  80.7  80.6  80.8   0.0
  6.|-- xe-1-2-0-0.nyk10.core-backbone.com  0.0%    10   80.4  80.4  80.3  80.4   0.0
  7.|-- ae10-2092.lon10.core-backbone.com   0.0%    10  141.8 142.0 141.7 142.6   0.0
  8.|-- core-backbone.cdn77.com             0.0%    10  181.4 181.4 181.3 181.4   0.0
  9.|-- london-20.cdn77.com                 0.0%    10  182.2 182.2 182.1 182.3   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io Paris:
Start: Sun Feb 25 20:01:53 2018
HOST: hostname                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                               0.0%    10   19.0  19.2  12.2  44.4   9.5
  2.|-- vl101.fr3.lax.llnw.net                0.0%    10    0.3   0.3   0.2   0.8   0.0
  3.|-- ae10-352.cr0-lax1.ip4.gtt.net         0.0%    10    0.3   2.8   0.3  25.0   7.8
  4.|-- et-0-0-43.cr6-sjc1.ip4.gtt.net        0.0%    10   14.7  12.3   8.8  17.3   3.0
  5.|-- as2914.cr6-sjc1.ip4.gtt.net           0.0%    10    9.3   9.1   8.9   9.3   0.0
  6.|-- ae-1.r22.snjsca04.us.bb.gin.ntt.net   0.0%    10    9.0   9.9   8.9  16.1   2.2
  7.|-- ae-7.r23.asbnva02.us.bb.gin.ntt.net   0.0%    10   73.1  67.2  65.0  74.2   3.3
  8.|-- ae-2.r25.amstnl02.nl.bb.gin.ntt.net   0.0%    10  164.2 164.1 163.9 164.2   0.0
  9.|-- ae-3.r24.amstnl02.nl.bb.gin.ntt.net   0.0%    10  164.8 164.4 164.0 166.3   0.7
 10.|-- ae-14.r02.parsfr02.fr.bb.gin.ntt.net  0.0%    10  147.5 147.5 147.4 148.0   0.0
 11.|-- 81.25.206.50                          0.0%    10  182.7 182.7 182.7 182.8   0.0
 12.|-- unn-185-93-2-193.10gbps.io            0.0%    10  182.9 182.9 182.8 183.0   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io Singapore:
Start: Sun Feb 25 20:02:10 2018
HOST: hostname                        Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                       0.0%    10   13.8  16.3  12.8  22.0   3.4
  2.|-- vl101.fr3.lax.llnw.net                        0.0%    10    0.2   0.2   0.2   0.3   0.0
  3.|-- ix-ae-20-0.tcore1.EQL-Los-Angeles.as6453.net  0.0%    10   34.5  33.8  33.4  35.3   0.5
  4.|-- if-ae-6-2.tcore2.LVW-Los-Angeles.as6453.net   0.0%    10  205.7 205.7 205.6 205.8   0.0
  5.|-- if-ae-7-2.tcore2.SVW-Singapore.as6453.net     0.0%    10  214.3 215.5 214.2 225.5   3.5
  6.|-- if-ae-2-2.tcore1.SVW-Singapore.as6453.net     0.0%    10  205.2 205.2 205.1 205.3   0.0
  7.|-- if-ae-11-2.thar1.SVQ-Singapore.as6453.net     0.0%    10  205.5 205.5 205.3 205.9   0.0
  8.|-- 180.87.98.50                                  0.0%    10  257.9 247.0 245.7 257.9   3.8
  9.|-- ???                                          100.0    10    0.0   0.0   0.0   0.0   0.0
 10.|-- hol14.homeoflim.com                           0.0%    10  176.8 176.8 176.7 176.9   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io Stockholm:
Start: Sun Feb 25 20:02:29 2018
HOST: hostname              Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10   13.8  18.6  13.8  22.9   2.7
  2.|-- vl101.fr3.lax.llnw.net              0.0%    10    0.5   2.6   0.2   8.9   3.1
  3.|-- lag45.fr3.sjc.llnw.net              0.0%    10   10.0  10.0   9.8  10.0   0.0
  4.|-- lag41.fr3.ord.llnw.net              0.0%    10   64.2  64.2  64.1  64.5   0.0
  5.|-- lag44.fr3.lga.llnw.net              0.0%    10   80.7  80.6  80.5  80.7   0.0
  6.|-- xe-1-2-0-0.nyk10.core-backbone.com  0.0%    10   80.5  80.4  80.4  80.5   0.0
  7.|-- ae1-2081.sth10.core-backbone.com    0.0%    10  191.6 191.7 191.6 191.8   0.0
  8.|-- core-backbone.cdn77.com             0.0%    10  190.9 191.0 190.9 191.1   0.0
  9.|-- stockholm-10.cdn77.com              0.0%    10  229.1 195.1 191.3 229.1  11.9

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io Moscow:
Start: Sun Feb 25 20:02:45 2018
HOST: hostname       Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                      0.0%    10   18.3  25.1  13.4  62.1  15.5
  2.|-- vl101.fr3.lax.llnw.net       0.0%    10    0.3   0.4   0.2   1.3   0.0
  3.|-- lag45.fr3.sjc.llnw.net       0.0%    10   10.2  10.0   9.9  10.2   0.0
  4.|-- lag41.fr3.ord.llnw.net       0.0%    10   67.9  65.2  64.2  70.0   2.0
  5.|-- lag44.fr3.lga.llnw.net       0.0%    10   80.5  80.6  80.5  80.7   0.0
  6.|-- nyiix.retn.net               0.0%    10  190.4 190.4 190.2 190.4   0.0
  7.|-- ae1-3.RT.MR.MSK.RU.retn.net  0.0%    10  209.8 209.8 209.7 209.9   0.0
  8.|-- 87.245.204.20                0.0%    10  210.2 210.2 210.1 210.3   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io Miami:
Start: Sun Feb 25 20:03:01 2018
HOST: hostname               Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                              0.0%    10   21.8  22.5  11.6  76.9  19.5
  2.|-- vl101.fr3.lax.llnw.net               0.0%    10    0.3   0.4   0.2   1.9   0.5
  3.|-- ae10-352.cr0-lax1.ip4.gtt.net        0.0%    10    0.4   0.4   0.3   0.8   0.0
  4.|-- et-0-0-43.cr6-sjc1.ip4.gtt.net       0.0%    10    9.1   9.1   8.8   9.6   0.0
  5.|-- as2914.cr6-sjc1.ip4.gtt.net          0.0%    10    9.2   9.0   8.9   9.2   0.0
  6.|-- ae-1.r22.snjsca04.us.bb.gin.ntt.net  0.0%    10    9.0   9.2   9.0  10.2   0.0
  7.|-- ae-7.r23.asbnva02.us.bb.gin.ntt.net  0.0%    10   65.9  66.1  65.9  66.5   0.0
  8.|-- ae-1.r20.miamfl02.us.bb.gin.ntt.net  0.0%    10   81.4  88.7  81.4 154.2  23.0
  9.|-- ae-1.r04.miamfl02.us.bb.gin.ntt.net  0.0%    10   81.4  81.5  81.4  81.7   0.0
 10.|-- 165.254.21.26                        0.0%    10   81.2  81.2  81.2  81.3   0.0
 11.|-- unn-195-181-162-195.cdn77.com        0.0%    10   81.4  81.3  81.2  81.5   0.0

    Code (Text):
    -------------------------------------------
Check system entropy pool availability
-------------------------------------------

entropy_avail: 4067
entropy_avail: 4067
entropy_avail: 4067
entropy_avail: 4067

    Code (Text):
    -------------------------------------------
OpenSSL System Benchmark
-------------------------------------------

OpenSSL 1.0.2k-fips  26 Jan 2017
-------------------------------------------
openssl speed -multi 8 rsa4096 rsa2048 ecdsap256 sha256 sha1 md5 rc4 aes-256-cbc aes-128-cbc
OpenSSL 1.0.2k-fips  26 Jan 2017
built on: reproducible build, date unspecified
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
md5             284699.40k   898011.78k  2258044.50k  3537850.03k  4124409.86k
sha1            294420.36k   870333.63k  2106821.03k  3250504.02k  3869204.48k
rc4            2845331.75k  4172376.00k  4627773.10k  4733641.73k  4785782.78k
aes-128 cbc     547909.02k   579002.97k   586271.66k   589405.53k   590345.56k
aes-256 cbc     396894.33k   413164.05k   416706.56k   418214.23k   418734.08k
sha256          302442.80k   675528.70k  1284426.75k  1639872.51k  1784987.65k
                  sign    verify    sign/s verify/s
rsa 2048 bits 0.000151s 0.000004s   6627.3 223809.5
rsa 4096 bits 0.001018s 0.000016s    982.3  64387.1
                              sign    verify    sign/s verify/s
 256 bit ecdsa (nistp256)   0.0000s   0.0000s 118525.0  51864.3
-------------------------------------------
openssl speed -evp aes256 -multi 8
OpenSSL 1.0.2k-fips  26 Jan 2017
built on: reproducible build, date unspecified
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
evp            5098231.54k  6559981.31k  6880439.89k  6972779.52k  7005604.52k
-------------------------------------------
openssl speed -evp aes128 -multi 8
OpenSSL 1.0.2k-fips  26 Jan 2017
built on: reproducible build, date unspecified
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
evp            5837247.91k  8792008.90k  9402493.10k  9585271.47k  9647740.25k

    Code (Text):
    -------------------------------------------
Running mysqlslap
-------------------------------------------

mysqlslap --auto-generate-sql --auto-generate-sql-add-autoincrement --auto-generate-sql-secondary-indexes=5 --number-int-cols=5 --number-char-cols=5 --number-of-queries=25000 --auto-generate-sql-unique-query-number=40 --auto-generate-sql-unique-write-number=40 --auto-generate-sql-write-number=1000 --concurrency=64 --iterations=10 --engine=myisam
Benchmark
        Running for engine myisam
        Average number of seconds to run all queries: 0.587 seconds
        Minimum number of seconds to run all queries: 0.576 seconds
        Maximum number of seconds to run all queries: 0.598 seconds
        Number of clients running queries: 64
        Average number of queries per client: 390

Dropping the database is potentially a very bad thing to do.
Any data stored in the database will be destroyed.

Do you really want to drop the 'test' database [y/N] Database "test" dropped

    Code (Text):
    -------------------------------------------
System PHP Info
-------------------------------------------

CPU: 8 x Intel(R) Xeon(R) CPU E3-1578L v5 @ 2.00GHz
              total        used        free      shared  buff/cache   available
Mem:          32084        1478       29998          33         607       30199
Low:          32084        2086       29998
High:             0           0           0
Swap:          1949           0        1949
----------------------------------------------
PHP 5.6.33 (cli) (built: Feb 25 2018 19:39:56)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
    with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies
----------------------------------------------
Configuration File (php.ini) Path: /usr/local/lib
Loaded Configuration File:         /usr/local/lib/php.ini
Scan for additional .ini files in: /etc/centminmod/php.d
Additional .ini files parsed:      /etc/centminmod/php.d/a_customphp.ini,
/etc/centminmod/php.d/curlcainfo.ini,
/etc/centminmod/php.d/geoip.ini,
/etc/centminmod/php.d/igbinary.ini,
/etc/centminmod/php.d/imagick.ini,
/etc/centminmod/php.d/mailparse.ini,
/etc/centminmod/php.d/memcache.ini,
/etc/centminmod/php.d/memcached.ini,
/etc/centminmod/php.d/redis.ini,
/etc/centminmod/php.d/zendopcache.ini

----------------------------------------------
[PHP Modules]
bcmath
bz2
calendar
Core
ctype
curl
date
dom
enchant
ereg
exif
filter
ftp
gd
geoip
gettext
gmp
hash
iconv
igbinary
imagick
imap
intl
json
ldap
libxml
mailparse
mbstring
mcrypt
memcache
memcached
mhash
mysql
mysqli
mysqlnd
openssl
pcntl
pcre
PDO
pdo_mysql
pdo_sqlite
Phar
posix
pspell
readline
redis
Reflection
session
shmop
SimpleXML
snmp
soap
sockets
SPL
sqlite3
standard
sysvmsg
sysvsem
sysvshm
tidy
tokenizer
xml
xmlreader
xmlrpc
xmlwriter
xsl
Zend OPcache
zip
zlib

[Zend Modules]
Zend OPcache

----------------------------------------------
-------------------------------------------
Run PHP test Zend/bench.php
-------------------------------------------

simple             0.051
simplecall         0.081
simpleucall        0.077
simpleudcall       0.078
mandel             0.131
mandel2            0.177
ackermann(7)       0.068
ary(50000)         0.014
ary2(50000)        0.013
ary3(2000)         0.110
fibo(30)           0.244
hash1(50000)       0.021
hash2(500)         0.022
heapsort(20000)    0.049
matrix(20)         0.061
nestedloop(12)     0.091
sieve(30)          0.065
strcat(200000)     0.009
------------------------
Total              1.361
real: 1.39s user: 1.37s sys: 0.01s cpu: 99% maxmem: 31404 KB cswaits: 7

simple             0.053
simplecall         0.082
simpleucall        0.079
simpleudcall       0.081
mandel             0.133
mandel2            0.183
ackermann(7)       0.066
ary(50000)         0.013
ary2(50000)        0.012
ary3(2000)         0.110
fibo(30)           0.245
hash1(50000)       0.021
hash2(500)         0.022
heapsort(20000)    0.049
matrix(20)         0.059
nestedloop(12)     0.091
sieve(30)          0.065
strcat(200000)     0.009
------------------------
Total              1.372
real: 1.40s user: 1.38s sys: 0.01s cpu: 99% maxmem: 31408 KB cswaits: 4

simple             0.051
simplecall         0.084
simpleucall        0.080
simpleudcall       0.079
mandel             0.135
mandel2            0.224
ackermann(7)       0.067
ary(50000)         0.014
ary2(50000)        0.013
ary3(2000)         0.110
fibo(30)           0.248
hash1(50000)       0.021
hash2(500)         0.023
heapsort(20000)    0.050
matrix(20)         0.061
nestedloop(12)     0.091
sieve(30)          0.066
strcat(200000)     0.009
------------------------
Total              1.425
real: 1.45s user: 1.43s sys: 0.01s cpu: 99% maxmem: 31404 KB cswaits: 3

bench.php results from 3 runs
1.361
1.372
1.425

bench.php avg: 1.3860
Avg: real: 1.41s user: 1.39s sys: 0.01s cpu: 99.00% maxmem: 31405.33KB cswaits: 4.67
created results log at /home/phpbench_logs/bench_250218-194652.log
server PHP info log at /home/phpbench_logs/bench_phpinfo_250218-194652.log

-------------------------------------------
Run PHP test Zend/micro_bench.php
-------------------------------------------

empty_loop         0.079
func()             0.246    0.167
undef_func()       0.254    0.175
int_func()         0.218    0.140
$x = self::$x      0.170    0.091
self::$x = 0       0.167    0.088
isset(self::$x)    0.163    0.084
empty(self::$x)    0.166    0.087
$x = Foo::$x       0.162    0.083
Foo::$x = 0        0.147    0.068
isset(Foo::$x)     0.146    0.067
empty(Foo::$x)     0.155    0.076
self::f()          0.269    0.190
Foo::f()           0.243    0.164
$x = $this->x      0.152    0.073
$this->x = 0       0.186    0.107
$this->x += 2      0.159    0.080
++$this->x         0.143    0.064
--$this->x         0.140    0.061
$this->x++         0.159    0.080
$this->x--         0.171    0.092
isset($this->x)    0.146    0.067
empty($this->x)    0.164    0.085
$this->f()         0.269    0.190
$x = Foo::TEST     0.157    0.078
new Foo()          0.465    0.386
$x = TEST          0.119    0.040
$x = $_GET         0.152    0.073
$x = $GLOBALS['v'] 0.209    0.130
$x = $hash['v']    0.165    0.086
$x = $str[0]       0.224    0.145
$x = $a ?: null    0.143    0.064
$x = $f ?: tmp     0.197    0.118
$x = $f ? $f : $a  0.153    0.074
$x = $f ? $f : tmp 0.207    0.128
------------------------
Total              6.465
real: 6.49s user: 6.48s sys: 0.00s cpu: 99% maxmem: 19444 KB cswaits: 6

empty_loop         0.078
func()             0.247    0.168
undef_func()       0.253    0.175
int_func()         0.219    0.141
$x = self::$x      0.170    0.091
self::$x = 0       0.167    0.089
isset(self::$x)    0.164    0.085
empty(self::$x)    0.168    0.089
$x = Foo::$x       0.151    0.073
Foo::$x = 0        0.148    0.070
isset(Foo::$x)     0.147    0.069
empty(Foo::$x)     0.155    0.077
self::f()          0.280    0.202
Foo::f()           0.239    0.160
$x = $this->x      0.149    0.071
$this->x = 0       0.191    0.113
$this->x += 2      0.159    0.081
++$this->x         0.142    0.063
--$this->x         0.148    0.070
$this->x++         0.159    0.081
$this->x--         0.164    0.086
isset($this->x)    0.146    0.067
empty($this->x)    0.165    0.086
$this->f()         0.269    0.190
$x = Foo::TEST     0.157    0.079
new Foo()          0.461    0.382
$x = TEST          0.125    0.047
$x = $_GET         0.152    0.074
$x = $GLOBALS['v'] 0.209    0.131
$x = $hash['v']    0.165    0.086
$x = $str[0]       0.230    0.152
$x = $a ?: null    0.144    0.065
$x = $f ?: tmp     0.198    0.120
$x = $f ? $f : $a  0.153    0.075
$x = $f ? $f : tmp 0.207    0.129
------------------------
Total              6.478
real: 6.50s user: 6.50s sys: 0.00s cpu: 99% maxmem: 19444 KB cswaits: 4

empty_loop         0.079
func()             0.247    0.168
undef_func()       0.253    0.175
int_func()         0.219    0.140
$x = self::$x      0.169    0.091
self::$x = 0       0.167    0.089
isset(self::$x)    0.163    0.085
empty(self::$x)    0.166    0.088
$x = Foo::$x       0.156    0.078
Foo::$x = 0        0.147    0.068
isset(Foo::$x)     0.155    0.076
empty(Foo::$x)     0.155    0.076
self::f()          0.265    0.187
Foo::f()           0.239    0.160
$x = $this->x      0.150    0.071
$this->x = 0       0.187    0.108
$this->x += 2      0.160    0.081
++$this->x         0.141    0.063
--$this->x         0.140    0.062
$this->x++         0.159    0.081
$this->x--         0.164    0.086
isset($this->x)    0.145    0.067
empty($this->x)    0.165    0.086
$this->f()         0.268    0.190
$x = Foo::TEST     0.157    0.078
new Foo()          0.462    0.383
$x = TEST          0.119    0.041
$x = $_GET         0.152    0.074
$x = $GLOBALS['v'] 0.205    0.127
$x = $hash['v']    0.165    0.086
$x = $str[0]       0.218    0.140
$x = $a ?: null    0.143    0.065
$x = $f ?: tmp     0.205    0.126
$x = $f ? $f : $a  0.153    0.074
$x = $f ? $f : tmp 0.206    0.128
------------------------
Total              6.446
real: 6.47s user: 6.47s sys: 0.00s cpu: 99% maxmem: 19444 KB cswaits: 3

micro_bench.php results from 3 runs
6.465
6.478
6.446

micro_bench.php avg: 6.4630
Avg: real: 6.49s user: 6.48s sys: 0.00s cpu: 99.00% maxmem: 19444.00KB cswaits: 4.33
created results log at /home/phpbench_logs/bench_micro_250218-194652.log
server PHP info log at /home/phpbench_logs/bench_phpinfo_250218-194652.log

    Code (Text):
    ========================================================================
   BYTE UNIX Benchmarks (Version 5.1.3)

   System: hostname: GNU/Linux
   OS: GNU/Linux -- 3.10.0-693.17.1.el7.x86_64 -- #1 SMP Thu Jan 25 20:13:58 UTC 2018
   Machine: x86_64 (x86_64)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel(R) Xeon(R) CPU E3-1578L v5 @ 2.00GHz (3984.0 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   CPU 1: Intel(R) Xeon(R) CPU E3-1578L v5 @ 2.00GHz (3984.0 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   CPU 2: Intel(R) Xeon(R) CPU E3-1578L v5 @ 2.00GHz (3984.0 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   CPU 3: Intel(R) Xeon(R) CPU E3-1578L v5 @ 2.00GHz (3984.0 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   CPU 4: Intel(R) Xeon(R) CPU E3-1578L v5 @ 2.00GHz (3984.0 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   CPU 5: Intel(R) Xeon(R) CPU E3-1578L v5 @ 2.00GHz (3984.0 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   CPU 6: Intel(R) Xeon(R) CPU E3-1578L v5 @ 2.00GHz (3984.0 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   CPU 7: Intel(R) Xeon(R) CPU E3-1578L v5 @ 2.00GHz (3984.0 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   20:07:52 up 54 min,  2 users,  load average: 3.27, 3.11, 1.64; runlevel 3

    Code (Text):
    ------------------------------------------------------------------------
Benchmark Run: Sun Feb 25 2018 20:07:52 - 20:32:10
8 CPUs in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       42338214.0 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4142.7 MWIPS (11.9 s, 7 samples)
Execl Throughput                               3565.4 lps   (30.0 s, 2 samples)
Pipe Throughput                             1269415.3 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 251854.6 lps   (10.0 s, 7 samples)
Process Creation                              13937.3 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   8575.4 lpm   (60.0 s, 2 samples)
Shell Scripts (16 concurrent)                  2266.5 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   3928.1 lpm   (60.0 s, 2 samples)
System Call Overhead                        1079320.0 lps   (10.0 s, 7 samples)

System Benchmarks Partial Index              BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   42338214.0   3628.0
Double-Precision Whetstone                       55.0       4142.7    753.2
Execl Throughput                                 43.0       3565.4    829.2
Pipe Throughput                               12440.0    1269415.3   1020.4
Pipe-based Context Switching                   4000.0     251854.6    629.6
Process Creation                                126.0      13937.3   1106.1
Shell Scripts (1 concurrent)                     42.4       8575.4   2022.5
Shell Scripts (16 concurrent)                     ---       2266.5      ---
Shell Scripts (8 concurrent)                      6.0       3928.1   6546.8
System Call Overhead                          15000.0    1079320.0    719.5
                                                                   ========
System Benchmarks Index Score (Partial Only)                         1354.5

    Code (Text):
    ------------------------------------------------------------------------
Benchmark Run: Sun Feb 25 2018 20:32:10 - 20:56:12
8 CPUs in system; running 8 parallel copies of tests

Dhrystone 2 using register variables      196525904.0 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                    33111.4 MWIPS (10.0 s, 7 samples)
Execl Throughput                              26435.2 lps   (30.0 s, 2 samples)
Pipe Throughput                             7459463.6 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                1553486.6 lps   (10.0 s, 7 samples)
Process Creation                              85442.4 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                  38681.7 lpm   (60.0 s, 2 samples)
Shell Scripts (16 concurrent)                  2703.6 lpm   (60.1 s, 2 samples)
Shell Scripts (8 concurrent)                   5377.3 lpm   (60.0 s, 2 samples)
System Call Overhead                        6366871.0 lps   (10.0 s, 7 samples)

System Benchmarks Partial Index              BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0  196525904.0  16840.3
Double-Precision Whetstone                       55.0      33111.4   6020.3
Execl Throughput                                 43.0      26435.2   6147.7
Pipe Throughput                               12440.0    7459463.6   5996.4
Pipe-based Context Switching                   4000.0    1553486.6   3883.7
Process Creation                                126.0      85442.4   6781.1
Shell Scripts (1 concurrent)                     42.4      38681.7   9123.0
Shell Scripts (16 concurrent)                     ---       2703.6      ---
Shell Scripts (8 concurrent)                      6.0       5377.3   8962.2
System Call Overhead                          15000.0    6366871.0   4244.6
                                                                   ========
System Benchmarks Index Score (Partial Only)                         6871.6

    Code (Text):
    -------------------------------------------
centminmodbench.sh completed
-------------------------------------------

centminmodbench.sh Total Run Time: 4151.409660621 seconds


    Current billing cost for testing :)

    type1e-billing-01.png

    GCC 8.0.1 Compile & RPM Build Times



    Centmin Mod LEMP Stack's Nginx web server supports multiple GCC/Clang version compilers from GCC 4.8, 5.3, 6.3, 7.2 and 8.x and Clang 3.4, 4, 5, and 6. Currently testing custom GCC 8.0.1 compilations and RPM builds so let's see how the Packet.net bare metal server holds up.

    It took a total of ~2716 seconds or ~45 minutes to compile and build both binutils 2.30 + GCC 8.0.1 with Profile Guided Optimisations and build the custom CentOS 7 RPMs.

    Code (Text):
    RPMs Built
/svr-setup/binutils-gcc8-2.30-1.el7.x86_64.rpm
/svr-setup/gcc8-pgo-8.0-1.el7.x86_64.rpm

ls -lah /svr-setup | egrep 'gcc8-pgo-8.0|binutils-gcc8-2.30'
-rw-r--r--  1 root      root       5.2M Feb 25 22:04 binutils-gcc8-2.30-1.el7.x86_64.rpm
-rw-r--r--  1 root      root        39M Feb 25 22:47 gcc8-pgo-8.0-1.el7.x86_64.rpm

Installed Packages
Name        : binutils-gcc8
Arch        : x86_64
Version     : 2.30
Release     : 1.el7
Size        : 48 M
Repo        : installed
Summary     : binutils-gcc8 for centminmod.com LEMP stack installs
URL         : https://centminmod.com
License     : unknown
Description : binutils-gcc8 for centminmod.com LEMP stacks

* Sun Feb 25 2018 George Liu <centminmod.com> 2.30
- Binutils 2.30 for centminmod.com LEMP stack installs

    Code (Text):
    Installed Packages
Name        : gcc8-pgo
Arch        : x86_64
Version     : 8.0
Release     : 1.el7
Size        : 287 M
Repo        : installed
Summary     : gcc8-pgo for centminmod.com LEMP stack installs
URL         : https://centminmod.com
License     : unknown
Description : gcc8-pgo for centminmod.com LEMP stacks


* Sun Feb 25 2018 George Liu <centminmod.com> 8
- GCC 8 for centminmod.com LEMP stack installs

*************************************************
Compile GCC Completed
log: /root/centminlogs/tools-gcc-install-pgo_250218-220157.log
*************************************************

Total Binutils + GCC Install Time: 2716.625791497 seconds
     
    Last edited: Feb 27, 2018
  3. Feb 26, 2018 #3
    eva2000

    eva2000 Administrator Staff Member

    53,280
    12,117
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,655
    Local Time:
    2:23 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    Packet.net TYPE 2A - Cavium ThunderX CN8890 @2.0Ghz



    Next up is testing the TYPE 2A bare metal server for a 96 physical core 2x Cavium ThunderX CN8890 ARMv8 64bit processor running at 2.0Ghz using spot bidding pricing at US$0.20/hr on current market rate of US$0.10/hr (on demand pricing is US$0.50/hr) for Centmin Mod 123.09beta01 install testing and centminmodbench.sh testing.

    System
    • 96 Physical Cores @ 2.0 GHz (2 × Cavium ThunderX)
    • 128 GB of DDR4 ECC RAM
    • 1x 340 GB of SSD
    • 20Gbps Bonded Network
    • CentOS 7.4 64bit
    • New Jersey
    type2a-hardware-01.png

    Cavium ThunderX CN8890 Details



    Not much details but 96 cpu cores, 48 per socket in a 2 socket - 2 Numa node configuration.

    Code (Text):
    lscpu
Architecture:          aarch64
Byte Order:            Little Endian
CPU(s):                96
On-line CPU(s) list:   0-95
Thread(s) per core:    1
Core(s) per socket:    48
Socket(s):             2
NUMA node(s):          2
NUMA node0 CPU(s):     0-47
NUMA node1 CPU(s):     48-95


    cat /proc/cpuinfo output for 96 cores
    Code (Text):
    cat /proc/cpuinfo
processor       : 0
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 1
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 2
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 3
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 4
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 5
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 6
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 7
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 8
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 9
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 10
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 11
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 12
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 13
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 14
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 15
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 16
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 17
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 18
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 19
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 20
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 21
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 22
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 23
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 24
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 25
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 26
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 27
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 28
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 29
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 30
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 31
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 32
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 33
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 34
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 35
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 36
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 37
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 38
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 39
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 40
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 41
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 42
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 43
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 44
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 45
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 46
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 47
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 48
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 49
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 50
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 51
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 52
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 53
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 54
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 55
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 56
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 57
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 58
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 59
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 60
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 61
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 62
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 63
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 64
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 65
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 66
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 67
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 68
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 69
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 70
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 71
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 72
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 73
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 74
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 75
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 76
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 77
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 78
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 79
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 80
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 81
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 82
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 83
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 84
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 85
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 86
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 87
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 88
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 89
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 90
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 91
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 92
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 93
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 94
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

processor       : 95
BogoMIPS        : 200.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x43
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0x0a1
CPU revision    : 1

    128GB memory
    Code (Text):
    free -m
              total        used        free      shared  buff/cache   available
Mem:         128719         649      127363          29         706      127225
Swap:             0           0           0

    disk config
    Code (Text):
    df -hT
Filesystem     Type      Size  Used Avail Use% Mounted on
devtmpfs       devtmpfs   63G     0   63G   0% /dev
tmpfs          tmpfs      63G     0   63G   0% /dev/shm
tmpfs          tmpfs      63G   30M   63G   1% /run
tmpfs          tmpfs      63G     0   63G   0% /sys/fs/cgroup
/dev/sda3      ext4      227G  918M  214G   1% /
/dev/sda1      vfat      500M  368K  499M   1% /boot/efi
tmpfs          tmpfs      13G     0   13G   0% /run/user/0


    Centmin Mod 123.09beta01 Install



    Centmin Mod install routine is optimised to detect Intel processors for further Intel related tuned performance. So not sure how it will do with 2x Cavium ThunderX CN8890 ARMv8 64bit based processors. This is what's great about Packet.net hourly billed bare metal - for developers like me, testing such bare metal servers to develop Centmin Mod support for these higher end processors can be done relatively cheaper than renting and committing to a full month !

    Used default curl installer command below
    Code (Text):
    yum -y update; curl -O https://centminmod.com/betainstaller.sh && chmod 0700 betainstaller.sh && bash betainstaller.sh

    Looks like the install failed as some YUM repos including Remi and MariaDB official YUM repo Centmin Mod sets up doesn't seem to be for ARMv8 64bit based Cavium ThunderX. In a Intel dominant server world, most YUM repos are for x86_64 architecture and not AMRv8 which would be aarch64 architecture.

    not Intel x86_64 based
    Code (Text):
    uname -m
aarch64


    Will have to inspect the full initial install log to see where the compatibility issues are :D

    Edit: there is no official MariaDB ARM aarch64 supported YUM repository for CentOS MariaDB - Setting up MariaDB Repositories - MariaDB :(

    Seems like it would be cheaper to do ARMv8 development/testing working on Scaleway's ARMv8 based cloud VPS servers instead though.
     
    Last edited: Feb 27, 2018
  4. Feb 26, 2018 #4
    eva2000

    eva2000 Administrator Staff Member

    53,280
    12,117
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,655
    Local Time:
    2:23 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    Packet.net AMD EPYC 7401P c2.medium



    Packet.net has rolled out AMD EPYC 7401P under the plan name = c2.medium in Sunnyvale Califorina and Parsippany, New Jersey at US$1.00/hr ondemand pricing. AMD EPYC 7401 official product page lists base clock at 2.0Ghz, with all core turbo boost at 2.8Ghz and max turbo boost at 3Ghz. Using CentOS 7.x default 3.10 Linux Kernel may not allow AMD EPYC cpus to fully shine as alot of AMD EPYC/Ryzen performance improvements landed in upstream 4.10+ Linux Kernels.

    packetnet-amd-epyc-c2medium-01.png packetnet-amd-epyc-c2medium-02.png
    Also despite officially listing just Ubuntu as default supported OS, it looks like CentOS 7 is available

    packetnet-amd-epyc-c2medium-03.png

    System:
    • 24 Physical Cores @ 2.2 GHz (1 x AMD EPYC 7401P)
    • 64 GB of ECC RAM
    • 1 x 120 GB boot SSD and 1x 120 GB unformatted
    • 960 GB of SSD (2 x 480 GB)
    • 20 Gbps Bonded Network
    • (2 x Mellanox 10Gbps w/ LACP)
    • CentOS 7.4 64bit
    • Parsippany, New Jersey

    Packet.net AMD EPYC 7401P Details



    Web site lists 1x 120GB Boot SSD but the management area and fdisk list 2x 120GB SSD - 1x 120GB for boot and other 1x 120GB unformatted and web site lists 980GB but it's 2x480GB = 960GB and comes unformatted ready for you to format. The 120GB SSD boot disk are relatively slow so probably the intention would be instead of using default CentOS 7 profile, is to install via iPXE with custom partition configuration for CentOS 7. I can see probably using the 2x480GB SSD in raid 1 as boot. But then what to do with the 2x120GB SSDs ? Maybe better for Packet.net to just switch the offering to 4x480GB SSD and can configure 2x 480GB SSD raid 1 by default for CentOS 7 selected OS?

    Code (Text):
    free -m
              total        used        free      shared  buff/cache   available
Mem:          64159        1141       62809          16         208       62597
Swap:          1949           0        1949

    Code (Text):
    df -hT
Filesystem     Type      Size  Used Avail Use% Mounted on
/dev/sdc3      ext4      108G  1.3G  101G   2% /
devtmpfs       devtmpfs   32G     0   32G   0% /dev
tmpfs          tmpfs      32G     0   32G   0% /dev/shm
tmpfs          tmpfs      32G   17M   32G   1% /run
tmpfs          tmpfs      32G     0   32G   0% /sys/fs/cgroup
/dev/sdc1      vfat      511M  148K  511M   1% /boot/efi
tmpfs          tmpfs     6.3G     0  6.3G   0% /run/user/0

    Code (Text):
    fdisk -l
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/sdc: 120.0 GB, 120034123776 bytes, 234441648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: gpt
Disk identifier: F2619121-B58D-434C-9A0C-FC200530EAB9


#         Start          End    Size  Type            Name
 1         2048      1050623    512M  EFI System      BIOS
 2      1050624      5044223    1.9G  Linux filesyste SWAP
 3      5044224    234441614  109.4G  Linux filesyste ROOT

Disk /dev/sda: 480.1 GB, 480103981056 bytes, 937703088 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/sdb: 480.1 GB, 480103981056 bytes, 937703088 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/sdd: 120.0 GB, 120034123776 bytes, 234441648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

    Yes there's 48 of these !

    packet-amd-epyc-carbon-03.png
    Code (Text):
    lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                48
On-line CPU(s) list:   0-47
Thread(s) per core:    2
Core(s) per socket:    24
Socket(s):             1
NUMA node(s):          4
Vendor ID:             AuthenticAMD
CPU family:            23
Model:                 1
Model name:            AMD EPYC 7401P 24-Core Processor
Stepping:              2
CPU MHz:               1996.203
BogoMIPS:              3992.40
Virtualization:        AMD-V
L1d cache:             32K
L1i cache:             64K
L2 cache:              512K
L3 cache:              8192K
NUMA node0 CPU(s):     0,4,8,12,16,20,24,28,32,36,40,44
NUMA node1 CPU(s):     1,5,9,13,17,21,25,29,33,37,41,45
NUMA node2 CPU(s):     2,6,10,14,18,22,26,30,34,38,42,46
NUMA node3 CPU(s):     3,7,11,15,19,23,27,31,35,39,43,47
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca

    All 24 core / 48 cpu threads from /proc/cpuinfo output

    Code (Text):
    cat /proc/cpuinfo
processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 0
cpu cores       : 24
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 1
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 8
cpu cores       : 24
apicid          : 16
initial apicid  : 16
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 2
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 16
cpu cores       : 24
apicid          : 32
initial apicid  : 32
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 3
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 24
cpu cores       : 24
apicid          : 48
initial apicid  : 48
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 4
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 4
cpu cores       : 24
apicid          : 8
initial apicid  : 8
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 5
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 12
cpu cores       : 24
apicid          : 24
initial apicid  : 24
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 6
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 20
cpu cores       : 24
apicid          : 40
initial apicid  : 40
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 7
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 28
cpu cores       : 24
apicid          : 56
initial apicid  : 56
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 8
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 1
cpu cores       : 24
apicid          : 2
initial apicid  : 2
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 9
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 9
cpu cores       : 24
apicid          : 18
initial apicid  : 18
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 10
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 17
cpu cores       : 24
apicid          : 34
initial apicid  : 34
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 11
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 25
cpu cores       : 24
apicid          : 50
initial apicid  : 50
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 12
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 5
cpu cores       : 24
apicid          : 10
initial apicid  : 10
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 13
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 13
cpu cores       : 24
apicid          : 26
initial apicid  : 26
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 14
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 21
cpu cores       : 24
apicid          : 42
initial apicid  : 42
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 15
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 29
cpu cores       : 24
apicid          : 58
initial apicid  : 58
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 16
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 2
cpu cores       : 24
apicid          : 4
initial apicid  : 4
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 17
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 10
cpu cores       : 24
apicid          : 20
initial apicid  : 20
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 18
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 18
cpu cores       : 24
apicid          : 36
initial apicid  : 36
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 19
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 26
cpu cores       : 24
apicid          : 52
initial apicid  : 52
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 20
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 6
cpu cores       : 24
apicid          : 12
initial apicid  : 12
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 21
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 14
cpu cores       : 24
apicid          : 28
initial apicid  : 28
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 22
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 22
cpu cores       : 24
apicid          : 44
initial apicid  : 44
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 23
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 30
cpu cores       : 24
apicid          : 60
initial apicid  : 60
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 24
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 0
cpu cores       : 24
apicid          : 1
initial apicid  : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 25
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 8
cpu cores       : 24
apicid          : 17
initial apicid  : 17
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 26
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 16
cpu cores       : 24
apicid          : 33
initial apicid  : 33
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 27
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 24
cpu cores       : 24
apicid          : 49
initial apicid  : 49
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 28
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 4
cpu cores       : 24
apicid          : 9
initial apicid  : 9
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 29
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 12
cpu cores       : 24
apicid          : 25
initial apicid  : 25
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 30
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 20
cpu cores       : 24
apicid          : 41
initial apicid  : 41
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 31
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 28
cpu cores       : 24
apicid          : 57
initial apicid  : 57
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 32
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 1
cpu cores       : 24
apicid          : 3
initial apicid  : 3
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 33
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 9
cpu cores       : 24
apicid          : 19
initial apicid  : 19
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 34
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 17
cpu cores       : 24
apicid          : 35
initial apicid  : 35
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 35
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 25
cpu cores       : 24
apicid          : 51
initial apicid  : 51
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 36
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 5
cpu cores       : 24
apicid          : 11
initial apicid  : 11
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 37
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 13
cpu cores       : 24
apicid          : 27
initial apicid  : 27
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 38
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 21
cpu cores       : 24
apicid          : 43
initial apicid  : 43
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 39
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 29
cpu cores       : 24
apicid          : 59
initial apicid  : 59
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 40
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 2
cpu cores       : 24
apicid          : 5
initial apicid  : 5
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 41
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 10
cpu cores       : 24
apicid          : 21
initial apicid  : 21
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 42
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 18
cpu cores       : 24
apicid          : 37
initial apicid  : 37
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 43
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 26
cpu cores       : 24
apicid          : 53
initial apicid  : 53
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 44
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 6
cpu cores       : 24
apicid          : 13
initial apicid  : 13
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 45
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 14
cpu cores       : 24
apicid          : 29
initial apicid  : 29
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 46
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 22
cpu cores       : 24
apicid          : 45
initial apicid  : 45
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor       : 47
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7401P 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 1996.203
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 30
cpu cores       : 24
apicid          : 61
initial apicid  : 61
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 3992.40
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]


    Centmin Mod 123.09beta01 Install



    Centmin Mod install routine is optimised to detect Intel processors for further Intel related tuned performance so definitely will need to in future start optimising Centmin Mod detection routines to detect AMD EPYC and Packet.net hourly billed bare metal - for developers like me, make testing such higher end AMD EPYCprocessors relatively cheaper than renting and committing to a full month !

    Used default curl installer command below
    Code (Text):
    yum -y update; curl -O https://centminmod.com/betainstaller.sh && chmod 0700 betainstaller.sh && bash betainstaller.sh

    Looks like install times were let down by slower YUM related downloads (+245 seconds extra for YUM related installs) so will need to investigate while actual Nginx and PHP-FPM source compile times are relative fast but not faster than Packet.net TYPE 1E Intel Xeon E3-1578Lv5 Skylake tests. Expected as AMD EPYC 7401P is clocked lower with base clock at 2.0Ghz, with all core turbo boost at 2.8Ghz and max turbo boost at 3Ghz and using slower single 1x 120GB boot disk compared to E3-1578Lv5 setup with 1x 240GB SSD. You can check out the centminmodbench.sh disk I/O performance benchmarks further below to see how much slower the 120GB SSD boot disk is. But don't judge just Centmin Mod initial install times for overall performance. AMD EPYC 7401P's 24 core / 48 cpu threads and massive 64MB L3 Cache will enable much more multi threaded performance work benefits compared to the typical 4 core / 8 cpu thread servers. Just check out the below centminmodbench.sh OpenSSL benchmark numbers to see how far ahead AMD EPYC 7401P is.

    You can compare that to other times posted at Install - Post your Centmin Mod 123.09beta01 install time stats. Though at as Feb 24th, 2018, the 123.09beta01 installed changed some defaults to more performance at the expensive of initial install time.

    AMD EPYC 7401P Centmin Mod 123.09beta01 Install Times
    Code (Text):
    ---------------------------------------------------------------------------
Total Curl Installer YUM or DNF Time: 340.5943 seconds
Total YUM Time: 60.142379961 seconds
Total YUM or DNF + Source Download Time: 86.9888
Total Nginx First Time Install Time: 182.0339
Total PHP First Time Install Time: 183.7518
Download Zip From Github Time: 1.6182
Total Time Other eg. source compiles: 418.1967
Total Centmin Mod Install Time: 870.9712
---------------------------------------------------------------------------
Total Install Time (curl yum + cm install + zip download): 1213.1837 seconds
---------------------------------------------------------------------------

    compared to Intel Xeon E3-1578Lv5 Skylake Centmin Mod 123.09beta01 Install Times
    Code (Text):
    ---------------------------------------------------------------------------
Total Curl Installer YUM or DNF Time: 91.2912 seconds
Total YUM Time: 64.039117358 seconds
Total YUM or DNF + Source Download Time: 87.3015
Total Nginx First Time Install Time: 135.2867
Total PHP First Time Install Time: 141.0516
Download Zip From Github Time: 1.8695
Total Time Other eg. source compiles: 264.2750
Total Centmin Mod Install Time: 627.9147
---------------------------------------------------------------------------
Total Install Time (curl yum + cm install + zip download): 721.0754 seconds
---------------------------------------------------------------------------

    AMD EPYC 7401P Centmin Mod 123.09beta01 post install check list
    Code (Text):
    *************************************************
* Post-Install Check List....
*************************************************

--------------------------------------------------------
Check ccache Version:
--------------------------------------------------------
ccache version 3.4.1

Copyright (C) 2002-2007 Andrew Tridgell
Copyright (C) 2009-2018 Joel Rosdahl

This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation; either version 3 of the License, or (at your option) any later
version.

--------------------------------------------------------
Check Nginx Version:
--------------------------------------------------------
nginx version: nginx/1.13.9
built by gcc 7.2.1 20170829 (Red Hat 7.2.1-1) (GCC)
built with OpenSSL 1.1.0g  2 Nov 2017
TLS SNI support enabled
configure arguments: --with-ld-opt='-ljemalloc -Wl,-z,relro -Wl,-rpath,/usr/local/lib' --with-cc-opt='-g -O3 -fstack-protector-strong -flto -fuse-ld=gold --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wimplicit-fallthrough=0 -fcode-hoisting -Wp,-D_FORTIFY_SOURCE=2 -gsplit-dwarf' --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-compat --with-http_stub_status_module --with-http_secure_link_module --with-libatomic --with-http_gzip_static_module --with-http_sub_module --with-http_addition_module --with-http_image_filter_module=dynamic --with-http_geoip_module --with-stream_geoip_module --with-stream_realip_module --with-stream_ssl_preread_module --with-threads --with-stream=dynamic --with-stream_ssl_module --with-http_realip_module --add-dynamic-module=../ngx-fancyindex-0.4.2 --add-module=../ngx_cache_purge-2.4.2 --add-module=../ngx_devel_kit-0.3.0 --add-dynamic-module=../set-misc-nginx-module-0.31 --add-dynamic-module=../echo-nginx-module-0.61 --add-module=../redis2-nginx-module-0.14 --add-module=../ngx_http_redis-0.3.7 --add-module=../memc-nginx-module-0.18 --add-module=../srcache-nginx-module-0.31 --add-dynamic-module=../headers-more-nginx-module-0.33 --with-pcre=../pcre-8.41 --with-pcre-jit --with-zlib=../zlib-cloudflare-1.3.0 --with-http_ssl_module --with-http_v2_module --with-openssl=../openssl-1.1.0g --with-openssl-opt='enable-ec_nistp_64_gcc_128'

--------------------------------------------------------
Check PHP-FPM Version:
--------------------------------------------------------
PHP 5.6.33 (cli) (built: Feb 26 2018 08:24:24)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
    with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies

--------------------------------------------------------
Check MariaDB installed RPM Versions:
--------------------------------------------------------
MariaDB-devel-10.1.31-1.el7.centos.x86_64
MariaDB-client-10.1.31-1.el7.centos.x86_64
MariaDB-server-10.1.31-1.el7.centos.x86_64
MariaDB-shared-10.1.31-1.el7.centos.x86_64
MariaDB-common-10.1.31-1.el7.centos.x86_64
MariaDB-compat-10.1.31-1.el7.centos.x86_64

--------------------------------------------------------
Check Memcached Server Version:
--------------------------------------------------------
memcached 1.5.5

--------------------------------------------------------
Check CSF Firewall Version:
--------------------------------------------------------
csf: v11.07 (generic)

--------------------------------------------------------
Check Siege Benchmark Version:
--------------------------------------------------------
SIEGE 4.0.4

Copyright (C) 2017 by Jeffrey Fulmer, et al.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE.


--------------------------------------------------------
Check mysqlreport version:
--------------------------------------------------------
mysqlreport v3.5-maria11 Jul 4 2013

--------------------------------------------------------
Check pure-ftpd Version:
--------------------------------------------------------
pure-ftpd v1.0.42

--------------------------------------------------------
Check YUM Repo List
--------------------------------------------------------
Loaded plugins: fastestmirror, priorities, versionlock
Loading mirror speeds from cached hostfile
 * base: mirrors.centos.webair.com
 * epel: fedora-epel.mirrors.tds.net
 * extras: mirrors.centos.webair.com
 * rpmforge: mirror.team-cymru.org
 * updates: mirrors.centos.webair.com
228 packages excluded due to repository priority protections
repo id                        repo name                             status
base/7/x86_64                  CentOS-7 - Base                       7,420+2,171
centos-sclo-rh/x86_64          CentOS-7 - SCLo rh                       6,546+94
centos-sclo-sclo/x86_64        CentOS-7 - SCLo sclo                          500
epel/x86_64                    Extra Packages for Enterprise Linux 7  11,376+923
extras/7/x86_64                CentOS-7 - Extras                             388
mariadb                        MariaDB                                      15+2
rpmforge                       RHEL 7 - RPMforge.net - dag               139+106
updates/7/x86_64               CentOS-7 - Updates                      1,499+430
!varnishcache_varnish41/x86_64 varnishcache_varnish41                         46
!varnishcache_varnish41-source varnishcache_varnish41-source                   0
repolist: 27,929

--------------------------------------------------------
mytimes check:
/usr/bin/mytimes
--------------------------------------------------------
Mon Feb 26 08:29:14 UTC 2018    [UTC]
Mon Feb 26 18:29:14 AEST 2018   [Australia/Brisbane]
Mon Feb 26 00:29:14 PST 2018    [America/Los_Angeles]
Mon Feb 26 02:29:14 CST 2018    [America/Chicago]
Mon Feb 26 03:29:14 EST 2018    [America/New_York]
Mon Feb 26 03:29:14 EST 2018    [America/Montreal]
Mon Feb 26 08:29:14 GMT 2018    [Europe/London]
Mon Feb 26 09:29:14 CET 2018    [Europe/Berlin]
Mon Feb 26 15:29:14 +07 2018    [Asia/Bangkok]
Mon Feb 26 15:29:14 +07 2018    [Asia/Ho_Chi_Minh]
Mon Feb 26 15:29:14 WIB 2018    [Asia/Jakarta]
Mon Feb 26 16:29:14 +08 2018    [Asia/Kuala_Lumpur]
Mon Feb 26 16:29:14 +08 2018    [Asia/Singapore]
--------------------------------------------------------


    centminmodbench.sh results



    All new VPS/Dedicated servers I get, I will run my centminmodbench.sh benchmark script at bench.centminmod.com to fully evaluate the performance of the server. The results are below.
    • The beast of the AMD EPYC 7401P processor was definitely let down by the slow 120GB SSD boot disk which only managed FIO random read/writes of less than 170MB/s and 43K/37K IOPs. Ioping measured sequential disk I/O at very low 518 IOPs and 129.6MB/s.
    • Memory bandwidth from ramdisk tests were also pretty low around 2.7GB/s for DD tests
    • Network bandwidth for wget single threaded tests were relatively slower at ~50MB/s for US East Coast. The Axel multi-threaded bandwidth tests were a bit better for US East Coast ranging from 113-168MB/s
    • UnixBench was very lower for single index = 386.5 and 48 cpus index at 7478. Maybe 3.10 Linux Kernel related ?
    • OpenSSL numbers as expected are much better with AMD EPYC 7401P hardware AES-NI assisted aes256 results being ~4.57x times faster and ecdsa sign/s being ~4.4x times faster than above Intel Xeon E3-1578Lv5 Skylake.

    Code (Text):
    -------------------------------------------
centminmodbench.sh 0.9.5
http://bench.centminmod.com
written by: George Liu (eva2000)
http://centminmod.com
-------------------------------------------

-------------------------------------------
System Information
-------------------------------------------

3.10.0-693.17.1.el7.x86_64

CentOS Linux release 7.4.1708 (Core)

Centmin Mod
----------------------------------------------
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                48
On-line CPU(s) list:   0-47
Thread(s) per core:    2
Core(s) per socket:    24
Socket(s):             1
NUMA node(s):          4
Vendor ID:             AuthenticAMD
CPU family:            23
Model:                 1
Model name:            AMD EPYC 7401P 24-Core Processor
Stepping:              2
CPU MHz:               1996.203
BogoMIPS:              3992.40
Virtualization:        AMD-V
L1d cache:             32K
L1i cache:             64K
L2 cache:              512K
L3 cache:              8192K
NUMA node0 CPU(s):     0,4,8,12,16,20,24,28,32,36,40,44
NUMA node1 CPU(s):     1,5,9,13,17,21,25,29,33,37,41,45
NUMA node2 CPU(s):     2,6,10,14,18,22,26,30,34,38,42,46
NUMA node3 CPU(s):     3,7,11,15,19,23,27,31,35,39,43,47
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca

CPU Flags
 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca

----------------------------------------------
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE
0   0    0      0    0:0:0:0       yes
1   1    0      1    1:1:1:1       yes
2   2    0      2    2:2:2:2       yes
3   3    0      3    3:3:3:3       yes
4   0    0      4    4:4:4:4       yes
5   1    0      5    5:5:5:5       yes
6   2    0      6    6:6:6:6       yes
7   3    0      7    7:7:7:7       yes
8   0    0      8    8:8:8:0       yes
9   1    0      9    9:9:9:1       yes
10  2    0      10   10:10:10:2    yes
11  3    0      11   11:11:11:3    yes
12  0    0      12   12:12:12:4    yes
13  1    0      13   13:13:13:5    yes
14  2    0      14   14:14:14:6    yes
15  3    0      15   15:15:15:7    yes
16  0    0      16   16:16:16:0    yes
17  1    0      17   17:17:17:1    yes
18  2    0      18   18:18:18:2    yes
19  3    0      19   19:19:19:3    yes
20  0    0      20   20:20:20:4    yes
21  1    0      21   21:21:21:5    yes
22  2    0      22   22:22:22:6    yes
23  3    0      23   23:23:23:7    yes
24  0    0      0    0:0:0:0       yes
25  1    0      1    1:1:1:1       yes
26  2    0      2    2:2:2:2       yes
27  3    0      3    3:3:3:3       yes
28  0    0      4    4:4:4:4       yes
29  1    0      5    5:5:5:5       yes
30  2    0      6    6:6:6:6       yes
31  3    0      7    7:7:7:7       yes
32  0    0      8    8:8:8:0       yes
33  1    0      9    9:9:9:1       yes
34  2    0      10   10:10:10:2    yes
35  3    0      11   11:11:11:3    yes
36  0    0      12   12:12:12:4    yes
37  1    0      13   13:13:13:5    yes
38  2    0      14   14:14:14:6    yes
39  3    0      15   15:15:15:7    yes
40  0    0      16   16:16:16:0    yes
41  1    0      17   17:17:17:1    yes
42  2    0      18   18:18:18:2    yes
43  3    0      19   19:19:19:3    yes
44  0    0      20   20:20:20:4    yes
45  1    0      21   21:21:21:5    yes
46  2    0      22   22:22:22:6    yes
47  3    0      23   23:23:23:7    yes

----------------------------------------------
              total        used        free      shared  buff/cache   available
Mem:          64159        3013       60590          30         555       60729
Low:          64159        3568       60590
High:             0           0           0
Swap:          1949           0        1949

----------------------------------------------
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdc3       108G  6.2G   96G   7% /
devtmpfs         32G  4.0K   32G   1% /dev
tmpfs            32G     0   32G   0% /dev/shm
tmpfs            32G   17M   32G   1% /run
tmpfs            32G     0   32G   0% /sys/fs/cgroup
/dev/sdc1       511M  148K  511M   1% /boot/efi
tmpfs           6.3G     0  6.3G   0% /run/user/0
tmpfs            32G  332K   32G   1% /tmp

    Code (Text):
    -------------------------------------------
disk ioping tests
-------------------------------------------

Running IOPing I/O benchmark...

IOPing I/O: ./ioping -c 10 .
4.0 KiB from . (ext4 /dev/sdc3): request=1 time=2.2 ms
4.0 KiB from . (ext4 /dev/sdc3): request=2 time=227 us
4.0 KiB from . (ext4 /dev/sdc3): request=3 time=272 us
4.0 KiB from . (ext4 /dev/sdc3): request=4 time=249 us
4.0 KiB from . (ext4 /dev/sdc3): request=5 time=213 us
4.0 KiB from . (ext4 /dev/sdc3): request=6 time=267 us
4.0 KiB from . (ext4 /dev/sdc3): request=7 time=269 us
4.0 KiB from . (ext4 /dev/sdc3): request=8 time=243 us
4.0 KiB from . (ext4 /dev/sdc3): request=9 time=251 us
4.0 KiB from . (ext4 /dev/sdc3): request=10 time=214 us

--- . (ext4 /dev/sdc3) ioping statistics ---
10 requests completed in 9.0 s, 2.3 k iops, 8.8 MiB/s
min/avg/max/mdev = 213 us / 442 us / 2.2 ms / 591 us

IOPing seek rate: ./ioping -RD .

--- . (ext4 /dev/sdc3) ioping statistics ---
15.9 k requests completed in 3.0 s, 5.6 k iops, 21.7 MiB/s
min/avg/max/mdev = 150 us / 180 us / 1.9 ms / 33 us

IOPing sequential: ./ioping -RL .

--- . (ext4 /dev/sdc3) ioping statistics ---
1.5 k requests completed in 3.0 s, 518 iops, 129.6 MiB/s
min/avg/max/mdev = 1.1 ms / 1.9 ms / 4.9 ms / 353 us

IOPing cached: ./ioping -RC .

--- . (ext4 /dev/sdc3) ioping statistics ---
645.6 k requests completed in 3.0 s, 228.2 k iops, 891.3 MiB/s
min/avg/max/mdev = 3 us / 4 us / 43 us / 0 us

    Code (Text):
    -------------------------------------------
disk DD tests
-------------------------------------------

dd if=/dev/zero of=sb-io-test bs=1M count=1k conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 6.02867 s, 178 MB/s

dd if=/dev/zero of=sb-io-test bs=64k count=16k conv=fdatasync
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 6.12825 s, 175 MB/s

dd if=/dev/zero of=sb-io-test bs=1M count=1k oflag=dsync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 5.65778 s, 190 MB/s

dd if=/dev/zero of=sb-io-test bs=64k count=16k oflag=dsync
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 11.2722 s, 95.3 MB/s

    Code (Text):
    -------------------------------------------
disk FIO tests
-------------------------------------------

Running FIO benchmark...

FIO_VERSION = fio-2.0.9

FIO random reads:
randomreads: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.0.9
Starting 1 process
randomreads: Laying out IO file(s) (1 file(s) / 1024MB)

randomreads: (groupid=0, jobs=1): err= 0: pid=67001: Mon Feb 26 08:46:57 2018
  read : io=1024.3MB, bw=172760KB/s, iops=43190 , runt=  6071msec
  cpu          : usr=8.95%, sys=58.42%, ctx=23870, majf=0, minf=251
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=262207/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=1024.3MB, aggrb=172760KB/s, minb=172760KB/s, maxb=172760KB/s, mint=6071msec, maxt=6071msec

Disk stats (read/write):
  sdc: ios=254729/2, merge=51/1, ticks=352718/22, in_queue=352676, util=98.37%

FIO random writes:
randomwrites: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.0.9
Starting 1 process

randomwrites: (groupid=0, jobs=1): err= 0: pid=67005: Mon Feb 26 08:47:04 2018
  write: io=1024.3MB, bw=148307KB/s, iops=37076 , runt=  7072msec
  cpu          : usr=7.00%, sys=50.12%, ctx=20903, majf=0, minf=185
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=0/w=262207/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=1024.3MB, aggrb=148307KB/s, minb=148307KB/s, maxb=148307KB/s, mint=7072msec, maxt=7072msec

Disk stats (read/write):
  sdc: ios=0/255776, merge=0/55, ticks=0/417556, in_queue=417513, util=98.63%

    Code (Text):
    -------------------------------------------
ramdisk DD tests - memory bandwidth
-------------------------------------------

dd if=/dev/zero of=sb-io-test bs=128k count=1k conv=fdatasync
1024+0 records in
1024+0 records out
134217728 bytes (134 MB) copied, 0.0490407 s, 2.7 GB/s

dd if=/dev/zero of=sb-io-test bs=8k count=16k conv=fdatasync
16384+0 records in
16384+0 records out
134217728 bytes (134 MB) copied, 0.186969 s, 718 MB/s

dd if=/dev/zero of=sb-io-test bs=128k count=1k oflag=dsync
1024+0 records in
1024+0 records out
134217728 bytes (134 MB) copied, 0.0483516 s, 2.8 GB/s

dd if=/dev/zero of=sb-io-test bs=8k count=16k oflag=dsync
16384+0 records in
16384+0 records out
134217728 bytes (134 MB) copied, 0.186472 s, 720 MB/s

    Code (Text):
    -------------------------------------------
ramdisk ioping tests - memory bandwidth
-------------------------------------------

Running IOPing I/O ramdisk benchmark...

IOPing I/O: ./ioping -c 10 .
4.0 KiB from . (tmpfs tmpfs): request=1 time=6 us
4.0 KiB from . (tmpfs tmpfs): request=2 time=5 us
4.0 KiB from . (tmpfs tmpfs): request=3 time=5 us
4.0 KiB from . (tmpfs tmpfs): request=4 time=4 us
4.0 KiB from . (tmpfs tmpfs): request=5 time=4 us
4.0 KiB from . (tmpfs tmpfs): request=6 time=4 us
4.0 KiB from . (tmpfs tmpfs): request=7 time=4 us
4.0 KiB from . (tmpfs tmpfs): request=8 time=5 us
4.0 KiB from . (tmpfs tmpfs): request=9 time=4 us
4.0 KiB from . (tmpfs tmpfs): request=10 time=8 us

--- . (tmpfs tmpfs) ioping statistics ---
10 requests completed in 9.0 s, 204.1 k iops, 797.2 MiB/s
min/avg/max/mdev = 4 us / 4 us / 8 us / 1 us

IOPing seek rate: ./ioping -R .

--- . (tmpfs tmpfs) ioping statistics ---
247.2 k requests completed in 3.0 s, 226.6 k iops, 885.2 MiB/s
min/avg/max/mdev = 4 us / 4 us / 18 us / 0 us

IOPing sequential: ./ioping -RL .

--- . (tmpfs tmpfs) ioping statistics ---
59.2 k requests completed in 3.0 s, 35.6 k iops, 8.7 GiB/s
min/avg/max/mdev = 26 us / 28 us / 60 us / 1 us

IOPing cached: ./ioping -RC .

--- . (tmpfs tmpfs) ioping statistics ---
632.4 k requests completed in 3.0 s, 223.4 k iops, 872.6 MiB/s
min/avg/max/mdev = 4 us / 4 us / 35 us / 0 us

    Code (Text):
    
-------------------------------------------
Running bandwidth benchmark...
-------------------------------------------

----------------------------------------------
Download from Cachefly (http://cachefly.cachefly.net/100mb.test)
Download Cachefly: 52.4MB/s

-------------------------------------------
USA bandwidth tests...
-------------------------------------------

----------------------------------------------
Download from Linode, Atlanta, GA, USA (http://speedtest.atlanta.linode.com/100MB-atlanta.bin)
Download Linode, Atlanta, GA, USA: 50.3MB/s
----------------------------------------------
Download from Linode, Dallas, TX, USA (http://speedtest.dallas.linode.com/100MB-dallas.bin)
Download Linode, Dallas, TX, USA: 35.2MB/s
----------------------------------------------
Download from Leaseweb, Manassas, VA, USA (http://mirror.us.leaseweb.net/speedtest/100mb.bin)
Download Leaseweb, Manassas, VA, USA: 52.6MB/s
----------------------------------------------
Download from Softlayer, Seattle, WA, USA (http://speedtest.sea01.softlayer.com/downloads/test100.zip)
Download Softlayer, Seattle, WA, USA: 25.6MB/s
----------------------------------------------
Download from Softlayer, San Jose, CA, USA (http://speedtest.sjc01.softlayer.com/downloads/test100.zip)
Download Softlayer, San Jose, CA, USA: 23.7MB/s
----------------------------------------------
Download from Softlayer, Washington, DC, USA (http://speedtest.wdc01.softlayer.com/downloads/test100.zip)
Download Softlayer, Washington, DC, USA: 20.8MB/s
----------------------------------------------
Download from VersaWeb, Las Vegas, Nevada (http://104.143.15.254/100MB.test)
Download VersaWeb, Las Vegas, Nevada: 4.96MB/s
----------------------------------------------
Download from OVH, BHS, Canada (http://bhs.proof.ovh.net/files/100Mio.dat)
Download OVH, BHS, Canada: 47.0MB/s
----------------------------------------------
Download from Vultr, Los Angeles, California (http://lax-ca-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Los Angeles, California: 25.9MB/s
----------------------------------------------
Download from Vultr, Seattle, Washington (http://wa-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Seattle, Washington: 31.6MB/s
----------------------------------------------
Download from Vultr, Dallas, Texas (http://tx-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Dallas, Texas: 45.8MB/s
----------------------------------------------
Download from Vultr, Chicago, Illinois (http://il-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Chicago, Illinois: 50.0MB/s
----------------------------------------------
Download from Vultr, Atlanta, Georgia (http://ga-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Atlanta, Georgia: 49.9MB/s
----------------------------------------------
Download from Vultr, Miami, Florida (http://fl-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Miami, Florida: 47.9MB/s
----------------------------------------------
Download from Vultr, New York / New Jersey (http://nj-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, New York / New Jersey: 50.3MB/s

-------------------------------------------
Asia bandwidth tests...
-------------------------------------------

----------------------------------------------
Download from Linode, Tokyo, JP (http://speedtest.tokyo.linode.com/100MB-tokyo.bin)
Download Linode, Tokyo, JP: 13.5MB/s
----------------------------------------------
Download from Softlayer, Singapore (http://speedtest.sng01.softlayer.com/downloads/test100.zip)
Download Softlayer, Singapore: 6.82MB/s
----------------------------------------------
Download from Vultr, Tokyo, Japan (http://hnd-jp-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Tokyo, Japan: 13.0MB/s

-------------------------------------------
Europe bandwidth tests...
-------------------------------------------

----------------------------------------------
Download from Linode, London, UK (http://speedtest.london.linode.com/100MB-london.bin)
Download Linode, London, UK: 33.0MB/s
----------------------------------------------
Download from OVH, Paris, France (http://proof.ovh.net/files/100Mio.dat)
Download OVH, Paris, France: 8.11MB/s
----------------------------------------------
Download from SmartDC, Rotterdam, Netherlands (http://mirror.i3d.net/100mb.bin)
Download SmartDC, Rotterdam, Netherlands: 7.78MB/s
----------------------------------------------
Download from Vultr, Amsterdam, Netherlands (http://ams-nl-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Amsterdam, Netherlands: 29.3MB/s
----------------------------------------------
Download from Vultr, London, UK (http://lon-gb-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, London, UK: 32.1MB/s
----------------------------------------------
Download from Vultr, Paris, France (http://par-fr-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Paris, France: 26.9MB/s

-------------------------------------------
Australia bandwidth tests...
-------------------------------------------

----------------------------------------------
Download from Vultr, Sydney, Australia (http://syd-au-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Sydney, Australia: 9.70MB/s

    Code (Text):
    
-------------------------------------------
Running Axel multi-threaded bandwidth benchmark...
-------------------------------------------

----------------------------------------------
Axel Download from Cachefly (http://cachefly.cachefly.net/100mb.test)
Axel Download Cachefly: 128.00MB/s

-------------------------------------------
USA bandwidth tests...
-------------------------------------------

----------------------------------------------
Axel Download from Linode, Atlanta, GA, USA (http://speedtest.atlanta.linode.com/100MB-atlanta.bin)
Axel Download Linode, Atlanta, GA, USA: 113.57MB/s
----------------------------------------------
Axel Download from Linode, Dallas, TX, USA (http://speedtest.dallas.linode.com/100MB-dallas.bin)
Axel Download Linode, Dallas, TX, USA: 55.08MB/s
----------------------------------------------
Axel Download from Leaseweb, Manassas, VA, USA (http://mirror.us.leaseweb.net/speedtest/100mb.bin)
Axel Download Leaseweb, Manassas, VA, USA: 59.42MB/s
----------------------------------------------
Axel Download from Softlayer, Seattle, WA, USA (http://speedtest.sea01.softlayer.com/downloads/test100.zip)
Axel Download Softlayer, Seattle, WA, USA: 25.45MB/s
----------------------------------------------
Axel Download from Softlayer, San Jose, CA, USA (http://speedtest.sjc01.softlayer.com/downloads/test100.zip)
Axel Download Softlayer, San Jose, CA, USA: 22.61MB/s
----------------------------------------------
Axel Download from Softlayer, Washington, DC, USA (http://speedtest.wdc01.softlayer.com/downloads/test100.zip)
Axel Download Softlayer, Washington, DC, USA: 31.35MB/s
----------------------------------------------
Axel Download from VersaWeb, Las Vegas, Nevada (http://104.143.15.254/100MB.test)
Axel Download VersaWeb, Las Vegas, Nevada: 14.47MB/s
----------------------------------------------
Axel Download from OVH, BHS, Canada (http://bhs.proof.ovh.net/files/100Mio.dat)
Axel Download OVH, BHS, Canada: 94.33MB/s
----------------------------------------------
Axel Download from Vultr, Los Angeles, California (http://lax-ca-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Los Angeles, California: 50.08MB/s
----------------------------------------------
Axel Download from Vultr, Seattle, Washington (http://wa-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Seattle, Washington: 58.48MB/s
----------------------------------------------
Axel Download from Vultr, Dallas, Texas (http://tx-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Dallas, Texas: 65.64MB/s
----------------------------------------------
Axel Download from Vultr, Chicago, Illinois (http://il-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Chicago, Illinois: 126.54MB/s
----------------------------------------------
Axel Download from Vultr, Atlanta, Georgia (http://ga-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Atlanta, Georgia: 113.29MB/s
----------------------------------------------
Axel Download from Vultr, Miami, Florida (http://fl-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Miami, Florida: 106.75MB/s
----------------------------------------------
Axel Download from Vultr, New York / New Jersey (http://nj-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, New York / New Jersey: 168.45MB/s

-------------------------------------------
Asia bandwidth tests...
-------------------------------------------

----------------------------------------------
Axel Download from Linode, Tokyo, JP (http://speedtest.tokyo.linode.com/100MB-tokyo.bin)
Axel Download Linode, Tokyo, JP: 14.44MB/s
----------------------------------------------
Axel Download from Softlayer, Singapore (http://speedtest.sng01.softlayer.com/downloads/test100.zip)
Axel Download Softlayer, Singapore: 6.53MB/s
----------------------------------------------
Axel Download from Vultr, Tokyo, Japan (http://hnd-jp-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Tokyo, Japan: 18.89MB/s

-------------------------------------------
Europe bandwidth tests...
-------------------------------------------

----------------------------------------------
Axel Download from Linode, London, UK (http://speedtest.london.linode.com/100MB-london.bin)
Axel Download Linode, London, UK: 32.40MB/s
----------------------------------------------
Axel Download from OVH, Paris, France (http://proof.ovh.net/files/100Mio.dat)
Axel Download OVH, Paris, France: 21.19MB/s
----------------------------------------------
Axel Download from SmartDC, Rotterdam, Netherlands (http://mirror.i3d.net/100mb.bin)
Axel Download SmartDC, Rotterdam, Netherlands: 12.86MB/s
----------------------------------------------
Axel Download from Vultr, Amsterdam, Netherlands (http://ams-nl-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Amsterdam, Netherlands: 52.78MB/s
----------------------------------------------
Axel Download from Vultr, London, UK (http://lon-gb-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, London, UK: 48.78MB/s
----------------------------------------------
Axel Download from Vultr, Paris, France (http://par-fr-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Paris, France: 31.15MB/s

-------------------------------------------
Australia bandwidth tests...
-------------------------------------------

----------------------------------------------
Axel Download from Vultr, Sydney, Australia (http://syd-au-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Sydney, Australia: 17.93MB/s

    Code (Text):
    
-------------------------------------------
Running ping tests...
-------------------------------------------

----------------------------------------------
Pings (cachefly.cachefly.net):
PING vip1.g5.cachefly.net (205.234.175.175) 56(84) bytes of data.
64 bytes from vip1.G-anycast1.cachefly.net (205.234.175.175): icmp_seq=1 ttl=58 time=0.969 ms
64 bytes from vip1.G-anycast1.cachefly.net (205.234.175.175): icmp_seq=2 ttl=58 time=0.968 ms
64 bytes from vip1.G-anycast1.cachefly.net (205.234.175.175): icmp_seq=3 ttl=58 time=0.991 ms

--- vip1.g5.cachefly.net ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 0.968/0.976/0.991/0.010 ms

----------------------------------------------
Pings (syd-au-ping.vultr.com):
PING syd-au-ping.vultr.com (108.61.212.117) 56(84) bytes of data.
64 bytes from 108.61.212.117.vultr.com (108.61.212.117): icmp_seq=1 ttl=51 time=222 ms
64 bytes from 108.61.212.117.vultr.com (108.61.212.117): icmp_seq=2 ttl=51 time=222 ms
64 bytes from 108.61.212.117.vultr.com (108.61.212.117): icmp_seq=3 ttl=51 time=223 ms

--- syd-au-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 222.735/222.875/223.076/0.412 ms

----------------------------------------------
Pings (hnd-jp-ping.vultr.com):
PING hnd-jp-ping.vultr.com (108.61.201.151) 56(84) bytes of data.
64 bytes from 108.61.201.151.vultr.com (108.61.201.151): icmp_seq=1 ttl=50 time=175 ms
64 bytes from 108.61.201.151.vultr.com (108.61.201.151): icmp_seq=2 ttl=50 time=175 ms
64 bytes from 108.61.201.151.vultr.com (108.61.201.151): icmp_seq=3 ttl=50 time=175 ms

--- hnd-jp-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 175.640/175.652/175.663/0.484 ms

----------------------------------------------
Pings (lax-ca-us-ping.vultr.com):
PING lax-ca-us-ping.vultr.com (108.61.219.200) 56(84) bytes of data.
64 bytes from 108.61.219.200.vultr.com (108.61.219.200): icmp_seq=1 ttl=53 time=87.4 ms
64 bytes from 108.61.219.200.vultr.com (108.61.219.200): icmp_seq=2 ttl=53 time=87.4 ms
64 bytes from 108.61.219.200.vultr.com (108.61.219.200): icmp_seq=3 ttl=53 time=87.4 ms

--- lax-ca-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 87.433/87.456/87.471/0.242 ms

----------------------------------------------
Pings (wa-us-ping.vultr.com):
PING wa-us-ping.vultr.com (108.61.194.105) 56(84) bytes of data.
64 bytes from 108.61.194.105.vultr.com (108.61.194.105): icmp_seq=1 ttl=53 time=71.4 ms
64 bytes from 108.61.194.105.vultr.com (108.61.194.105): icmp_seq=2 ttl=53 time=71.4 ms
64 bytes from 108.61.194.105.vultr.com (108.61.194.105): icmp_seq=3 ttl=53 time=71.5 ms

--- wa-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 71.472/71.494/71.514/0.309 ms

----------------------------------------------
Pings (tx-us-ping.vultr.com):
PING tx-us-ping.vultr.com (108.61.224.175) 56(84) bytes of data.
64 bytes from 108.61.224.175.vultr.com (108.61.224.175): icmp_seq=1 ttl=52 time=39.1 ms
64 bytes from 108.61.224.175.vultr.com (108.61.224.175): icmp_seq=2 ttl=52 time=39.1 ms
64 bytes from 108.61.224.175.vultr.com (108.61.224.175): icmp_seq=3 ttl=52 time=39.1 ms

--- tx-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 39.132/39.154/39.195/0.029 ms

----------------------------------------------
Pings (il-us-ping.vultr.com):
PING il-us-ping.vultr.com (107.191.51.12) 56(84) bytes of data.
64 bytes from 107.191.51.12.vultr.com (107.191.51.12): icmp_seq=1 ttl=54 time=24.9 ms
64 bytes from 107.191.51.12.vultr.com (107.191.51.12): icmp_seq=2 ttl=54 time=24.9 ms
64 bytes from 107.191.51.12.vultr.com (107.191.51.12): icmp_seq=3 ttl=54 time=24.9 ms

--- il-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 24.969/24.970/24.972/0.182 ms

----------------------------------------------
Pings (ga-us-ping.vultr.com):
PING ga-us-ping.vultr.com (108.61.193.166) 56(84) bytes of data.
64 bytes from 108.61.193.166.vultr.com (108.61.193.166): icmp_seq=1 ttl=53 time=18.5 ms
64 bytes from 108.61.193.166.vultr.com (108.61.193.166): icmp_seq=2 ttl=53 time=18.6 ms
64 bytes from 108.61.193.166.vultr.com (108.61.193.166): icmp_seq=3 ttl=53 time=18.6 ms

--- ga-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 18.590/18.629/18.680/0.162 ms

----------------------------------------------
Pings (fl-us-ping.vultr.com):
PING fl-us-ping.vultr.com (104.156.244.232) 56(84) bytes of data.
64 bytes from 104.156.244.232 (104.156.244.232): icmp_seq=1 ttl=53 time=34.0 ms
64 bytes from 104.156.244.232 (104.156.244.232): icmp_seq=2 ttl=53 time=34.0 ms
64 bytes from 104.156.244.232 (104.156.244.232): icmp_seq=3 ttl=53 time=34.0 ms

--- fl-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 34.010/34.020/34.037/0.012 ms

----------------------------------------------
Pings (nj-us-ping.vultr.com):
PING nj-us-ping.vultr.com (108.61.149.182) 56(84) bytes of data.
64 bytes from 108.61.149.182.vultr.com (108.61.149.182): icmp_seq=1 ttl=56 time=2.01 ms
64 bytes from 108.61.149.182.vultr.com (108.61.149.182): icmp_seq=2 ttl=57 time=2.04 ms
64 bytes from 108.61.149.182.vultr.com (108.61.149.182): icmp_seq=3 ttl=56 time=2.13 ms

--- nj-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 2.019/2.065/2.132/0.071 ms

----------------------------------------------
Pings (fra-de-ping.vultr.com):
PING fra-de-ping.vultr.com (108.61.210.117) 56(84) bytes of data.
64 bytes from 108.61.210.117.vultr.com (108.61.210.117): icmp_seq=1 ttl=51 time=81.5 ms
64 bytes from 108.61.210.117.vultr.com (108.61.210.117): icmp_seq=2 ttl=51 time=81.5 ms
64 bytes from 108.61.210.117.vultr.com (108.61.210.117): icmp_seq=3 ttl=51 time=81.6 ms

--- fra-de-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 81.558/81.587/81.622/0.234 ms

----------------------------------------------
Pings (ams-nl-ping.vultr.com):
PING ams-nl-ping.vultr.com (108.61.198.102) 56(84) bytes of data.
64 bytes from 108.61.198.102.vultr.com (108.61.198.102): icmp_seq=1 ttl=53 time=79.5 ms
64 bytes from 108.61.198.102.vultr.com (108.61.198.102): icmp_seq=2 ttl=53 time=79.4 ms
64 bytes from 108.61.198.102.vultr.com (108.61.198.102): icmp_seq=3 ttl=53 time=79.4 ms

--- ams-nl-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 79.457/79.484/79.529/0.327 ms

----------------------------------------------
Pings (lon-gb-ping.vultr.com):
PING lon-gb-ping.vultr.com (108.61.196.101) 56(84) bytes of data.
64 bytes from 108.61.196.101.vultr.com (108.61.196.101): icmp_seq=1 ttl=53 time=71.2 ms
64 bytes from 108.61.196.101.vultr.com (108.61.196.101): icmp_seq=2 ttl=53 time=71.0 ms
64 bytes from 108.61.196.101.vultr.com (108.61.196.101): icmp_seq=3 ttl=53 time=71.2 ms

--- lon-gb-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 71.096/71.205/71.265/0.077 ms

----------------------------------------------
Pings (par-fr-ping.vultr.com):
PING par-fr-ping.vultr.com (108.61.209.127) 56(84) bytes of data.
64 bytes from 108.61.209.127.vultr.com (108.61.209.127): icmp_seq=1 ttl=54 time=85.4 ms
64 bytes from 108.61.209.127.vultr.com (108.61.209.127): icmp_seq=2 ttl=54 time=85.4 ms
64 bytes from 108.61.209.127.vultr.com (108.61.209.127): icmp_seq=3 ttl=54 time=85.4 ms

--- par-fr-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 85.412/85.441/85.467/0.022 ms

----------------------------------------------
Pings (VersaWeb Las Vegas):
PING 104.143.15.254 (104.143.15.254) 56(84) bytes of data.
64 bytes from 104.143.15.254: icmp_seq=1 ttl=48 time=76.2 ms
64 bytes from 104.143.15.254: icmp_seq=2 ttl=48 time=76.2 ms
64 bytes from 104.143.15.254: icmp_seq=3 ttl=48 time=76.2 ms

--- 104.143.15.254 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 76.230/76.235/76.239/0.225 ms

----------------------------------------------
Pings (VersaWeb Seattle):
PING 76.164.234.1 (76.164.234.1) 56(84) bytes of data.
64 bytes from 76.164.234.1: icmp_seq=1 ttl=54 time=66.4 ms
64 bytes from 76.164.234.1: icmp_seq=2 ttl=54 time=66.4 ms
64 bytes from 76.164.234.1: icmp_seq=3 ttl=54 time=66.4 ms

--- 76.164.234.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 66.444/66.476/66.499/0.298 ms

----------------------------------------------
Pings (OVH Canada):
PING bhs.proof.ovh.net (192.99.19.165) 56(84) bytes of data.
64 bytes from bhs.proof.ovh.net (192.99.19.165): icmp_seq=1 ttl=56 time=8.91 ms
64 bytes from bhs.proof.ovh.net (192.99.19.165): icmp_seq=2 ttl=56 time=8.87 ms
64 bytes from bhs.proof.ovh.net (192.99.19.165): icmp_seq=3 ttl=56 time=8.88 ms

--- bhs.proof.ovh.net ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 8.871/8.890/8.918/0.020 ms

----------------------------------------------
Pings (10gbps.io Frankfurt):
PING 195.181.174.20 (195.181.174.20) 56(84) bytes of data.
64 bytes from 195.181.174.20: icmp_seq=1 ttl=51 time=93.1 ms
64 bytes from 195.181.174.20: icmp_seq=2 ttl=51 time=93.1 ms
64 bytes from 195.181.174.20: icmp_seq=3 ttl=51 time=93.1 ms

--- 195.181.174.20 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 93.110/93.143/93.180/0.028 ms

----------------------------------------------
Pings (10gbps.io London):
PING 185.59.221.20 (185.59.221.20) 56(84) bytes of data.
64 bytes from 185.59.221.20: icmp_seq=1 ttl=51 time=94.5 ms
64 bytes from 185.59.221.20: icmp_seq=2 ttl=51 time=94.6 ms
64 bytes from 185.59.221.20: icmp_seq=3 ttl=51 time=94.5 ms

--- 185.59.221.20 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 94.562/94.591/94.618/0.355 ms

----------------------------------------------
Pings (10gbps.io Paris):
PING 185.93.2.193 (185.93.2.193) 56(84) bytes of data.
64 bytes from 185.93.2.193: icmp_seq=1 ttl=51 time=101 ms
64 bytes from 185.93.2.193: icmp_seq=2 ttl=51 time=101 ms
64 bytes from 185.93.2.193: icmp_seq=3 ttl=51 time=101 ms

--- 185.93.2.193 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 101.089/101.165/101.214/0.054 ms

----------------------------------------------
Pings (10gbps.io Singapore):
PING 203.174.85.38 (203.174.85.38) 56(84) bytes of data.
64 bytes from 203.174.85.38: icmp_seq=1 ttl=51 time=251 ms
64 bytes from 203.174.85.38: icmp_seq=2 ttl=51 time=251 ms
64 bytes from 203.174.85.38: icmp_seq=3 ttl=51 time=251 ms

--- 203.174.85.38 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 251.396/251.576/251.931/0.480 ms

----------------------------------------------
Pings (10gbps.io Stockholm):
PING 185.76.9.10 (185.76.9.10) 56(84) bytes of data.
64 bytes from 185.76.9.10: icmp_seq=1 ttl=53 time=108 ms
64 bytes from 185.76.9.10: icmp_seq=2 ttl=53 time=108 ms
64 bytes from 185.76.9.10: icmp_seq=3 ttl=53 time=108 ms

--- 185.76.9.10 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 108.173/108.207/108.234/0.380 ms

----------------------------------------------
Pings (10gbps.io Moscow):
PING 87.245.204.20 (87.245.204.20) 56(84) bytes of data.
64 bytes from 87.245.204.20: icmp_seq=1 ttl=59 time=113 ms
64 bytes from 87.245.204.20: icmp_seq=2 ttl=59 time=113 ms
64 bytes from 87.245.204.20: icmp_seq=3 ttl=59 time=113 ms

--- 87.245.204.20 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 113.478/113.496/113.525/0.275 ms

----------------------------------------------
Pings (10gbps.io Miami):
PING 195.181.162.195 (195.181.162.195) 56(84) bytes of data.
64 bytes from 195.181.162.195: icmp_seq=1 ttl=55 time=55.9 ms
64 bytes from 195.181.162.195: icmp_seq=2 ttl=55 time=55.8 ms
64 bytes from 195.181.162.195: icmp_seq=3 ttl=55 time=55.6 ms

--- 195.181.162.195 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 55.669/55.849/55.982/0.303 ms

    Code (Text):
    
-------------------------------------------
Running mtr tests...
-------------------------------------------

----------------------------------------------
mtr --report -w 200 --report-cycles=10 cachefly.cachefly.net
Start: Mon Feb 26 08:52:11 2018
HOST: hostname          Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                       0.0%    10    7.8  19.0   7.8  23.6   4.3
  2.|-- 147.75.36.4                   0.0%    10   78.4 155.8  12.7 1040. 318.4
  3.|-- 0.ae12.bbr1.ewr1.packet.net   0.0%    10    0.3   3.6   0.2  33.0  10.3
  4.|-- ???                          100.0    10    0.0   0.0   0.0   0.0   0.0
  5.|-- nyk-b6-link.telia.net         0.0%    10    1.0   1.1   1.0   2.0   0.0
  6.|-- nyk-b5-link.telia.net         0.0%    10    1.3   1.1   1.0   1.3   0.0
  7.|-- ???                          100.0    10    0.0   0.0   0.0   0.0   0.0
  8.|-- vip1.G-anycast1.cachefly.net  0.0%    10    1.0   1.0   1.0   1.0   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.atlanta.linode.com
Start: Mon Feb 26 08:52:27 2018
HOST: hostname                  Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                               0.0%    10    8.8   6.2   1.0  11.0   3.8
  2.|-- 2604:1380:2:79::                      0.0%    10    4.6   6.2   0.9  11.2   3.3
  3.|-- 2604:1380:2:20::                      0.0%    10    0.2   0.2   0.2   0.4   0.0
  4.|-- 2001:518:6900:1::5                    0.0%    10    1.7   0.7   0.4   1.7   0.3
  5.|-- 2001:518:2001:15::1                   0.0%    10    0.3   0.8   0.2   4.8   1.2
  6.|-- 2001:518:6001:6::2                    0.0%    10    6.6   6.5   1.1  12.0   4.1
  7.|-- nyk-b3-link.telia.net                 0.0%    10    1.2   1.2   1.1   1.2   0.0
  8.|-- nyk-bb4-v6.telia.net                  0.0%    10   21.2  21.2  21.1  21.3   0.0
  9.|-- ash-bb4-v6.telia.net                  0.0%    10   21.6  22.8  21.6  32.1   3.2
 10.|-- atl-b22-v6.telia.net                  0.0%    10   32.9  25.3  24.1  32.9   2.7
 11.|-- linode-ic-321031-atl-b22.c.telia.net  0.0%    10   21.9  21.9  21.8  22.0   0.0
 12.|-- 2600:3c02:4444:4::2                   0.0%    10   21.9  22.0  21.8  22.2   0.0
 13.|-- 2600:3c02::4b                         0.0%    10   21.5  21.5  21.4  21.7   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.dallas.linode.com
Start: Mon Feb 26 08:52:44 2018
HOST: hostname           Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                        0.0%    10   17.8  21.1  15.1  24.9   3.0
  2.|-- 2604:1380:2:79::               0.0%    10    2.4  10.4   1.0  61.2  18.1
  3.|-- 2604:1380:2:20::               0.0%    10    0.2   0.2   0.2   0.3   0.0
  4.|-- 2001:518:6900:1::5             0.0%    10    0.5  15.7   0.4 134.0  41.8
  5.|-- 2001:518:2001:16::1            0.0%    10    1.6   0.4   0.2   1.6   0.3
  6.|-- 2001:518:6001:7::2             0.0%    10    1.1   1.1   1.1   1.1   0.0
  7.|-- ae2-63.cr0-nyc2.ip6.gtt.net    0.0%    10   70.1  70.1  70.0  70.4   0.0
  8.|-- 2001:668:0:2:ffff:0:5995:8f3d  0.0%    10   80.3  80.3  80.2  80.5   0.0
  9.|-- 2001:668:0:3:ffff:2:0:26a      0.0%    10   42.2  42.2  42.1  42.5   0.0
 10.|-- 2600:3c00:2222:13::2           0.0%    10   42.8  42.8  42.6  42.9   0.0
 11.|-- speedtest.dallas.linode.com    0.0%    10   45.0  45.0  44.9  45.2   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 mirror.us.leaseweb.net
Start: Mon Feb 26 08:53:00 2018
HOST: hostname           Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                        0.0%    10   30.4  20.2  11.5  41.6  10.3
  2.|-- 2604:1380:2:78::               0.0%    10    1.1   6.5   1.1  11.4   3.6
  3.|-- 2604:1380:2:2::1               0.0%    10    6.1   5.6   0.3  40.5  12.5
  4.|-- 2001:518:6900:1::1             0.0%    10  162.9  18.6   0.4 162.9  51.0
  5.|-- 2001:518:2001:14::1            0.0%    10    0.2   0.3   0.2   1.5   0.3
  6.|-- 2001:518:6001:7::2             0.0%    10    1.1   1.1   1.0   1.1   0.0
  7.|-- ae2-63.cr0-nyc2.ip6.gtt.net    0.0%    10   70.0  70.2  70.0  71.2   0.3
  8.|-- 2001:668:0:2:ffff:0:d5fe:d6aa  0.0%    10   76.4  76.4  76.3  76.6   0.0
  9.|-- 2001:668:0:3:ffff:0:45ae:326   0.0%    10   52.2  52.1  52.0  52.2   0.0
 10.|-- 2604:9a00::3d                  0.0%    10   33.8  33.8  33.7  33.8   0.0
 11.|-- 2604:9a00:2010:a0b8::5         0.0%    10   34.0  34.0  33.9  34.1   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.sea01.softlayer.com
Start: Mon Feb 26 08:53:16 2018
HOST: hostname                                               Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                                            0.0%    10   23.2  34.2   6.8 118.8  38.0
  2.|-- 2604:1380:2:79::                                                   0.0%    10    8.4   6.3   2.4  10.6   2.7
  3.|-- 2604:1380:2:20::                                                   0.0%    10    0.3   0.2   0.2   0.3   0.0
  4.|-- de-cix.nyc.us.ibm.com                                              0.0%    10    1.1   1.4   1.1   3.2   0.5
  5.|-- 6320.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10    1.8   1.9   1.7   2.3   0.0
  6.|-- a320.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   22.8  22.9  22.7  23.2   0.0
  7.|-- 5220.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   23.5  23.5  23.0  26.8   1.1
  8.|-- ee20.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   64.5  64.5  63.9  64.9   0.0
  9.|-- 1630.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   62.5  62.6  62.5  62.7   0.0
 10.|-- po3.bbr01.sr01.sea01.networklayer.com                              0.0%    10   62.4  62.5  62.4  63.1   0.0
 11.|-- po1.fcr01.sr01.sea01.networklayer.com                              0.0%    10   63.3  65.0  63.3  74.9   3.8
 12.|-- softlayer.com                                                      0.0%    10   62.8  62.8  62.8  62.9   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.sjc01.softlayer.com
Start: Mon Feb 26 08:53:33 2018
HOST: hostname                                               Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                                            0.0%    10   20.6  26.6  12.5  57.1  13.9
  2.|-- 2604:1380:2:79::                                                   0.0%    10    6.1   8.5   1.3  41.0  11.8
  3.|-- 2604:1380:2:20::                                                   0.0%    10    0.2   1.3   0.2  10.1   3.1
  4.|-- de-cix.nyc.us.ibm.com                                              0.0%    10    1.2   1.1   1.1   1.2   0.0
  5.|-- 6320.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10    2.6   2.6   1.7   5.0   0.9
  6.|-- a320.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   22.6  22.7  22.5  22.9   0.0
  7.|-- 5220.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   22.9  22.9  22.7  23.4   0.0
  8.|-- 8320.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   45.1  44.8  44.6  45.2   0.0
  9.|-- 3620.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   67.0  67.1  66.9  67.4   0.0
 10.|-- 3520.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   79.4  77.8  69.1 133.0  19.7
 11.|-- 9700.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   66.7  67.2  66.6  69.3   0.9
 12.|-- 2607:f0d0:2100:1::3                                                0.0%    10   68.9  71.1  68.6  84.9   5.0
 13.|-- softlayer.com                                                      0.0%    10   70.9  70.9  70.9  71.0   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.wdc01.softlayer.com
Start: Mon Feb 26 08:53:50 2018
HOST: hostname                                               Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                                            0.0%    10   11.3   6.2   0.5  11.3   3.2
  2.|-- 2604:1380:2:78::                                                   0.0%    10   10.3   7.1   2.2  14.9   3.6
  3.|-- 2604:1380:2:5::1                                                   0.0%    10    0.2   0.9   0.2   7.2   2.1
  4.|-- de-cix.nyc.us.ibm.com                                              0.0%    10    1.2   1.2   1.1   1.6   0.0
  5.|-- 2320.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10    2.2   1.8   1.7   2.2   0.0
  6.|-- 2830.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10    9.9   7.9   7.1   9.9   0.9
  7.|-- e730.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10    9.1   9.7   7.4  26.8   6.0
  8.|-- bbr01.eq01.wdc02.networklayer.com                                  0.0%    10    7.1   8.3   7.1  12.5   2.0
  9.|-- po2.dar01.sr01.wdc01.networklayer.com                              0.0%    10    7.1   7.2   7.1   7.5   0.0
 10.|-- po1.fcr01.sr01.wdc01.networklayer.com                              0.0%    10    7.4   7.6   7.3   8.1   0.0
 11.|-- softlayer.com                                                      0.0%    10    7.5   7.5   7.4   7.8   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.tokyo.linode.com
Start: Mon Feb 26 08:54:06 2018
HOST: hostname          Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                       0.0%    10    0.6   5.2   0.6  10.2   3.1
  2.|-- 2604:1380:2:78::              0.0%    10    0.8   9.0   0.8  31.2   8.7
  3.|-- 2604:1380:2:5::1              0.0%    10    0.2   4.4   0.2  24.3   8.7
  4.|-- 10ge3-9.core1.nyc6.he.net     0.0%    10    1.2   1.1   1.1   1.2   0.0
  5.|-- peering.kddi.com              0.0%    10   80.2  79.9  76.7  94.6   5.4
  6.|-- 6nytjbb001.int-gw.kddi.ne.jp  0.0%    10   79.9  80.3  79.8  83.0   0.7
  7.|-- 6lacGCS001.int-gw.kddi.ne.jp  0.0%    10   65.6  68.6  65.5  78.6   4.9
  8.|-- 2001:268:fb80:300::1          0.0%    10  239.5 239.2 238.3 240.0   0.0
  9.|-- 2001:268:fa00:282::2          0.0%    10  255.9 254.5 237.8 267.4   9.7
 10.|-- 6cm-fcu204.int-gw.kddi.ne.jp  0.0%    10  250.7 242.5 238.4 250.7   4.6
 11.|-- 2001:268:f702:6d::2           0.0%    10  243.7 243.0 242.9 243.7   0.0
 12.|-- speedtest.tokyo.linode.com    0.0%    10  238.6 238.6 238.5 238.7   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.sng01.softlayer.com
Start: Mon Feb 26 08:54:28 2018
HOST: hostname                                               Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                                            0.0%    10    8.4   5.1   0.6   9.4   2.8
  2.|-- 2604:1380:2:78::                                                   0.0%    10   10.7   3.9   0.6  10.7   3.0
  3.|-- 2604:1380:2:2::1                                                   0.0%    10    0.2   0.7   0.2   5.1   1.4
  4.|-- 2604:1380:2:1::                                                    0.0%    10    0.4   1.5   0.2  11.8   3.6
  5.|-- de-cix.nyc.us.ibm.com                                              0.0%    10    1.3   1.2   1.2   1.3   0.0
  6.|-- 6320.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10    1.8   2.5   1.7   6.9   1.6
  7.|-- a320.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   22.8  22.8  22.7  23.2   0.0
  8.|-- 5220.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   23.5  23.6  23.0  25.2   0.5
  9.|-- 8320.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   47.4  48.8  47.3  60.1   3.9
 10.|-- a120.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   47.8  47.7  47.3  48.2   0.0
 11.|-- c520.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   72.8  73.6  72.7  79.7   2.0
 12.|-- 8b40.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10  214.8 215.1 214.7 217.0   0.5
 13.|-- 7940.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10  240.4 240.8 240.4 243.3   0.6
 14.|-- b840.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10  242.0 240.7 240.3 242.0   0.3
 15.|-- db00.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10  240.4 240.4 240.3 240.6   0.0
 16.|-- 3.00.0000.ip4.static.sl-reverse.com                                0.0%    10  240.4 240.7 240.4 242.6   0.6
 17.|-- 2.00.0000.ip4.static.sl-reverse.com                                0.0%    10  240.3 240.4 240.3 240.7   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.london.linode.com
Start: Mon Feb 26 08:54:45 2018
HOST: hostname                 Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                              0.0%    10    3.3   4.3   1.4   9.6   2.3
  2.|-- 2604:1380:2:78::                     0.0%    10    0.6   6.1   0.6  10.4   3.1
  3.|-- 2604:1380:2:2::1                     0.0%    10    0.3   2.2   0.2  19.8   6.2
  4.|-- 2001:518:6900:1::1                   0.0%    10    0.5   3.9   0.4  35.3  11.0
  5.|-- 2001:518:2001:13::1                  0.0%    10    0.2   1.6   0.2   4.7   1.7
  6.|-- 2001:518:6001:6::2                   0.0%    10    1.1   1.7   1.0   3.6   0.7
  7.|-- nyk-b3-link.telia.net                0.0%    10    1.2   1.2   1.1   1.3   0.0
  8.|-- nyk-bb4-v6.telia.net                 0.0%    10   70.7  70.8  70.7  71.1   0.0
  9.|-- 2001:2000:3019:7b::1                 0.0%    10   70.9  71.1  70.7  72.9   0.6
 10.|-- ldn-bb2-v6.telia.net                 0.0%    10   71.4  83.2  70.6 126.6  18.9
 11.|-- ldn-b4-v6.telia.net                  0.0%    10   70.6  77.0  70.6 112.0  13.1
 12.|-- linode-ic-318542-ldn-b3.c.telia.net  0.0%    10   73.5  71.8  71.4  73.5   0.5
 13.|-- 2a01:7e00:7777:4::2                  0.0%    10   71.5  71.5  71.5  71.6   0.0
 14.|-- speedtest.london.linode.com          0.0%    10   71.0  71.0  71.0  71.3   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 mirror.i3d.net
Start: Mon Feb 26 08:55:02 2018
HOST: hostname              Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                           0.0%    10    7.1  38.8   4.9 118.9  41.6
  2.|-- 2604:1380:2:78::                  0.0%    10    9.1   6.7   1.2  11.2   3.3
  3.|-- 2604:1380:2:2::1                  0.0%    10    0.2   2.5   0.2  22.3   6.9
  4.|-- 2001:518:6900:1::1                0.0%    10  164.9  62.8   0.4 198.8  83.8
  5.|-- 2001:518:2001:14::1               0.0%    10    0.4   0.3   0.2   0.5   0.0
  6.|-- 2001:518:1001:7::2                0.0%    10    9.3   5.1   0.9  11.7   4.5
  7.|-- 10ge.eqx-ashburn.misi3d.com       0.0%    10   10.2  12.2   6.1  16.9   3.7
  8.|-- 10ge.lr0-iadr0.th-n.lon.i3d.net   0.0%    10   82.0  83.7  77.8  89.0   3.8
  9.|-- 30ge.cr1-lr0.smartdc.rtm.i3d.net  0.0%    10   84.4  84.4  84.4  84.5   0.0
 10.|-- 80ge.br2-cr1.smartdc.rtd.i3d.net  0.0%    10   84.4  84.4  84.4  84.4   0.0
 11.|-- hosted-by.i3d.net                 0.0%    10   84.4  84.5  84.4  84.6   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 syd-au-ping.vultr.com
Start: Mon Feb 26 08:55:21 2018
HOST: hostname                         Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                      0.0%    10   25.5  28.7   5.4  39.6  11.1
  2.|-- 147.75.36.4                                  0.0%    10   14.6 239.4  14.6 1098. 383.6
  3.|-- 0.ae22.bbr2.ewr1.packet.net                  0.0%    10    0.3   0.7   0.2   4.9   1.3
  4.|-- 10ge3-9.core1.nyc6.he.net                    0.0%    10    1.2   1.2   1.2   1.3   0.0
  5.|-- 100ge13-1.core1.nyc4.he.net                  0.0%    10    1.4   2.6   1.3   6.3   1.6
  6.|-- 100ge8-1.core1.sjc2.he.net                   0.0%    10   63.0  63.2  63.0  64.1   0.0
  7.|-- 10ge4-4.core1.sjc1.he.net                    0.0%    10   73.6  65.9  63.1  73.6   4.3
  8.|-- vocus.gigabitethernet2-13.core1.sjc1.he.net  0.0%    10   68.5  68.6  68.4  69.5   0.0
  9.|-- bundle-153.cor02.sjc01.ca.vocus.net          0.0%    10  218.2 218.3 218.1 218.8   0.0
 10.|-- BE-201.cor02.syd04.nsw.VOCUS.net.au          0.0%    10  218.1 218.1 218.0 218.3   0.0
 11.|-- bundle-101.bdr01.syd03.nsw.vocus.net.au      0.0%    10  218.9 218.7 218.5 219.0   0.0
 12.|-- static-73.7.255.49.in-addr.VOCUS.net.au      0.0%    10  217.9 217.9 217.8 218.0   0.0
 13.|-- xe-1-0-1.gw102.sy3.ap.equinix.com            0.0%    10  213.7 214.0 213.6 216.1   0.7
 14.|-- 180.189.25.6                                 0.0%    10  231.4 231.5 231.4 231.7   0.0
 15.|-- vl901-ds1-b5-r1120.aus1.choopa.net           0.0%    10  232.0 232.1 232.0 232.2   0.0
 16.|-- ???                                         100.0    10    0.0   0.0   0.0   0.0   0.0
 17.|-- 108.61.212.117.vultr.com                     0.0%    10  222.8 222.9 222.7 223.4   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 hnd-jp-ping.vultr.com
Start: Mon Feb 26 08:55:42 2018
HOST: hostname                        Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                     0.0%    10   15.8  16.9  12.7  22.1   2.8
  2.|-- 147.75.36.4                                 0.0%    10   17.6 188.9  13.5 1013. 327.5
  3.|-- 0.ae22.bbr2.ewr1.packet.net                 0.0%    10    0.2   0.2   0.2   0.4   0.0
  4.|-- 0.xe-0-0-0.csr1.ewr2.packet.net             0.0%    10  123.0  37.3   1.5 134.1  53.3
  6.|-- 0.ae1.BR2.NYC4.ALTER.NET                    0.0%    10    2.7   2.9   2.7   3.5   0.0
  7.|-- verizon.com.customer.alter.net              0.0%    10    3.0   3.0   2.9   3.3   0.0
  8.|-- ae-3.r24.nycmny01.us.bb.gin.ntt.net        10.0%    10    3.0   3.0   3.0   3.2   0.0
  9.|-- ae-2.r20.sttlwa01.us.bb.gin.ntt.net        10.0%    10   70.3  70.5  70.2  71.9   0.4
 10.|-- ae-13.r30.tokyjp05.jp.bb.gin.ntt.net       30.0%    10  173.8 173.7 173.7 173.8   0.0
 11.|-- ae-2.r02.tokyjp05.jp.bb.gin.ntt.net         0.0%    10  181.7 181.7 181.6 181.8   0.0
 12.|-- ce-0-14-0-1.r02.tokyjp05.jp.ce.gin.ntt.net  0.0%    10  200.9 188.6 182.5 200.9   7.0
 13.|-- ???                                        100.0    10    0.0   0.0   0.0   0.0   0.0
 14.|-- vl516-ds1-b5-r2605.tyo1.choopa.net          0.0%    10  182.4 182.3 182.1 183.8   0.5
 15.|-- ???                                        100.0    10    0.0   0.0   0.0   0.0   0.0
 16.|-- 108.61.201.151.vultr.com                    0.0%    10  176.1 175.8 175.6 176.1   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 lax-ca-us-ping.vultr.com
Start: Mon Feb 26 08:56:00 2018
HOST: hostname                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10   13.0  14.7   8.0  18.0   3.0
  2.|-- 147.75.36.2                         0.0%    10  428.8  84.8  11.4 428.8 138.3
  3.|-- 0.ae11.bbr1.ewr1.packet.net         0.0%    10    0.2   0.2   0.2   0.2   0.0
  4.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
  5.|-- nyk-b6-link.telia.net               0.0%    10    1.0   1.0   1.0   1.0   0.0
  6.|-- nyk-bb3-link.telia.net              0.0%    10    1.0   1.1   1.0   1.3   0.0
  7.|-- las-b24-link.telia.net              0.0%    10   90.9  92.2  87.9 105.3   5.9
  8.|-- las-b4-link.telia.net               0.0%    10   87.7  91.9  87.4 127.3  12.5
  9.|-- vultr-ic-318444-las-b4.c.telia.net  0.0%    10   87.5  89.2  87.5  96.0   2.8
 10.|-- vl901-ds1-b5-c705.lax3.choopa.net   0.0%    10   88.5 103.1  88.4 219.6  41.1
 11.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
 12.|-- 108.61.219.200.vultr.com            0.0%    10   87.6  87.5  87.4  87.6   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 wa-us-ping.vultr.com
Start: Mon Feb 26 08:56:16 2018
HOST: hostname                  Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                               0.0%    10   15.6  21.2   6.4  67.2  16.7
  2.|-- 147.75.36.2                           0.0%    10  563.1 129.8  12.0 563.1 202.4
  3.|-- 0.ae11.bbr1.ewr1.packet.net           0.0%    10    0.2   0.2   0.1   0.2   0.0
  4.|-- 0.xe-0-0-1.csr1.ewr2.packet.net       0.0%    10    4.8  17.8   3.1 132.3  40.2
  7.|-- lag-10.ear4.Newark1.Level3.net        0.0%    10    1.1   1.1   1.0   1.6   0.0
  8.|-- ae-2-3613.edge1.Seattle3.Level3.net   0.0%    10   74.7  74.7  74.7  74.7   0.0
  9.|-- CHOOPA-LLC.edge1.Seattle3.Level3.net  0.0%    10   71.5  72.0  71.5  74.9   0.9
 10.|-- ???                                  100.0    10    0.0   0.0   0.0   0.0   0.0
 11.|-- 108.61.194.105.vultr.com              0.0%    10   71.5  71.5  71.5  71.5   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 tx-us-ping.vultr.com
Start: Mon Feb 26 08:56:32 2018
HOST: hostname                 Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                              0.0%    10   21.5  15.9  11.9  21.5   2.7
  2.|-- 147.75.36.2                          0.0%    10    9.1 207.2   9.1 1037. 396.7
  3.|-- 0.ae21.bbr2.ewr1.packet.net          0.0%    10    0.2   0.2   0.1   0.2   0.0
  4.|-- 0.ae0.bbr1.ewr1.packet.net           0.0%    10    0.4   1.0   0.4   6.6   1.8
  5.|-- ???                                 100.0    10    0.0   0.0   0.0   0.0   0.0
  6.|-- nyk-b6-link.telia.net                0.0%    10    1.0   1.1   1.0   1.7   0.0
  7.|-- nyk-bb3-link.telia.net              10.0%    10   38.9  42.5  38.9  65.5   8.7
  8.|-- ash-bb3-link.telia.net               0.0%    10   36.3  36.4  36.3  36.5   0.0
  9.|-- atl-b22-link.telia.net               0.0%    10   20.4  22.8  18.6  33.0   4.2
 10.|-- dls-b22-link.telia.net               0.0%    10   44.2  40.0  38.7  45.5   2.6
 11.|-- vultr-ic-318447-dls-b22.c.telia.net  0.0%    10   39.0  39.4  39.0  42.9   1.1
 12.|-- ???                                 100.0    10    0.0   0.0   0.0   0.0   0.0
 13.|-- ???                                 100.0    10    0.0   0.0   0.0   0.0   0.0
 14.|-- 108.61.224.175.vultr.com             0.0%    10   39.2  39.2  39.1  39.3   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 il-us-ping.vultr.com
Start: Mon Feb 26 08:56:48 2018
HOST: hostname                 Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                              0.0%    10   16.2  13.5  11.0  16.2   1.9
  2.|-- 147.75.36.4                          0.0%    10   16.0 280.8  11.6 1006. 423.6
  3.|-- 0.ae12.bbr1.ewr1.packet.net          0.0%    10    0.2   0.2   0.2   0.6   0.0
  4.|-- ???                                 100.0    10    0.0   0.0   0.0   0.0   0.0
  5.|-- nyk-b6-link.telia.net                0.0%    10    1.0   1.0   1.0   1.1   0.0
  6.|-- nyk-bb4-link.telia.net               0.0%    10   23.6  23.8  23.6  24.0   0.0
  7.|-- chi-b21-link.telia.net               0.0%    10   24.6  24.7  24.6  24.8   0.0
  8.|-- vultr-ic-318445-chi-b21.c.telia.net  0.0%    10   26.2  27.9  26.2  30.4   1.3
  9.|-- vl820-ds1-b5-gj178.chi2.choopa.net   0.0%    10   25.0  25.2  24.9  27.4   0.6
 10.|-- ???                                 100.0    10    0.0   0.0   0.0   0.0   0.0
 11.|-- 107.191.51.12.vultr.com              0.0%    10   24.9  25.0  24.9  25.0   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 ga-us-ping.vultr.com
Start: Mon Feb 26 08:57:04 2018
HOST: hostname                 Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                              0.0%    10   14.8  15.8  11.2  21.2   3.7
  2.|-- 147.75.36.2                          0.0%    10   18.4 190.1   9.1 934.8 368.1
  3.|-- 0.ae21.bbr2.ewr1.packet.net          0.0%    10    0.2   1.1   0.2   9.9   3.1
  4.|-- 0.ae0.bbr1.ewr1.packet.net           0.0%    10    0.4   0.5   0.4   0.9   0.0
  5.|-- ???                                 100.0    10    0.0   0.0   0.0   0.0   0.0
  6.|-- nyk-b6-link.telia.net                0.0%    10    1.0   1.1   1.0   2.0   0.0
  7.|-- nyk-bb3-link.telia.net               0.0%    10   18.8  20.0  18.6  26.7   2.4
  8.|-- ash-bb3-link.telia.net               0.0%    10    7.6   8.7   7.4  16.9   2.9
  9.|-- atl-b21-link.telia.net               0.0%    10   18.5  18.5  18.4  18.6   0.0
 10.|-- vultr-ic-319646-atl-b21.c.telia.net  0.0%    10   18.4  18.4  18.4  18.9   0.0
 11.|-- vl901-ds1-b5-r7.atl2.choopa.net      0.0%    10   26.8  19.7  18.9  26.8   2.5
 12.|-- ???                                 100.0    10    0.0   0.0   0.0   0.0   0.0
 13.|-- 108.61.193.166.vultr.com             0.0%    10   18.6  18.6  18.6  18.7   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 fl-us-ping.vultr.com
Start: Mon Feb 26 08:57:20 2018
HOST: hostname                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10   48.3  34.5  25.0  48.3   7.1
  2.|-- 147.75.36.2                         0.0%    10   15.2  86.5  11.4 494.7 160.4
  3.|-- 0.ae21.bbr2.ewr1.packet.net         0.0%    10    0.2   0.2   0.2   0.2   0.0
  4.|-- 0.ae0.bbr1.ewr1.packet.net          0.0%    10    0.4   0.8   0.4   2.5   0.6
  5.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
  6.|-- nyk-b6-link.telia.net               0.0%    10    1.0   1.0   1.0   1.1   0.0
  7.|-- nyk-bb3-link.telia.net              0.0%    10    1.2   1.3   1.1   2.6   0.3
  8.|-- ash-bb3-link.telia.net              0.0%    10    7.3   7.3   7.3   7.4   0.0
  9.|-- mai-b1-link.telia.net               0.0%    10   34.2  34.4  34.1  35.5   0.0
 10.|-- vultr-ic-318945-mai-b1.c.telia.net  0.0%    10   36.0  34.8  34.5  36.0   0.3
 11.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
 12.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
 13.|-- 104.156.244.232                     0.0%    10   34.1  34.1  34.0  34.1   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 nj-us-ping.vultr.com
Start: Mon Feb 26 08:57:36 2018
HOST: hostname                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10   21.3  17.8  11.3  38.1   8.1
  2.|-- 147.75.36.4                         0.0%    10   20.7 174.3  12.4 871.9 334.2
  3.|-- 0.ae12.bbr1.ewr1.packet.net         0.0%    10    0.2   1.7   0.2  15.3   4.7
  4.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
  5.|-- nyk-b6-link.telia.net               0.0%    10    1.0   1.2   1.0   2.8   0.5
  6.|-- nyk-bb3-link.telia.net              0.0%    10   19.5  13.9   1.2  32.2  12.3
  7.|-- nyk-b2-link.telia.net               0.0%    10    1.7   5.1   1.7  34.7  10.4
  8.|-- vultr-ic-318452-nyk-b2.c.telia.net  0.0%    10    7.6   5.9   2.0  11.6   4.1
  9.|-- vl39-br1.pnj1.choopa.net            0.0%    10    8.3   5.2   2.1  13.1   4.1
 10.|-- vl139-c4-3-b5-2.pnj1.choopa.net     0.0%    10    2.6   2.4   2.4   2.6   0.0
 11.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
 12.|-- 108.61.149.182.vultr.com            0.0%    10    2.1   2.1   2.0   2.2   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 fra-de-ping.vultr.com
Start: Mon Feb 26 08:57:52 2018
HOST: hostname                 Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                              0.0%    10   12.4  21.3  12.4  65.8  15.9
  2.|-- 147.75.36.2                          0.0%    10   20.9 241.4  11.4 1023. 376.2
  3.|-- 0.ae21.bbr2.ewr1.packet.net          0.0%    10    0.5   0.2   0.2   0.5   0.0
  4.|-- packet.tbr2.wbr.nac.net              0.0%    10    0.4  12.0   0.3  85.3  27.0
  5.|-- 0.lag16.tbr1.wbr.nac.net             0.0%    10    6.6   6.1   0.2  12.8   4.8
  6.|-- 0.e3-4.tbr1.tl9.nac.net              0.0%    10    1.1   1.4   1.1   4.1   0.7
  7.|-- nyk-b3-link.telia.net                0.0%    10    1.2   1.2   1.2   1.2   0.0
  8.|-- nyk-bb4-link.telia.net               0.0%    10    1.8   1.9   1.8   2.2   0.0
  9.|-- prs-bb4-link.telia.net               0.0%    10   88.1  89.3  88.1  94.4   1.9
 10.|-- prs-b2-link.telia.net                0.0%    10   72.4  72.6  72.4  73.4   0.0
 11.|-- prs-bb3-link.telia.net               0.0%    10   81.7  81.8  81.6  82.8   0.0
 12.|-- ffm-bb3-link.telia.net               0.0%    10   87.9  87.9  87.8  88.2   0.0
 13.|-- ffm-b12-link.telia.net               0.0%    10   85.3  85.3  85.3  85.4   0.0
 14.|-- vultr-ic-312911-ffm-b12.c.telia.net  0.0%    10   81.7  82.1  81.6  84.0   0.6
 15.|-- ???                                 100.0    10    0.0   0.0   0.0   0.0   0.0
 16.|-- 108.61.210.117.vultr.com             0.0%    10   81.6  81.6  81.6  81.7   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 ams-nl-ping.vultr.com
Start: Mon Feb 26 08:58:09 2018
HOST: hostname                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10  103.4  56.0  20.0 103.4  33.1
  2.|-- 147.75.36.4                         0.0%    10   14.9 227.2  11.5 1122. 421.2
  3.|-- 0.ae22.bbr2.ewr1.packet.net         0.0%    10    0.2   0.2   0.1   0.3   0.0
  4.|-- packet.tbr2.wbr.nac.net             0.0%    10    0.4  17.5   0.3 170.9  53.9
  5.|-- 0.lag16.tbr1.wbr.nac.net            0.0%    10    0.2   2.6   0.2  11.4   4.1
  6.|-- 0.e3-4.tbr1.tl9.nac.net             0.0%    10    1.1   1.5   1.1   4.6   0.8
  7.|-- nyk-b3-link.telia.net               0.0%    10    1.4   3.3   1.2  20.8   6.1
  8.|-- nyk-bb3-link.telia.net              0.0%    10    1.3   1.7   1.1   6.3   1.5
  9.|-- ldn-bb3-link.telia.net              0.0%    10   71.0  72.8  71.0  82.1   3.5
 10.|-- adm-bb3-link.telia.net              0.0%    10   79.1  79.2  79.0  80.3   0.3
 11.|-- adm-b3-link.telia.net               0.0%    10   79.4  81.2  79.3  95.7   5.1
 12.|-- vultr-ic-313751-adm-b3.c.telia.net  0.0%    10   79.5  79.5  79.4  79.6   0.0
 13.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
 14.|-- 108.61.198.102.vultr.com            0.0%    10   79.5  79.6  79.4  79.7   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 lon-gb-ping.vultr.com
Start: Mon Feb 26 08:58:25 2018
HOST: hostname                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10   16.5  31.2  16.5  40.9   7.0
  2.|-- 147.75.36.2                         0.0%    10   13.9  94.2  12.6 383.6 134.4
  3.|-- 0.ae21.bbr2.ewr1.packet.net         0.0%    10    0.2   0.3   0.2   1.3   0.0
  4.|-- packet.tbr2.wbr.nac.net             0.0%    10    0.4   6.4   0.3  60.0  18.8
  5.|-- 0.lag16.tbr1.wbr.nac.net            0.0%    10    2.4   3.7   0.2   9.1   3.2
  6.|-- 0.e3-4.tbr1.tl9.nac.net             0.0%    10    1.1   2.1   1.1   5.2   1.3
  7.|-- nyk-b3-link.telia.net               0.0%    10    1.2   1.2   1.2   1.3   0.0
  8.|-- nyk-bb3-link.telia.net              0.0%    10    1.4   1.3   1.2   1.4   0.0
  9.|-- ldn-bb3-link.telia.net              0.0%    10   71.1  71.3  71.0  73.2   0.6
 10.|-- ldn-b9-link.telia.net               0.0%    10   71.0  71.0  70.9  71.3   0.0
 11.|-- vultr-ic-312913-ldn-b9.c.telia.net  0.0%    10   72.2  72.4  71.7  75.1   0.9
 12.|-- 10.69.1.90                         20.0%    10   78.5  77.8  72.1 107.5  12.2
 13.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
 14.|-- 108.61.196.101.vultr.com            0.0%    10   71.2  71.2  71.2  71.3   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 par-fr-ping.vultr.com
Start: Mon Feb 26 08:58:42 2018
HOST: hostname                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10   24.0  15.6  11.9  24.0   3.3
  2.|-- 147.75.36.2                         0.0%    10   21.3 258.8  13.4 847.0 375.6
  3.|-- 0.ae21.bbr2.ewr1.packet.net         0.0%    10   26.6   2.8   0.2  26.6   8.3
  4.|-- packet.tbr2.wbr.nac.net             0.0%    10    0.4   0.5   0.3   1.1   0.0
  5.|-- 0.lag16.tbr1.wbr.nac.net            0.0%    10    0.3   0.2   0.2   0.3   0.0
  6.|-- 0.e3-4.tbr1.tl9.nac.net             0.0%    10   10.2   4.5   1.2  11.5   4.2
  7.|-- nyk-b3-link.telia.net               0.0%    10    1.2   1.3   1.2   1.8   0.0
  8.|-- nyk-bb4-link.telia.net              0.0%    10    1.8   1.9   1.8   2.1   0.0
  9.|-- prs-bb4-link.telia.net              0.0%    10   72.3  74.2  72.1  82.2   3.6
 10.|-- prs-b7-link.telia.net               0.0%    10   75.6  73.2  72.2  75.6   1.4
 11.|-- vultr-ic-312910-prs-b7.c.telia.net  0.0%    10   85.5  85.7  85.4  86.4   0.0
 12.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
 13.|-- 108.61.209.127.vultr.com            0.0%    10   85.5  85.4  85.4  85.5   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 VersaWeb Las Vegas
Start: Mon Feb 26 08:58:57 2018
HOST: hostname                    Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                 0.0%    10   14.9  16.2  12.5  22.2   3.3
  2.|-- 147.75.36.2                             0.0%    10  986.3 252.3  10.3 986.3 391.4
  3.|-- 0.ae11.bbr1.ewr1.packet.net             0.0%    10    0.2   4.8   0.2  45.6  14.3
  4.|-- 10gigabitethernet1-2.core1.nyc6.he.net  0.0%    10    1.5   3.3   1.4   8.3   2.7
  5.|-- 100ge13-1.core1.nyc4.he.net             0.0%    10   12.2   3.2   1.4  12.2   3.5
  6.|-- 100ge16-1.core1.ash1.he.net             0.0%    10    6.5   6.8   6.4   9.2   0.7
  7.|-- 100ge14-2.core1.mci3.he.net             0.0%    10   30.6  32.6  30.5  41.6   3.7
  8.|-- 100ge12-1.core1.den1.he.net             0.0%    10   52.6  45.2  42.6  55.8   4.8
  9.|-- 100ge10-2.core1.slc1.he.net             0.0%    10   55.9  56.6  54.1  61.5   2.6
 10.|-- 10ge1-1.core1.las1.he.net               0.0%    10   70.3  66.6  62.9  73.3   4.6
 11.|-- 172.18.0.22                             0.0%    10   76.3  77.8  76.2  84.4   3.2
 12.|-- 192.228.109.121                         0.0%    10   76.3  76.6  76.2  79.1   0.7
 13.|-- 192.228.109.86                          0.0%    10   76.4  86.3  76.4 172.4  30.2
 14.|-- lg.las1.ultravps.net                    0.0%    10   76.2  76.2  76.2  76.3   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 VersaWeb Seattle
Start: Mon Feb 26 08:59:15 2018
HOST: hostname                       Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                    0.0%    10   15.2  14.7   4.9  21.8   4.8
  2.|-- 147.75.36.4                                0.0%    10   22.2 218.7  11.8 1067. 412.9
  3.|-- 0.ae22.bbr2.ewr1.packet.net                0.0%    10    0.2   0.5   0.2   3.2   0.9
  4.|-- packet.tbr2.wbr.nac.net                    0.0%    10    0.4   0.4   0.3   0.5   0.0
  5.|-- 0.lag16.tbr1.wbr.nac.net                   0.0%    10    8.1   4.8   0.2  12.6   5.1
  6.|-- 0.e1-3.tbr1.ewr.nac.net                    0.0%    10    1.0   2.9   0.9   7.7   2.2
  7.|-- te0-0-0-24.rcr21.ewr01.atlas.cogentco.com  0.0%    10    1.6   1.5   1.2   2.6   0.3
  8.|-- be3200.rcr24.jfk01.atlas.cogentco.com      0.0%    10    1.7   1.7   1.7   1.8   0.0
  9.|-- be2897.ccr42.jfk02.atlas.cogentco.com      0.0%    10    1.8   1.8   1.5   2.0   0.0
 10.|-- be2890.ccr22.cle04.atlas.cogentco.com      0.0%    10   18.6  18.4  18.2  18.7   0.0
 11.|-- be2718.ccr42.ord01.atlas.cogentco.com      0.0%    10   20.1  19.9  19.7  20.1   0.0
 12.|-- be2832.ccr22.mci01.atlas.cogentco.com      0.0%    10   48.1  48.1  47.9  48.3   0.0
 13.|-- be3036.ccr22.den01.atlas.cogentco.com      0.0%    10   59.8  59.7  59.4  59.9   0.0
 14.|-- be3038.ccr21.slc01.atlas.cogentco.com      0.0%    10   72.4  72.5  72.1  72.7   0.0
 15.|-- be2085.ccr21.sea02.atlas.cogentco.com      0.0%    10   66.9  66.8  66.8  67.0   0.0
 16.|-- 38.142.48.250                              0.0%    10   66.4  66.4  66.4  66.5   0.0
 17.|-- 76.164.234.1                               0.0%    10   66.5  66.5  66.5  66.8   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 OVH Canada
Start: Mon Feb 26 08:59:34 2018
HOST: hostname    Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                 0.0%    10    1.4   4.3   1.4  11.1   2.7
  2.|-- 2604:1380:2:79::        0.0%    10   10.1   5.4   0.7  10.1   3.7
  3.|-- 2604:1380:2:20::        0.0%    10    0.2   0.2   0.2   0.2   0.0
  4.|-- de-cix.nyc.ovh.com      0.0%    10    1.8   2.0   1.8   2.3   0.0
  5.|-- 2607:5300::1c6          0.0%    10   10.8  16.0   9.8  68.4  18.4
  6.|-- 2607:5300::1c3          0.0%    10    8.8   8.9   8.8   9.0   0.0
  7.|-- 2001:41d0:0:50::6:835   0.0%    10    9.0   8.9   8.9   9.0   0.0
  8.|-- 2001:41d0:0:50::2:404d  0.0%    10    9.4  10.0   9.3  12.2   0.7
  9.|-- 2607:5300:60:44a5::1    0.0%    10    9.0   8.9   8.9   9.0   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io Frankfurt:
Start: Mon Feb 26 08:59:50 2018
HOST: hostname              Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                           0.0%    10   20.8  27.6  11.2  49.1  10.0
  2.|-- 147.75.36.2                       0.0%    10   12.1  53.2  11.4 266.6  83.1
  3.|-- 0.ae21.bbr2.ewr1.packet.net       0.0%    10    6.4   0.8   0.1   6.4   1.9
  4.|-- ae4-1.nyk10.core-backbone.com     0.0%    10    1.1   1.1   1.1   1.3   0.0
  5.|-- ae2-2025.fra30.core-backbone.com  0.0%    10   81.0  80.9  80.8  81.1   0.0
  6.|-- core-backbone.cdn77.com           0.0%    10   93.2  93.2  93.2  93.3   0.0
  7.|-- frankfurt-20.cdn77.com            0.0%    10   93.2  93.2  93.1  93.3   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io London:
Start: Mon Feb 26 09:00:09 2018
HOST: hostname               Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                            0.0%    10   12.4  15.7  11.9  23.8   3.7
  2.|-- 147.75.36.4                        0.0%    10  149.7 192.4  11.3 990.1 346.4
  3.|-- 0.ae12.bbr1.ewr1.packet.net        0.0%    10    0.2   2.3   0.2  21.2   6.6
  4.|-- 0.ae0.bbr2.ewr1.packet.net         0.0%    10    0.5   0.4   0.2   1.5   0.3
  5.|-- ae4-1.nyk10.core-backbone.com      0.0%    10    1.1   1.5   1.1   5.1   1.2
  6.|-- ae10-2092.lon10.core-backbone.com  0.0%    10   70.3  70.4  70.3  70.6   0.0
  7.|-- core-backbone.cdn77.com            0.0%    10   93.8  93.8  93.7  93.8   0.0
  8.|-- london-20.cdn77.com                0.0%    10   94.6  94.6  94.5  94.7   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io Paris:
Start: Mon Feb 26 09:00:29 2018
HOST: hostname                 Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                              0.0%    10   19.0  16.2   8.6  21.9   3.8
  2.|-- 147.75.36.2                          0.0%    10  845.2 195.9  11.4 845.2 311.7
  3.|-- 0.ae21.bbr2.ewr1.packet.net          0.0%    10    0.2   1.6   0.2  13.8   4.3
  4.|-- 0.xe-0-0-0.csr1.ewr2.packet.net      0.0%    10    3.4  12.0   3.4  62.3  17.8
  6.|-- 0.et-10-0-2.BR2.NYC4.ALTER.NET       0.0%    10    3.2   3.9   2.5  10.7   2.6
  7.|-- verizon.com.customer.alter.net       0.0%    10    2.8   2.9   2.7   3.1   0.0
  8.|-- ae-3.r24.nycmny01.us.bb.gin.ntt.net 10.0%    10    2.8   3.5   2.8   8.6   1.8
  9.|-- ae-9.r24.londen12.uk.bb.gin.ntt.net  0.0%    10   74.6  74.7  74.6  74.9   0.0
 10.|-- ae-1.r25.londen12.uk.bb.gin.ntt.net  0.0%    10   75.0  75.1  74.9  76.1   0.0
 11.|-- ae-2.r04.parsfr01.fr.bb.gin.ntt.net  0.0%    10   92.3  92.4  92.3  92.7   0.0
 12.|-- ae-5.r03.parsfr02.fr.bb.gin.ntt.net  0.0%    10   92.6  92.3  92.2  92.6   0.0
 13.|-- ae-8.r02.parsfr02.fr.bb.gin.ntt.net  0.0%    10   79.6  79.6  79.4  79.8   0.0
 14.|-- 81.25.206.50                         0.0%    10  101.0 101.1 101.0 101.3   0.0
 15.|-- unn-185-93-2-193.10gbps.io           0.0%    10  101.2 101.2 101.1 101.5   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io Singapore:
Start: Mon Feb 26 09:00:47 2018
HOST: hostname                       Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                    0.0%    10   13.0  26.6  12.7 112.0  30.1
  2.|-- 147.75.36.2                                0.0%    10   19.8 154.7  12.2 1016. 314.2
  3.|-- 0.ae21.bbr2.ewr1.packet.net                0.0%    10    0.2   2.9   0.2  26.4   8.2
  4.|-- packet.tbr2.wbr.nac.net                    0.0%    10    0.4   0.9   0.3   5.1   1.3
  5.|-- 0.lag16.tbr1.wbr.nac.net                   0.0%    10    0.2   1.0   0.2   4.5   1.4
  6.|-- 0.e1-3.tbr1.ewr.nac.net                    0.0%    10   11.5   2.6   0.9  11.5   3.5
  7.|-- te0-0-0-24.rcr21.ewr01.atlas.cogentco.com  0.0%    10    2.5   1.4   1.1   2.5   0.3
  8.|-- be3199.rcr23.jfk01.atlas.cogentco.com      0.0%    10    1.8   1.9   1.7   2.0   0.0
  9.|-- be2896.ccr41.jfk02.atlas.cogentco.com      0.0%    10    1.6   1.7   1.5   1.9   0.0
 10.|-- be2806.ccr41.dca01.atlas.cogentco.com      0.0%    10    7.7   7.8   7.5   8.0   0.0
 11.|-- be2112.ccr41.atl01.atlas.cogentco.com      0.0%    10   19.8  19.8  19.6  20.0   0.0
 12.|-- be2687.ccr41.iah01.atlas.cogentco.com      0.0%    10   45.5  45.4  45.2  45.6   0.0
 13.|-- be2927.ccr21.elp01.atlas.cogentco.com      0.0%    10   73.3  73.5  73.3  73.7   0.0
 14.|-- be2929.ccr31.phx01.atlas.cogentco.com      0.0%    10   73.7  73.6  73.3  73.8   0.0
 15.|-- be2931.ccr41.lax01.atlas.cogentco.com      0.0%    10   73.7  73.6  73.3  73.8   0.0
 16.|-- be3271.ccr41.lax04.atlas.cogentco.com      0.0%    10   75.2  75.3  75.1  75.6   0.0
 17.|-- be3406.ccr21.sin01.atlas.cogentco.com      0.0%    10  246.7 246.9 246.7 247.9   0.3
 18.|-- 154.18.2.2                                 0.0%    10  245.0 245.0 245.0 245.2   0.0
 19.|-- hol14.homeoflim.com                        0.0%    10  251.5 251.5 251.4 251.7   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io Stockholm:
Start: Mon Feb 26 09:01:04 2018
HOST: hostname              Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                           0.0%    10  117.1  31.1  14.0 117.1  32.0
  2.|-- 147.75.36.4                       0.0%    10   15.2  88.7  12.8 699.4 214.8
  3.|-- 0.ae12.bbr1.ewr1.packet.net       0.0%    10    0.3   0.6   0.2   3.1   0.7
  4.|-- 0.ae0.bbr2.ewr1.packet.net        0.0%    10    0.2   1.3   0.2  11.6   3.5
  5.|-- ae4-1.nyk10.core-backbone.com     0.0%    10    1.1   1.7   1.1   6.6   1.6
  6.|-- ae1-2081.sth10.core-backbone.com  0.0%    10  101.4 101.5 101.3 103.2   0.3
  7.|-- core-backbone.cdn77.com           0.0%    10  101.0 101.0 101.0 101.1   0.0
  8.|-- stockholm-10.cdn77.com            0.0%    10  108.2 108.2 108.2 108.4   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io Moscow:
Start: Mon Feb 26 09:01:22 2018
HOST: hostname         Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                      0.0%    10   16.1  15.6  13.4  19.9   2.4
  2.|-- 147.75.36.2                  0.0%    10   19.9 190.8  14.3 1136. 361.8
  3.|-- 0.ae21.bbr2.ewr1.packet.net  0.0%    10    0.2   0.2   0.2   0.2   0.0
  4.|-- de-cix.nyc.retn.net          0.0%    10    1.1   1.1   1.0   1.1   0.0
  5.|-- ae1-3.RT.MR.MSK.RU.retn.net  0.0%    10  114.7 114.5 114.5 114.7   0.0
  6.|-- 87.245.204.20                0.0%    10  113.6 113.5 113.5 113.6   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io Miami:
Start: Mon Feb 26 09:01:37 2018
HOST: hostname                 Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                              0.0%    10   13.1  14.4   8.4  19.1   3.0
  2.|-- 147.75.36.4                          0.0%    10   17.8 173.9  10.4 903.0 332.7
  3.|-- 0.ae22.bbr2.ewr1.packet.net          0.0%    10    0.3   3.7   0.2  34.5  10.8
  4.|-- 0.xe-0-0-0.csr1.ewr2.packet.net      0.0%    10    2.9   6.1   1.9  10.4   3.4
  6.|-- 0.et-10-0-2.BR2.NYC4.ALTER.NET       0.0%    10    2.7   3.1   2.6   6.7   1.2
  7.|-- verizon.com.customer.alter.net       0.0%    10    2.9   2.9   2.7   3.1   0.0
  8.|-- ae-3.r25.nycmny01.us.bb.gin.ntt.net  0.0%    10    2.9   3.7   2.8   9.7   2.1
  9.|-- ae-9.r22.asbnva02.us.bb.gin.ntt.net  0.0%    10    8.4   8.4   8.3   8.6   0.0
 10.|-- ae-0.r23.asbnva02.us.bb.gin.ntt.net  0.0%    10   10.4   9.0   8.4  10.4   0.5
 11.|-- ae-1.r20.miamfl02.us.bb.gin.ntt.net  0.0%    10   65.8  45.9  38.6  65.8   9.4
 12.|-- ae-1.r04.miamfl02.us.bb.gin.ntt.net  0.0%    10   38.6  38.6  38.5  38.9   0.0
 13.|-- 165.254.21.26                        0.0%    10   37.8  37.8  37.8  37.9   0.0
 14.|-- unn-195-181-162-195.cdn77.com        0.0%    10   55.7  55.8  55.7  56.0   0.0

    Code (Text):
    -------------------------------------------
Check system entropy pool availability
-------------------------------------------

entropy_avail: 4067
entropy_avail: 4067
entropy_avail: 4067
entropy_avail: 4067

    Code (Text):
    -------------------------------------------
OpenSSL System Benchmark
-------------------------------------------

OpenSSL 1.0.2k-fips  26 Jan 2017
-------------------------------------------
openssl speed -multi 48 rsa4096 rsa2048 ecdsap256 sha256 sha1 md5 rc4 aes-256-cbc aes-128-cbc
808.00:111310165.33 from 17
OpenSSL 1.0.2k-fips  26 Jan 2017
built on: reproducible build, date unspecified
options:bn(64,64) md2(int) rc4(8x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
md5            1711832.20k  5308997.99k 12297699.75k 18615076.52k 21547302.91k
sha1           2643534.69k  9116932.97k 24249715.46k 41606732.12k 51272840.53k
rc4           14108966.49k 17533703.91k 18091231.69k 18160244.39k 18219786.24k
aes-128 cbc    2889747.50k  3094950.14k  3190846.89k  7211115.63k  7326624.43k
aes-256 cbc    2107697.37k  2221035.03k  2260734.21k  5312491.86k  5346129.24k
sha256         5275014.02k 14868248.87k 33555368.62k 50065332.57k 58719780.86k
                  sign    verify    sign/s verify/s
rsa 2048 bits 0.000042s 0.000001s  23753.6 778961.7
rsa 4096 bits 0.000301s 0.000005s   3321.7 210777.5
                              sign    verify    sign/s verify/s
 256 bit ecdsa (nistp256)   0.0000s   0.0000s 521739.1 223621.8
-------------------------------------------
openssl speed -evp aes256 -multi 48
OpenSSL 1.0.2k-fips  26 Jan 2017
built on: reproducible build, date unspecified
options:bn(64,64) md2(int) rc4(8x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
evp           22921436.58k 28635047.76k 30956237.91k 31749383.51k 32001492.83k
-------------------------------------------
openssl speed -evp aes128 -multi 48
OpenSSL 1.0.2k-fips  26 Jan 2017
built on: reproducible build, date unspecified
options:bn(64,64) md2(int) rc4(8x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
evp           27692549.66k 38683828.80k 41936650.41k 43406140.76k 43849318.40k

    Code (Text):
    -------------------------------------------
Running mysqlslap
-------------------------------------------

mysqlslap --auto-generate-sql --auto-generate-sql-add-autoincrement --auto-generate-sql-secondary-indexes=5 --number-int-cols=5 --number-char-cols=5 --number-of-queries=25000 --auto-generate-sql-unique-query-number=40 --auto-generate-sql-unique-write-number=40 --auto-generate-sql-write-number=1000 --concurrency=64 --iterations=10 --engine=myisam
Benchmark
        Running for engine myisam
        Average number of seconds to run all queries: 1.459 seconds
        Minimum number of seconds to run all queries: 1.405 seconds
        Maximum number of seconds to run all queries: 1.509 seconds
        Number of clients running queries: 64
        Average number of queries per client: 390

Dropping the database is potentially a very bad thing to do.
Any data stored in the database will be destroyed.

Do you really want to drop the 'test' database [y/N] Database "test" dropped

    Code (Text):
    -------------------------------------------
System PHP Info
-------------------------------------------

CPU: 48 x AMD EPYC 7401P 24-Core Processor
              total        used        free      shared  buff/cache   available
Mem:          64159        3101       60371          30         686       60633
Low:          64159        3788       60371
High:             0           0           0
Swap:          1949           0        1949
----------------------------------------------
PHP 5.6.33 (cli) (built: Feb 26 2018 08:24:24)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
    with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies
----------------------------------------------
Configuration File (php.ini) Path: /usr/local/lib
Loaded Configuration File:         /usr/local/lib/php.ini
Scan for additional .ini files in: /etc/centminmod/php.d
Additional .ini files parsed:      /etc/centminmod/php.d/a_customphp.ini,
/etc/centminmod/php.d/curlcainfo.ini,
/etc/centminmod/php.d/geoip.ini,
/etc/centminmod/php.d/igbinary.ini,
/etc/centminmod/php.d/imagick.ini,
/etc/centminmod/php.d/mailparse.ini,
/etc/centminmod/php.d/memcache.ini,
/etc/centminmod/php.d/memcached.ini,
/etc/centminmod/php.d/redis.ini,
/etc/centminmod/php.d/zendopcache.ini

----------------------------------------------
[PHP Modules]
bcmath
bz2
calendar
Core
ctype
curl
date
dom
enchant
ereg
exif
filter
ftp
gd
geoip
gettext
gmp
hash
iconv
igbinary
imagick
imap
intl
json
ldap
libxml
mailparse
mbstring
mcrypt
memcache
memcached
mhash
mysql
mysqli
mysqlnd
openssl
pcntl
pcre
PDO
pdo_mysql
pdo_sqlite
Phar
posix
pspell
readline
redis
Reflection
session
shmop
SimpleXML
snmp
soap
sockets
SPL
sqlite3
standard
sysvmsg
sysvsem
sysvshm
tidy
tokenizer
xml
xmlreader
xmlrpc
xmlwriter
xsl
Zend OPcache
zip
zlib

[Zend Modules]
Zend OPcache

----------------------------------------------
-------------------------------------------
Run PHP test Zend/bench.php
-------------------------------------------

simple             0.127
simplecall         0.104
simpleucall        0.110
simpleudcall       0.115
mandel             0.261
mandel2            0.312
ackermann(7)       0.116
ary(50000)         0.028
ary2(50000)        0.027
ary3(2000)         0.132
fibo(30)           0.387
hash1(50000)       0.035
hash2(500)         0.026
heapsort(20000)    0.080
matrix(20)         0.161
nestedloop(12)     0.214
sieve(30)          0.081
strcat(200000)     0.012
------------------------
Total              2.326
real: 2.43s user: 2.23s sys: 0.19s cpu: 99% maxmem: 31304 KB cswaits: 8

simple             0.127
simplecall         0.104
simpleucall        0.109
simpleudcall       0.114
mandel             0.315
mandel2            0.306
ackermann(7)       0.116
ary(50000)         0.028
ary2(50000)        0.027
ary3(2000)         0.133
fibo(30)           0.383
hash1(50000)       0.034
hash2(500)         0.026
heapsort(20000)    0.080
matrix(20)         0.165
nestedloop(12)     0.214
sieve(30)          0.080
strcat(200000)     0.012
------------------------
Total              2.374
real: 2.48s user: 2.28s sys: 0.20s cpu: 99% maxmem: 31300 KB cswaits: 3

simple             0.125
simplecall         0.104
simpleucall        0.109
simpleudcall       0.114
mandel             0.259
mandel2            0.312
ackermann(7)       0.116
ary(50000)         0.028
ary2(50000)        0.027
ary3(2000)         0.132
fibo(30)           0.380
hash1(50000)       0.034
hash2(500)         0.026
heapsort(20000)    0.079
matrix(20)         0.168
nestedloop(12)     0.213
sieve(30)          0.080
strcat(200000)     0.012
------------------------
Total              2.317
real: 2.42s user: 2.24s sys: 0.18s cpu: 99% maxmem: 31300 KB cswaits: 3

bench.php results from 3 runs
2.326
2.374
2.317

bench.php avg: 2.3390
Avg: real: 2.44s user: 2.25s sys: 0.19s cpu: 99.00% maxmem: 31301.33KB cswaits: 4.67
created results log at /home/phpbench_logs/bench_260218-084535.log
server PHP info log at /home/phpbench_logs/bench_phpinfo_260218-084535.log

-------------------------------------------
Run PHP test Zend/micro_bench.php
-------------------------------------------

empty_loop         0.152
func()             0.335    0.183
undef_func()       0.340    0.188
int_func()         0.283    0.132
$x = self::$x      0.229    0.077
self::$x = 0       0.241    0.089
isset(self::$x)    0.239    0.087
empty(self::$x)    0.257    0.105
$x = Foo::$x       0.220    0.068
Foo::$x = 0        0.228    0.076
isset(Foo::$x)     0.216    0.064
empty(Foo::$x)     0.227    0.075
self::f()          0.374    0.222
Foo::f()           0.305    0.153
$x = $this->x      0.217    0.065
$this->x = 0       0.279    0.128
$this->x += 2      0.232    0.080
++$this->x         0.226    0.074
--$this->x         0.233    0.081
$this->x++         0.254    0.102
$this->x--         0.266    0.114
isset($this->x)    0.340    0.188
empty($this->x)    0.246    0.094
$this->f()         0.416    0.264
$x = Foo::TEST     0.231    0.079
new Foo()          0.643    0.491
$x = TEST          0.199    0.047
$x = $_GET         0.211    0.059
$x = $GLOBALS['v'] 0.286    0.134
$x = $hash['v']    0.244    0.092
$x = $str[0]       0.319    0.167
$x = $a ?: null    0.226    0.074
$x = $f ?: tmp     0.304    0.152
$x = $f ? $f : $a  0.231    0.079
$x = $f ? $f : tmp 0.305    0.153
------------------------
Total              9.553
real: 9.66s user: 9.57s sys: 0.08s cpu: 99% maxmem: 19396 KB cswaits: 4

empty_loop         0.152
func()             0.330    0.178
undef_func()       0.352    0.200
int_func()         0.302    0.150
$x = self::$x      0.258    0.106
self::$x = 0       0.264    0.112
isset(self::$x)    0.232    0.080
empty(self::$x)    0.258    0.106
$x = Foo::$x       0.228    0.076
Foo::$x = 0        0.228    0.076
isset(Foo::$x)     0.216    0.064
empty(Foo::$x)     0.227    0.075
self::f()          0.372    0.220
Foo::f()           0.325    0.173
$x = $this->x      0.229    0.078
$this->x = 0       0.281    0.129
$this->x += 2      0.231    0.079
++$this->x         0.235    0.083
--$this->x         0.239    0.087
$this->x++         0.255    0.103
$this->x--         0.273    0.121
isset($this->x)    0.325    0.173
empty($this->x)    0.237    0.086
$this->f()         0.399    0.248
$x = Foo::TEST     0.243    0.091
new Foo()          0.674    0.522
$x = TEST          0.187    0.035
$x = $_GET         0.207    0.055
$x = $GLOBALS['v'] 0.286    0.134
$x = $hash['v']    0.242    0.090
$x = $str[0]       0.322    0.170
$x = $a ?: null    0.227    0.075
$x = $f ?: tmp     0.303    0.151
$x = $f ? $f : $a  0.226    0.074
$x = $f ? $f : tmp 0.304    0.152
------------------------
Total              9.672
real: 9.77s user: 9.70s sys: 0.07s cpu: 99% maxmem: 19400 KB cswaits: 3

empty_loop         0.152
func()             0.334    0.182
undef_func()       0.357    0.204
int_func()         0.303    0.151
$x = self::$x      0.236    0.083
self::$x = 0       0.238    0.085
isset(self::$x)    0.234    0.082
empty(self::$x)    0.234    0.082
$x = Foo::$x       0.224    0.072
Foo::$x = 0        0.228    0.076
isset(Foo::$x)     0.216    0.064
empty(Foo::$x)     0.227    0.075
self::f()          0.365    0.213
Foo::f()           0.327    0.175
$x = $this->x      0.228    0.076
$this->x = 0       0.289    0.137
$this->x += 2      0.235    0.083
++$this->x         0.232    0.080
--$this->x         0.236    0.084
$this->x++         0.241    0.089
$this->x--         0.257    0.105
isset($this->x)    0.326    0.174
empty($this->x)    0.247    0.095
$this->f()         0.416    0.264
$x = Foo::TEST     0.252    0.100
new Foo()          0.686    0.534
$x = TEST          0.197    0.045
$x = $_GET         0.207    0.055
$x = $GLOBALS['v'] 0.286    0.134
$x = $hash['v']    0.241    0.089
$x = $str[0]       0.322    0.170
$x = $a ?: null    0.221    0.068
$x = $f ?: tmp     0.335    0.182
$x = $f ? $f : $a  0.237    0.085
$x = $f ? $f : tmp 0.325    0.173
------------------------
Total              9.692
real: 9.79s user: 9.72s sys: 0.07s cpu: 99% maxmem: 19396 KB cswaits: 3

micro_bench.php results from 3 runs
9.553
9.672
9.692

micro_bench.php avg: 9.6390
Avg: real: 9.74s user: 9.66s sys: 0.07s cpu: 99.00% maxmem: 19397.33KB cswaits: 3.33
created results log at /home/phpbench_logs/bench_micro_260218-084535.log
server PHP info log at /home/phpbench_logs/bench_phpinfo_260218-084535.log

    Code (Text):
    ========================================================================
   BYTE UNIX Benchmarks (Version 5.1.3)

   System: hostname: GNU/Linux
   OS: GNU/Linux -- 3.10.0-693.17.1.el7.x86_64 -- #1 SMP Thu Jan 25 20:13:58 UTC 2018
   Machine: x86_64 (x86_64)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 1: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 2: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 3: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 4: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 5: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 6: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 7: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 8: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 9: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 10: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 11: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 12: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 13: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 14: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 15: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 16: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 17: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 18: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 19: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 20: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 21: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 22: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 23: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 24: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 25: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 26: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 27: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 28: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 29: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 30: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 31: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 32: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 33: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 34: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 35: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 36: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 37: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 38: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 39: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 40: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 41: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 42: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 43: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 44: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 45: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 46: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 47: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   09:06:52 up  1:06,  2 users,  load average: 13.41, 17.36, 8.27; runlevel 3

    Code (Text):
    ------------------------------------------------------------------------
Benchmark Run: Mon Feb 26 2018 09:06:52 - 09:31:02
48 CPUs in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       34980535.2 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4338.1 MWIPS (11.0 s, 7 samples)
Execl Throughput                               1095.8 lps   (30.0 s, 2 samples)
Pipe Throughput                              128455.4 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                  62619.1 lps   (10.0 s, 7 samples)
Process Creation                               2893.3 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   2991.4 lpm   (60.0 s, 2 samples)
Shell Scripts (16 concurrent)                   893.4 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   1419.4 lpm   (60.0 s, 2 samples)
System Call Overhead                          77396.8 lps   (10.0 s, 7 samples)

System Benchmarks Partial Index              BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   34980535.2   2997.5
Double-Precision Whetstone                       55.0       4338.1    788.7
Execl Throughput                                 43.0       1095.8    254.8
Pipe Throughput                               12440.0     128455.4    103.3
Pipe-based Context Switching                   4000.0      62619.1    156.5
Process Creation                                126.0       2893.3    229.6
Shell Scripts (1 concurrent)                     42.4       2991.4    705.5
Shell Scripts (16 concurrent)                     ---        893.4      ---
Shell Scripts (8 concurrent)                      6.0       1419.4   2365.7
System Call Overhead                          15000.0      77396.8     51.6
                                                                   ========
System Benchmarks Index Score (Partial Only)                          386.5

    Code (Text):
    ------------------------------------------------------------------------
Benchmark Run: Mon Feb 26 2018 09:31:02 - 09:55:12
48 CPUs in system; running 48 parallel copies of tests

Dhrystone 2 using register variables      880376483.7 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                   165961.3 MWIPS (10.0 s, 7 samples)
Execl Throughput                              20934.6 lps   (29.9 s, 2 samples)
Pipe Throughput                             3310161.1 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                1285764.1 lps   (10.0 s, 7 samples)
Process Creation                              61193.6 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                  49131.7 lpm   (60.0 s, 2 samples)
Shell Scripts (16 concurrent)                  3363.2 lpm   (60.3 s, 2 samples)
Shell Scripts (8 concurrent)                   6743.5 lpm   (60.1 s, 2 samples)
System Call Overhead                        1830436.5 lps   (10.0 s, 7 samples)

System Benchmarks Partial Index              BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0  880376483.7  75439.3
Double-Precision Whetstone                       55.0     165961.3  30174.8
Execl Throughput                                 43.0      20934.6   4868.5
Pipe Throughput                               12440.0    3310161.1   2660.9
Pipe-based Context Switching                   4000.0    1285764.1   3214.4
Process Creation                                126.0      61193.6   4856.6
Shell Scripts (1 concurrent)                     42.4      49131.7  11587.7
Shell Scripts (16 concurrent)                     ---       3363.2      ---
Shell Scripts (8 concurrent)                      6.0       6743.5  11239.2
System Call Overhead                          15000.0    1830436.5   1220.3
                                                                   ========
System Benchmarks Index Score (Partial Only)                         7478.4

    Code (Text):
    -------------------------------------------
centminmodbench.sh completed
-------------------------------------------

centminmodbench.sh Total Run Time: 4161.857673157 seconds


    GCC 8.0.1 Compile & RPM Build Times



    Centmin Mod LEMP Stack's Nginx web server supports multiple GCC/Clang version compilers from GCC 4.8, 5.3, 6.3, 7.2 and 8.x and Clang 3.4, 4, 5, and 6. Currently testing custom GCC 8.0.1 compilations and RPM builds so let's see how the Packet.net bare metal server holds up.

    It took a total of ~2684 seconds or ~44.7 minutes to compile and build both binutils 2.30 + GCC 8.0.1 with Profile Guided Optimisations and build the custom CentOS 7 RPMs. Compared to above Intel Xeon E3-1578Lv5 at ~2716 seconds or ~45 minutes.

    Was expecting 48 cpu threads to be much faster, so not sure what's up. Maybe it's related to CentOS 7.4's 3.10 Linux Kernel and slower single 120GB SSD boot drive ?

    Code (Text):
    RPMs Built
/svr-setup/binutils-gcc8-2.30-1.el7.x86_64.rpm
/svr-setup/gcc8-pgo-8.0-1.el7.x86_64.rpm

ls -lah /svr-setup | egrep 'gcc8-pgo-8.0|binutils-gcc8-2.30'
-rw-r--r--  1 root      root       5.2M Feb 26 10:03 binutils-gcc8-2.30-1.el7.x86_64.rpm
-rw-r--r--  1 root      root        39M Feb 26 10:44 gcc8-pgo-8.0-1.el7.x86_64.rpm

Installed Packages
Name        : binutils-gcc8
Arch        : x86_64
Version     : 2.30
Release     : 1.el7
Size        : 48 M
Repo        : installed
Summary     : binutils-gcc8 for centminmod.com LEMP stack installs
URL         : https://centminmod.com
License     : unknown
Description : binutils-gcc8 for centminmod.com LEMP stacks


* Mon Feb 26 2018 George Liu <centminmod.com> 2.30
- Binutils 2.30 for centminmod.com LEMP stack installs

    Code (Text):
    Installed Packages
Name        : gcc8-pgo
Arch        : x86_64
Version     : 8.0
Release     : 1.el7
Size        : 287 M
Repo        : installed
Summary     : gcc8-pgo for centminmod.com LEMP stack installs
URL         : https://centminmod.com
License     : unknown
Description : gcc8-pgo for centminmod.com LEMP stacks


* Mon Feb 26 2018 George Liu <centminmod.com> 8
- GCC 8 for centminmod.com LEMP stack installs

*************************************************
Compile GCC Completed
log: /root/centminlogs/tools-gcc-install-pgo_260218-095942.log
*************************************************

Total Binutils + GCC Install Time: 2684.645239389 seconds


    Other AMD EPYC 7401P Reviews



    Folks may also want to check out other AMD EPYC 7401P Reviews
     
    Last edited: Feb 27, 2018
  5. Feb 26, 2018 #5
    eva2000

    eva2000 Administrator Staff Member

    53,280
    12,117
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,655
    Local Time:
    2:23 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    Packet.net AMD EPYC 7401P With 4.15.5 Linux Kernel



    To see if CentOS 7.4's default 3.10 Linux Kernel was a factor, I updated to latest ELrepo 4.15.5-1 Kernel-ml mainline Linux Kernel package and rebooted the AMD EPYC 7401P server.
    Code (Text):
    uname -r
4.15.5-1.el7.elrepo.x86_64


    Unfortunately, there were kernel related cpu lockups being reported
    Code (Text):
    48 x Dhrystone 2 using register variables  1 2 3 4 5 6 7 8 9 10
48 x Double-Precision Whetstone  1 2 3 4 5 6 7 8 9 10
48 x System Call Overhead  1 2 3 4 5 6 7 8 9 10
48 x Pipe Throughput  1 2 3 4 5 6 7 8 9 10
48 x Pipe-based Context Switching  1 2 3 4 5

Message from syslogd@epyc at Feb 26 12:39:47 ...
 kernel:watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [sshd:72234]

Message from syslogd@epyc at Feb 26 12:39:51 ...
 kernel:watchdog: BUG: soft lockup - CPU#21 stuck for 22s! [context1:72109]

Message from syslogd@epyc at Feb 26 12:40:03 ...
 kernel:watchdog: BUG: soft lockup - CPU#8 stuck for 22s! [kworker/8:0:4089]

Message from syslogd@epyc at Feb 26 12:40:15 ...
 kernel:watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [sshd:72234]

Message from syslogd@epyc at Feb 26 12:40:19 ...
 kernel:watchdog: BUG: soft lockup - CPU#21 stuck for 22s! [context1:72109]

Message from syslogd@epyc at Feb 26 12:40:31 ...
 kernel:watchdog: BUG: soft lockup - CPU#8 stuck for 22s! [kworker/8:0:4089]

    Then SSH session terminated itself. So using Serial Over SSH feature tried logging into the server and was met with a stream full of messages that bond0 NIC was down

    Code (Text):
     status down for interface enp1s0f0, disabling it in 200 ms
** 3 printk messages dropped **
[ 4470.051815] bond0: link status down for interface enp1s0f0, disabling it in 200 ms
** 4 printk messages dropped **
[ 4470.063819] bond0: link status down for interface enp1s0f0, disabling it in 200 ms
** 4 printk messages dropped **
[ 4470.075830] bond0: link status down for interface enp1s0f0, disabling it in 200 ms
** 3 printk messages dropped **
[ 4470.084888] bond0: link status down for interface enp1s0f0, disabling it in 200 ms
** 4 printk messages dropped **
[ 4470.097820] bond0: link status down for interface enp1s0f0, disabling it in 200 ms
** 3 printk messages dropped **
[ 4470.105818] bond0: link status down for interface enp1s0f0, disabling it in 200 ms
** 4 printk messages dropped **


    Bit of digging led me to soft lockup info at:
    On AMD EPYC 7401P there are 48 such watchdog threads
    Code (Text):
    ps uax | grep watchdog
root         12  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/0]
root         15  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/1]
root         22  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/2]
root         29  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/3]
root         36  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/4]
root         42  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/5]
root         48  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/6]
root         54  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/7]
root         60  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/8]
root         66  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/9]
root         72  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/10]
root         78  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/11]
root         84  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/12]
root         90  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/13]
root         96  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/14]
root        102  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/15]
root        108  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/16]
root        114  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/17]
root        120  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/18]
root        126  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/19]
root        132  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/20]
root        138  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/21]
root        144  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/22]
root        150  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/23]
root        156  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/24]
root        162  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/25]
root        168  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/26]
root        174  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/27]
root        180  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/28]
root        186  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/29]
root        192  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/30]
root        198  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/31]
root        204  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/32]
root        210  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/33]
root        216  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/34]
root        222  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/35]
root        228  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/36]
root        234  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/37]
root        240  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/38]
root        246  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/39]
root        252  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/40]
root        258  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/41]
root        264  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/42]
root        270  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/43]
root        276  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/44]
root        282  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/45]
root        288  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/46]
root        294  0.0  0.0      0     0 ?        S    13:01   0:00 [watchdog/47]
root        320  0.0  0.0      0     0 ?        I<   13:01   0:00 [watchdogd]

    Default watchdog_thresh value is 10 so 2x10 = 20 seconds time limit
    Code (Text):
    sysctl -a 2>&1 | egrep 'watchdog|softlockup'
kernel.nmi_watchdog = 1
kernel.soft_watchdog = 1
kernel.softlockup_all_cpu_backtrace = 0
kernel.softlockup_panic = 0
kernel.watchdog = 1
kernel.watchdog_cpumask = 0-127
kernel.watchdog_thresh = 10

    So looks like UnixBench test just pushed the softlock threshold above the 2x10 = 20 seconds, holding onto the CPU for 22 seconds
    Code (Text):
    Message from syslogd@epyc at Feb 26 12:39:47 ...
 kernel:watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [sshd:72234]

Message from syslogd@epyc at Feb 26 12:39:51 ...
 kernel:watchdog: BUG: soft lockup - CPU#21 stuck for 22s! [context1:72109]

    Seeing as it's stuck with sshd it could be related to running UnixBench within a screen session and the 48x CPU section of UnixBench test ?

    So maybe adjusting kernel.watchdog_thresh from 10 to 20 would be enough ?
    Code (Text):
    echo "kernel.watchdog_thresh = 20" >> /etc/sysctl.d/101-sysctl.conf
sysctl --system

    Code (Text):
    sysctl -a 2>&1 | egrep 'watchdog|softlockup'                 
kernel.nmi_watchdog = 1
kernel.soft_watchdog = 1
kernel.softlockup_all_cpu_backtrace = 0
kernel.softlockup_panic = 0
kernel.watchdog = 1
kernel.watchdog_cpumask = 0-127
kernel.watchdog_thresh = 20


    Looks like that might have done the job and allowed UnixBench to complete. Below are results from re-run of centminmodbench.sh
    • The beast of the AMD EPYC 7401P processor was definitely let down by the slow 120GB SSD boot disk which showed no improvement
    • Memory bandwidth from ramdisk tests were also better now on the lower side of numbers
    • Network bandwidth was improved for wget single threaded tests were at ~104-307MB/s compared to ~50MB/s previously for US East Coast. The Axel multi-threaded bandwidth tests were a bit better for US East Coast ranging from 128-295MB/s compared to previously at 113-168MB/s
    • UnixBench was much better for single index = 1444.8 and 48 cpus index at 17,557.2 compared to the previously very low for single index = 386.5 and 48 cpus index at 7478.
    • OpenSSL numbers as expected were also slightly improved with AMD EPYC 7401P hardware AES-NI assisted aes256 results being ~4.6x times faster vs ~4.57x times faster on CentOS 7 3.10 Kernel and ecdsa sign/s being ~4.67x times faster vs ~4.4x times faster on CentOS 3.10 Kernel than above Intel Xeon E3-1578Lv5 Skylake.
    Code (Text):
    -------------------------------------------
centminmodbench.sh 0.9.5
http://bench.centminmod.com
written by: George Liu (eva2000)
http://centminmod.com
-------------------------------------------

-------------------------------------------
System Information
-------------------------------------------

4.15.5-1.el7.elrepo.x86_64

CentOS Linux release 7.4.1708 (Core)

Centmin Mod
----------------------------------------------
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                48
On-line CPU(s) list:   0-47
Thread(s) per core:    2
Core(s) per socket:    24
Socket(s):             1
NUMA node(s):          4
Vendor ID:             AuthenticAMD
CPU family:            23
Model:                 1
Model name:            AMD EPYC 7401P 24-Core Processor
Stepping:              2
CPU MHz:               2419.247
BogoMIPS:              3992.40
Virtualization:        AMD-V
L1d cache:             32K
L1i cache:             64K
L2 cache:              512K
L3 cache:              8192K
NUMA node0 CPU(s):     0,4,8,12,16,20,24,28,32,36,40,44
NUMA node1 CPU(s):     1,5,9,13,17,21,25,29,33,37,41,45
NUMA node2 CPU(s):     2,6,10,14,18,22,26,30,34,38,42,46
NUMA node3 CPU(s):     3,7,11,15,19,23,27,31,35,39,43,47
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid amd_dcm aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb hw_pstate sme vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 xsaves clzero irperf xsaveerptr ibpb arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif overflow_recov succor smca

CPU Flags
 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid amd_dcm aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb hw_pstate sme vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 xsaves clzero irperf xsaveerptr ibpb arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif overflow_recov succor smca

----------------------------------------------
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE
0   0    0      0    0:0:0:0       yes
1   1    0      1    1:1:1:1       yes
2   2    0      2    2:2:2:2       yes
3   3    0      3    3:3:3:3       yes
4   0    0      4    4:4:4:4       yes
5   1    0      5    5:5:5:5       yes
6   2    0      6    6:6:6:6       yes
7   3    0      7    7:7:7:7       yes
8   0    0      8    8:8:8:0       yes
9   1    0      9    9:9:9:1       yes
10  2    0      10   10:10:10:2    yes
11  3    0      11   11:11:11:3    yes
12  0    0      12   12:12:12:4    yes
13  1    0      13   13:13:13:5    yes
14  2    0      14   14:14:14:6    yes
15  3    0      15   15:15:15:7    yes
16  0    0      16   16:16:16:0    yes
17  1    0      17   17:17:17:1    yes
18  2    0      18   18:18:18:2    yes
19  3    0      19   19:19:19:3    yes
20  0    0      20   20:20:20:4    yes
21  1    0      21   21:21:21:5    yes
22  2    0      22   22:22:22:6    yes
23  3    0      23   23:23:23:7    yes
24  0    0      0    0:0:0:0       yes
25  1    0      1    1:1:1:1       yes
26  2    0      2    2:2:2:2       yes
27  3    0      3    3:3:3:3       yes
28  0    0      4    4:4:4:4       yes
29  1    0      5    5:5:5:5       yes
30  2    0      6    6:6:6:6       yes
31  3    0      7    7:7:7:7       yes
32  0    0      8    8:8:8:0       yes
33  1    0      9    9:9:9:1       yes
34  2    0      10   10:10:10:2    yes
35  3    0      11   11:11:11:3    yes
36  0    0      12   12:12:12:4    yes
37  1    0      13   13:13:13:5    yes
38  2    0      14   14:14:14:6    yes
39  3    0      15   15:15:15:7    yes
40  0    0      16   16:16:16:0    yes
41  1    0      17   17:17:17:1    yes
42  2    0      18   18:18:18:2    yes
43  3    0      19   19:19:19:3    yes
44  0    0      20   20:20:20:4    yes
45  1    0      21   21:21:21:5    yes
46  2    0      22   22:22:22:6    yes
47  3    0      23   23:23:23:7    yes

----------------------------------------------
              total        used        free      shared  buff/cache   available
Mem:          64146        3591       59847          14         707       59881
Low:          64146        4298       59847
High:             0           0           0
Swap:          1949           0        1949

----------------------------------------------
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs         32G     0   32G   0% /dev
tmpfs            32G     0   32G   0% /dev/shm
tmpfs            32G  852K   32G   1% /run
tmpfs            32G     0   32G   0% /sys/fs/cgroup
/dev/sda3       108G   17G   86G  17% /
/dev/sda1       511M  148K  511M   1% /boot/efi
tmpfs           6.3G     0  6.3G   0% /run/user/0

    Code (Text):
    -------------------------------------------
disk ioping tests
-------------------------------------------


Running IOPing I/O benchmark...

IOPing I/O: ./ioping -c 10 .
4.0 KiB from . (ext4 /dev/sda3): request=1 time=3.1 ms
4.0 KiB from . (ext4 /dev/sda3): request=2 time=232 us
4.0 KiB from . (ext4 /dev/sda3): request=3 time=265 us
4.0 KiB from . (ext4 /dev/sda3): request=4 time=229 us
4.0 KiB from . (ext4 /dev/sda3): request=5 time=269 us
4.0 KiB from . (ext4 /dev/sda3): request=6 time=267 us
4.0 KiB from . (ext4 /dev/sda3): request=7 time=272 us
4.0 KiB from . (ext4 /dev/sda3): request=8 time=266 us
4.0 KiB from . (ext4 /dev/sda3): request=9 time=267 us
4.0 KiB from . (ext4 /dev/sda3): request=10 time=268 us

--- . (ext4 /dev/sda3) ioping statistics ---
10 requests completed in 9.0 s, 1.8 k iops, 7.2 MiB/s
min/avg/max/mdev = 229 us / 542 us / 3.1 ms / 847 us

IOPing seek rate: ./ioping -RD .

--- . (ext4 /dev/sda3) ioping statistics ---
18.4 k requests completed in 3.0 s, 6.3 k iops, 24.7 MiB/s
min/avg/max/mdev = 137 us / 157 us / 1.7 ms / 23 us

IOPing sequential: ./ioping -RL .

--- . (ext4 /dev/sda3) ioping statistics ---
1.6 k requests completed in 3.0 s, 526 iops, 131.6 MiB/s
min/avg/max/mdev = 1.2 ms / 1.9 ms / 5.2 ms / 396 us

IOPing cached: ./ioping -RC .

--- . (ext4 /dev/sda3) ioping statistics ---
3.4 M requests completed in 3.0 s, 1.2 M iops, 4.6 GiB/s
min/avg/max/mdev = 0 us / 0 us / 53 us / 0 us

    Code (Text):
    -------------------------------------------
disk DD tests
-------------------------------------------

dd if=/dev/zero of=sb-io-test bs=1M count=1k conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 5.98212 s, 179 MB/s

dd if=/dev/zero of=sb-io-test bs=64k count=16k conv=fdatasync
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 5.9581 s, 180 MB/s

dd if=/dev/zero of=sb-io-test bs=1M count=1k oflag=dsync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 5.6338 s, 191 MB/s

dd if=/dev/zero of=sb-io-test bs=64k count=16k oflag=dsync
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 11.0268 s, 97.4 MB/s

    Code (Text):
    -------------------------------------------
disk FIO tests
-------------------------------------------


Running FIO benchmark...


FIO random reads:
randomreads: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.0.9
Starting 1 process
randomreads: Laying out IO file(s) (1 file(s) / 1024MB)

randomreads: (groupid=0, jobs=1): err= 0: pid=3702: Mon Feb 26 13:35:37 2018
  read : io=1024.3MB, bw=172306KB/s, iops=43076 , runt=  6087msec
  cpu          : usr=1.96%, sys=10.53%, ctx=23835, majf=0, minf=71
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=262207/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=1024.3MB, aggrb=172306KB/s, minb=172306KB/s, maxb=172306KB/s, mint=6087msec, maxt=6087msec

Disk stats (read/write):
  sda: ios=257601/6, merge=52/5, ticks=374401/7, in_queue=374419, util=98.39%

FIO random writes:
randomwrites: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.0.9
Starting 1 process

randomwrites: (groupid=0, jobs=1): err= 0: pid=3707: Mon Feb 26 13:35:44 2018
  write: io=1024.3MB, bw=147059KB/s, iops=36764 , runt=  7132msec
  cpu          : usr=2.03%, sys=10.03%, ctx=23834, majf=0, minf=6
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=0/w=262207/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=1024.3MB, aggrb=147059KB/s, minb=147059KB/s, maxb=147059KB/s, mint=7132msec, maxt=7132msec

Disk stats (read/write):
  sda: ios=0/253731, merge=0/75, ticks=0/432621, in_queue=432660, util=98.63%

    Code (Text):
    -------------------------------------------
ramdisk DD tests - memory bandwidth
-------------------------------------------

dd if=/dev/zero of=sb-io-test bs=128k count=1k conv=fdatasync
1024+0 records in
1024+0 records out
134217728 bytes (134 MB) copied, 0.0389545 s, 3.4 GB/s

dd if=/dev/zero of=sb-io-test bs=8k count=16k conv=fdatasync
16384+0 records in
16384+0 records out
134217728 bytes (134 MB) copied, 0.0472288 s, 2.8 GB/s

dd if=/dev/zero of=sb-io-test bs=128k count=1k oflag=dsync
1024+0 records in
1024+0 records out
134217728 bytes (134 MB) copied, 0.0365683 s, 3.7 GB/s

dd if=/dev/zero of=sb-io-test bs=8k count=16k oflag=dsync
16384+0 records in
16384+0 records out
134217728 bytes (134 MB) copied, 0.0436984 s, 3.1 GB/s

    Code (Text):
    -------------------------------------------
ramdisk ioping tests - memory bandwidth
-------------------------------------------

Running IOPing I/O ramdisk benchmark...

IOPing I/O: ./ioping -c 10 .
4.0 KiB from . (tmpfs tmpfs): request=1 time=2 us
4.0 KiB from . (tmpfs tmpfs): request=2 time=3 us
4.0 KiB from . (tmpfs tmpfs): request=3 time=2 us
4.0 KiB from . (tmpfs tmpfs): request=4 time=2 us
4.0 KiB from . (tmpfs tmpfs): request=5 time=2 us
4.0 KiB from . (tmpfs tmpfs): request=6 time=3 us
4.0 KiB from . (tmpfs tmpfs): request=7 time=2 us
4.0 KiB from . (tmpfs tmpfs): request=8 time=2 us
4.0 KiB from . (tmpfs tmpfs): request=9 time=2 us
4.0 KiB from . (tmpfs tmpfs): request=10 time=2 us

--- . (tmpfs tmpfs) ioping statistics ---
10 requests completed in 9.0 s, 454.5 k iops, 1.7 GiB/s
min/avg/max/mdev = 2 us / 2 us / 3 us / 0 us

IOPing seek rate: ./ioping -R .

--- . (tmpfs tmpfs) ioping statistics ---
3.6 M requests completed in 3.0 s, 1.5 M iops, 5.8 GiB/s
min/avg/max/mdev = 0 us / 0 us / 35 us / 0 us

IOPing sequential: ./ioping -RL .

--- . (tmpfs tmpfs) ioping statistics ---
128.5 k requests completed in 3.0 s, 43.2 k iops, 10.5 GiB/s
min/avg/max/mdev = 21 us / 23 us / 61 us / 1 us

IOPing cached: ./ioping -RC .

--- . (tmpfs tmpfs) ioping statistics ---
4.2 M requests completed in 3.0 s, 1.5 M iops, 5.8 GiB/s
min/avg/max/mdev = 0 us / 0 us / 20 us / 0 us

    Code (Text):
    
-------------------------------------------
Running bandwidth benchmark...
-------------------------------------------

----------------------------------------------
Download from Cachefly (http://cachefly.cachefly.net/100mb.test)
Download Cachefly: 152MB/s

-------------------------------------------
USA bandwidth tests...
-------------------------------------------

----------------------------------------------
Download from Linode, Atlanta, GA, USA (http://speedtest.atlanta.linode.com/100MB-atlanta.bin)
Download Linode, Atlanta, GA, USA: 104MB/s
----------------------------------------------
Download from Linode, Dallas, TX, USA (http://speedtest.dallas.linode.com/100MB-dallas.bin)
Download Linode, Dallas, TX, USA: 10.2MB/s
----------------------------------------------
Download from Leaseweb, Manassas, VA, USA (http://mirror.us.leaseweb.net/speedtest/100mb.bin)
Download Leaseweb, Manassas, VA, USA: 53.2MB/s
----------------------------------------------
Download from Softlayer, Seattle, WA, USA (http://speedtest.sea01.softlayer.com/downloads/test100.zip)
Download Softlayer, Seattle, WA, USA: 38.3MB/s
----------------------------------------------
Download from Softlayer, San Jose, CA, USA (http://speedtest.sjc01.softlayer.com/downloads/test100.zip)
Download Softlayer, San Jose, CA, USA: 7.68MB/s
----------------------------------------------
Download from Softlayer, Washington, DC, USA (http://speedtest.wdc01.softlayer.com/downloads/test100.zip)
Download Softlayer, Washington, DC, USA: 18.8MB/s
----------------------------------------------
Download from VersaWeb, Las Vegas, Nevada (http://104.143.15.254/100MB.test)
Download VersaWeb, Las Vegas, Nevada: 5.42MB/s
----------------------------------------------
Download from OVH, BHS, Canada (http://bhs.proof.ovh.net/files/100Mio.dat)
Download OVH, BHS, Canada: 115MB/s
----------------------------------------------
Download from Vultr, Los Angeles, California (http://lax-ca-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Los Angeles, California: 25.9MB/s
----------------------------------------------
Download from Vultr, Seattle, Washington (http://wa-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Seattle, Washington: 31.7MB/s
----------------------------------------------
Download from Vultr, Dallas, Texas (http://tx-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Dallas, Texas: 64.7MB/s
----------------------------------------------
Download from Vultr, Chicago, Illinois (http://il-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Chicago, Illinois: 97.9MB/s
----------------------------------------------
Download from Vultr, Atlanta, Georgia (http://ga-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Atlanta, Georgia: 119MB/s
----------------------------------------------
Download from Vultr, Miami, Florida (http://fl-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Miami, Florida: 68.7MB/s
----------------------------------------------
Download from Vultr, New York / New Jersey (http://nj-us-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, New York / New Jersey: 307MB/s

-------------------------------------------
Asia bandwidth tests...
-------------------------------------------

----------------------------------------------
Download from Linode, Tokyo, JP (http://speedtest.tokyo.linode.com/100MB-tokyo.bin)
Download Linode, Tokyo, JP: 13.2MB/s
----------------------------------------------
Download from Softlayer, Singapore (http://speedtest.sng01.softlayer.com/downloads/test100.zip)
Download Softlayer, Singapore: 8.56MB/s
----------------------------------------------
Download from Vultr, Tokyo, Japan (http://hnd-jp-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Tokyo, Japan: 13.3MB/s

-------------------------------------------
Europe bandwidth tests...
-------------------------------------------

----------------------------------------------
Download from Linode, London, UK (http://speedtest.london.linode.com/100MB-london.bin)
Download Linode, London, UK: 16.7MB/s
----------------------------------------------
Download from OVH, Paris, France (http://proof.ovh.net/files/100Mio.dat)
Download OVH, Paris, France: 9.16MB/s
----------------------------------------------
Download from SmartDC, Rotterdam, Netherlands (http://mirror.i3d.net/100mb.bin)
Download SmartDC, Rotterdam, Netherlands: 14.8MB/s
----------------------------------------------
Download from Vultr, Amsterdam, Netherlands (http://ams-nl-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Amsterdam, Netherlands: 29.3MB/s
----------------------------------------------
Download from Vultr, London, UK (http://lon-gb-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, London, UK: 32.3MB/s
----------------------------------------------
Download from Vultr, Paris, France (http://par-fr-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Paris, France: 31.6MB/s

-------------------------------------------
Australia bandwidth tests...
-------------------------------------------

----------------------------------------------
Download from Vultr, Sydney, Australia (http://syd-au-ping.vultr.com/vultr.com.100MB.bin)
Download Vultr, Sydney, Australia: 9.75MB/s

    Code (Text):
    
-------------------------------------------
Running Axel multi-threaded bandwidth benchmark...
-------------------------------------------

----------------------------------------------
Axel Download from Cachefly (http://cachefly.cachefly.net/100mb.test)
Axel Download Cachefly: 286.82MB/s

-------------------------------------------
USA bandwidth tests...
-------------------------------------------

----------------------------------------------
Axel Download from Linode, Atlanta, GA, USA (http://speedtest.atlanta.linode.com/100MB-atlanta.bin)
Axel Download Linode, Atlanta, GA, USA: 128.62MB/s
----------------------------------------------
Axel Download from Linode, Dallas, TX, USA (http://speedtest.dallas.linode.com/100MB-dallas.bin)
Axel Download Linode, Dallas, TX, USA: 75.97MB/s
----------------------------------------------
Axel Download from Leaseweb, Manassas, VA, USA (http://mirror.us.leaseweb.net/speedtest/100mb.bin)
Axel Download Leaseweb, Manassas, VA, USA: 56.68MB/s
----------------------------------------------
Axel Download from Softlayer, Seattle, WA, USA (http://speedtest.sea01.softlayer.com/downloads/test100.zip)
Axel Download Softlayer, Seattle, WA, USA: 33.03MB/s
----------------------------------------------
Axel Download from Softlayer, San Jose, CA, USA (http://speedtest.sjc01.softlayer.com/downloads/test100.zip)
Axel Download Softlayer, San Jose, CA, USA: 25.45MB/s
----------------------------------------------
Axel Download from Softlayer, Washington, DC, USA (http://speedtest.wdc01.softlayer.com/downloads/test100.zip)
Axel Download Softlayer, Washington, DC, USA: 64.84MB/s
----------------------------------------------
Axel Download from VersaWeb, Las Vegas, Nevada (http://104.143.15.254/100MB.test)
Axel Download VersaWeb, Las Vegas, Nevada: 15.58MB/s
----------------------------------------------
Axel Download from OVH, BHS, Canada (http://bhs.proof.ovh.net/files/100Mio.dat)
Axel Download OVH, BHS, Canada: 103.93MB/s
----------------------------------------------
Axel Download from Vultr, Los Angeles, California (http://lax-ca-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Los Angeles, California: 37.17MB/s
----------------------------------------------
Axel Download from Vultr, Seattle, Washington (http://wa-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Seattle, Washington: 58.97MB/s
----------------------------------------------
Axel Download from Vultr, Dallas, Texas (http://tx-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Dallas, Texas: 81.73MB/s
----------------------------------------------
Axel Download from Vultr, Chicago, Illinois (http://il-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Chicago, Illinois: 135.76MB/s
----------------------------------------------
Axel Download from Vultr, Atlanta, Georgia (http://ga-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Atlanta, Georgia: 112.35MB/s
----------------------------------------------
Axel Download from Vultr, Miami, Florida (http://fl-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Miami, Florida: 124.39MB/s
----------------------------------------------
Axel Download from Vultr, New York / New Jersey (http://nj-us-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, New York / New Jersey: 295.89MB/s

-------------------------------------------
Asia bandwidth tests...
-------------------------------------------

----------------------------------------------
Axel Download from Linode, Tokyo, JP (http://speedtest.tokyo.linode.com/100MB-tokyo.bin)
Axel Download Linode, Tokyo, JP: 12.71MB/s
----------------------------------------------
Axel Download from Softlayer, Singapore (http://speedtest.sng01.softlayer.com/downloads/test100.zip)
Axel Download Softlayer, Singapore: 8.89MB/s
----------------------------------------------
Axel Download from Vultr, Tokyo, Japan (http://hnd-jp-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Tokyo, Japan: 19.12MB/s

-------------------------------------------
Europe bandwidth tests...
-------------------------------------------

----------------------------------------------
Axel Download from Linode, London, UK (http://speedtest.london.linode.com/100MB-london.bin)
Axel Download Linode, London, UK: 48.96MB/s
----------------------------------------------
Axel Download from OVH, Paris, France (http://proof.ovh.net/files/100Mio.dat)
Axel Download OVH, Paris, France: 9.32MB/s
----------------------------------------------
Axel Download from SmartDC, Rotterdam, Netherlands (http://mirror.i3d.net/100mb.bin)
Axel Download SmartDC, Rotterdam, Netherlands: 13.45MB/s
----------------------------------------------
Axel Download from Vultr, Amsterdam, Netherlands (http://ams-nl-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Amsterdam, Netherlands: 44.50MB/s
----------------------------------------------
Axel Download from Vultr, London, UK (http://lon-gb-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, London, UK: 60.40MB/s
----------------------------------------------
Axel Download from Vultr, Paris, France (http://par-fr-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Paris, France: 20.88MB/s

-------------------------------------------
Australia bandwidth tests...
-------------------------------------------

----------------------------------------------
Axel Download from Vultr, Sydney, Australia (http://syd-au-ping.vultr.com/vultr.com.100MB.bin)
Axel Download Vultr, Sydney, Australia: 17.90MB/s

    Code (Text):
    
-------------------------------------------
Running ping tests...
-------------------------------------------

----------------------------------------------
Pings (cachefly.cachefly.net):
PING vip1.g5.cachefly.net (205.234.175.175) 56(84) bytes of data.
64 bytes from vip1.G-anycast1.cachefly.net (205.234.175.175): icmp_seq=1 ttl=58 time=0.970 ms
64 bytes from vip1.G-anycast1.cachefly.net (205.234.175.175): icmp_seq=2 ttl=58 time=0.966 ms
64 bytes from vip1.G-anycast1.cachefly.net (205.234.175.175): icmp_seq=3 ttl=58 time=0.981 ms

--- vip1.g5.cachefly.net ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 0.966/0.972/0.981/0.026 ms

----------------------------------------------
Pings (syd-au-ping.vultr.com):
PING syd-au-ping.vultr.com (108.61.212.117) 56(84) bytes of data.
64 bytes from 108.61.212.117.vultr.com (108.61.212.117): icmp_seq=1 ttl=51 time=222 ms
64 bytes from 108.61.212.117.vultr.com (108.61.212.117): icmp_seq=2 ttl=51 time=222 ms
64 bytes from 108.61.212.117.vultr.com (108.61.212.117): icmp_seq=3 ttl=51 time=222 ms

--- syd-au-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 222.613/222.668/222.751/0.389 ms

----------------------------------------------
Pings (hnd-jp-ping.vultr.com):
PING hnd-jp-ping.vultr.com (108.61.201.151) 56(84) bytes of data.
64 bytes from 108.61.201.151.vultr.com (108.61.201.151): icmp_seq=1 ttl=50 time=175 ms
64 bytes from 108.61.201.151.vultr.com (108.61.201.151): icmp_seq=2 ttl=50 time=175 ms
64 bytes from 108.61.201.151.vultr.com (108.61.201.151): icmp_seq=3 ttl=50 time=174 ms

--- hnd-jp-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 174.955/175.015/175.054/0.344 ms

----------------------------------------------
Pings (lax-ca-us-ping.vultr.com):
PING lax-ca-us-ping.vultr.com (108.61.219.200) 56(84) bytes of data.
64 bytes from 108.61.219.200.vultr.com (108.61.219.200): icmp_seq=1 ttl=53 time=87.4 ms
64 bytes from 108.61.219.200.vultr.com (108.61.219.200): icmp_seq=2 ttl=53 time=87.5 ms
64 bytes from 108.61.219.200.vultr.com (108.61.219.200): icmp_seq=3 ttl=53 time=87.4 ms

--- lax-ca-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 87.417/87.474/87.526/0.245 ms

----------------------------------------------
Pings (wa-us-ping.vultr.com):
PING wa-us-ping.vultr.com (108.61.194.105) 56(84) bytes of data.
64 bytes from 108.61.194.105.vultr.com (108.61.194.105): icmp_seq=1 ttl=53 time=71.5 ms
64 bytes from 108.61.194.105.vultr.com (108.61.194.105): icmp_seq=2 ttl=53 time=71.6 ms
64 bytes from 108.61.194.105.vultr.com (108.61.194.105): icmp_seq=3 ttl=53 time=71.4 ms

--- wa-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 71.498/71.537/71.612/0.053 ms

----------------------------------------------
Pings (tx-us-ping.vultr.com):
PING tx-us-ping.vultr.com (108.61.224.175) 56(84) bytes of data.
64 bytes from 108.61.224.175.vultr.com (108.61.224.175): icmp_seq=1 ttl=52 time=39.0 ms
64 bytes from 108.61.224.175.vultr.com (108.61.224.175): icmp_seq=2 ttl=52 time=40.8 ms
64 bytes from 108.61.224.175.vultr.com (108.61.224.175): icmp_seq=3 ttl=52 time=39.5 ms

--- tx-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 39.088/39.845/40.893/0.798 ms

----------------------------------------------
Pings (il-us-ping.vultr.com):
PING il-us-ping.vultr.com (107.191.51.12) 56(84) bytes of data.
64 bytes from 107.191.51.12.vultr.com (107.191.51.12): icmp_seq=1 ttl=54 time=25.0 ms
64 bytes from 107.191.51.12.vultr.com (107.191.51.12): icmp_seq=2 ttl=54 time=24.9 ms
64 bytes from 107.191.51.12.vultr.com (107.191.51.12): icmp_seq=3 ttl=54 time=24.9 ms

--- il-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 24.957/24.979/25.007/0.130 ms

----------------------------------------------
Pings (ga-us-ping.vultr.com):
PING ga-us-ping.vultr.com (108.61.193.166) 56(84) bytes of data.
64 bytes from 108.61.193.166.vultr.com (108.61.193.166): icmp_seq=1 ttl=53 time=18.6 ms
64 bytes from 108.61.193.166.vultr.com (108.61.193.166): icmp_seq=2 ttl=53 time=18.5 ms
64 bytes from 108.61.193.166.vultr.com (108.61.193.166): icmp_seq=3 ttl=53 time=19.0 ms

--- ga-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 18.542/18.747/19.088/0.267 ms

----------------------------------------------
Pings (fl-us-ping.vultr.com):
PING fl-us-ping.vultr.com (104.156.244.232) 56(84) bytes of data.
64 bytes from 104.156.244.232 (104.156.244.232): icmp_seq=1 ttl=53 time=34.6 ms
64 bytes from 104.156.244.232 (104.156.244.232): icmp_seq=2 ttl=53 time=34.1 ms
64 bytes from 104.156.244.232 (104.156.244.232): icmp_seq=3 ttl=53 time=38.4 ms

--- fl-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 34.161/35.743/38.433/1.917 ms

----------------------------------------------
Pings (nj-us-ping.vultr.com):
PING nj-us-ping.vultr.com (108.61.149.182) 56(84) bytes of data.
64 bytes from 108.61.149.182.vultr.com (108.61.149.182): icmp_seq=1 ttl=56 time=2.29 ms
64 bytes from 108.61.149.182.vultr.com (108.61.149.182): icmp_seq=2 ttl=57 time=2.10 ms
64 bytes from 108.61.149.182.vultr.com (108.61.149.182): icmp_seq=3 ttl=56 time=2.29 ms

--- nj-us-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 2.101/2.227/2.290/0.089 ms

----------------------------------------------
Pings (fra-de-ping.vultr.com):
PING fra-de-ping.vultr.com (108.61.210.117) 56(84) bytes of data.
64 bytes from 108.61.210.117.vultr.com (108.61.210.117): icmp_seq=1 ttl=51 time=82.2 ms
64 bytes from 108.61.210.117.vultr.com (108.61.210.117): icmp_seq=2 ttl=51 time=82.0 ms
64 bytes from 108.61.210.117.vultr.com (108.61.210.117): icmp_seq=3 ttl=51 time=82.3 ms

--- fra-de-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 82.076/82.200/82.320/0.099 ms

----------------------------------------------
Pings (ams-nl-ping.vultr.com):
PING ams-nl-ping.vultr.com (108.61.198.102) 56(84) bytes of data.
64 bytes from 108.61.198.102.vultr.com (108.61.198.102): icmp_seq=1 ttl=53 time=79.5 ms
64 bytes from 108.61.198.102.vultr.com (108.61.198.102): icmp_seq=2 ttl=53 time=79.4 ms
64 bytes from 108.61.198.102.vultr.com (108.61.198.102): icmp_seq=3 ttl=53 time=79.4 ms

--- ams-nl-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 79.470/79.501/79.549/0.327 ms

----------------------------------------------
Pings (lon-gb-ping.vultr.com):
PING lon-gb-ping.vultr.com (108.61.196.101) 56(84) bytes of data.
64 bytes from 108.61.196.101.vultr.com (108.61.196.101): icmp_seq=1 ttl=53 time=71.3 ms
64 bytes from 108.61.196.101.vultr.com (108.61.196.101): icmp_seq=2 ttl=53 time=71.1 ms
64 bytes from 108.61.196.101.vultr.com (108.61.196.101): icmp_seq=3 ttl=53 time=71.2 ms

--- lon-gb-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 71.154/71.234/71.332/0.316 ms

----------------------------------------------
Pings (par-fr-ping.vultr.com):
PING par-fr-ping.vultr.com (108.61.209.127) 56(84) bytes of data.
64 bytes from 108.61.209.127.vultr.com (108.61.209.127): icmp_seq=1 ttl=54 time=85.5 ms
64 bytes from 108.61.209.127.vultr.com (108.61.209.127): icmp_seq=2 ttl=54 time=86.3 ms
64 bytes from 108.61.209.127.vultr.com (108.61.209.127): icmp_seq=3 ttl=54 time=86.1 ms

--- par-fr-ping.vultr.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 85.500/86.017/86.398/0.508 ms

----------------------------------------------
Pings (VersaWeb Las Vegas):
PING 104.143.15.254 (104.143.15.254) 56(84) bytes of data.
64 bytes from 104.143.15.254: icmp_seq=1 ttl=48 time=76.4 ms
64 bytes from 104.143.15.254: icmp_seq=2 ttl=48 time=76.5 ms
64 bytes from 104.143.15.254: icmp_seq=3 ttl=48 time=76.2 ms

--- 104.143.15.254 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 76.285/76.410/76.545/0.336 ms

----------------------------------------------
Pings (VersaWeb Seattle):
PING 76.164.234.1 (76.164.234.1) 56(84) bytes of data.
64 bytes from 76.164.234.1: icmp_seq=1 ttl=54 time=66.4 ms
64 bytes from 76.164.234.1: icmp_seq=2 ttl=54 time=66.4 ms
64 bytes from 76.164.234.1: icmp_seq=3 ttl=54 time=66.4 ms

--- 76.164.234.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 66.437/66.447/66.456/0.007 ms

----------------------------------------------
Pings (OVH Canada):
PING bhs.proof.ovh.net (192.99.19.165) 56(84) bytes of data.
64 bytes from bhs.proof.ovh.net (192.99.19.165): icmp_seq=1 ttl=56 time=8.83 ms
64 bytes from bhs.proof.ovh.net (192.99.19.165): icmp_seq=2 ttl=56 time=9.06 ms
64 bytes from bhs.proof.ovh.net (192.99.19.165): icmp_seq=3 ttl=56 time=8.86 ms

--- bhs.proof.ovh.net ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 8.830/8.920/9.063/0.102 ms

----------------------------------------------
Pings (10gbps.io Frankfurt):
PING 195.181.174.20 (195.181.174.20) 56(84) bytes of data.
64 bytes from 195.181.174.20: icmp_seq=1 ttl=51 time=93.1 ms
64 bytes from 195.181.174.20: icmp_seq=2 ttl=51 time=93.1 ms
64 bytes from 195.181.174.20: icmp_seq=3 ttl=51 time=93.1 ms

--- 195.181.174.20 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 93.115/93.165/93.197/0.251 ms

----------------------------------------------
Pings (10gbps.io London):
PING 185.59.221.20 (185.59.221.20) 56(84) bytes of data.
64 bytes from 185.59.221.20: icmp_seq=1 ttl=51 time=94.5 ms
64 bytes from 185.59.221.20: icmp_seq=2 ttl=51 time=94.5 ms
64 bytes from 185.59.221.20: icmp_seq=3 ttl=51 time=94.5 ms

--- 185.59.221.20 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 94.514/94.537/94.549/0.251 ms

----------------------------------------------
Pings (10gbps.io Paris):
PING 185.93.2.193 (185.93.2.193) 56(84) bytes of data.
64 bytes from 185.93.2.193: icmp_seq=1 ttl=51 time=101 ms
64 bytes from 185.93.2.193: icmp_seq=2 ttl=51 time=101 ms
64 bytes from 185.93.2.193: icmp_seq=3 ttl=51 time=101 ms

--- 185.93.2.193 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 101.232/101.260/101.313/0.369 ms

----------------------------------------------
Pings (10gbps.io Singapore):
PING 203.174.85.38 (203.174.85.38) 56(84) bytes of data.
64 bytes from 203.174.85.38: icmp_seq=1 ttl=51 time=251 ms
64 bytes from 203.174.85.38: icmp_seq=2 ttl=51 time=251 ms
64 bytes from 203.174.85.38: icmp_seq=3 ttl=51 time=251 ms

--- 203.174.85.38 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 251.615/251.724/251.938/0.598 ms

----------------------------------------------
Pings (10gbps.io Stockholm):
PING 185.76.9.10 (185.76.9.10) 56(84) bytes of data.
64 bytes from 185.76.9.10: icmp_seq=1 ttl=53 time=108 ms
64 bytes from 185.76.9.10: icmp_seq=2 ttl=53 time=108 ms
64 bytes from 185.76.9.10: icmp_seq=3 ttl=53 time=108 ms

--- 185.76.9.10 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 108.212/108.255/108.277/0.030 ms

----------------------------------------------
Pings (10gbps.io Moscow):
PING 87.245.204.20 (87.245.204.20) 56(84) bytes of data.
64 bytes from 87.245.204.20: icmp_seq=1 ttl=59 time=113 ms
64 bytes from 87.245.204.20: icmp_seq=2 ttl=59 time=113 ms
64 bytes from 87.245.204.20: icmp_seq=3 ttl=59 time=113 ms

--- 87.245.204.20 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 113.235/113.289/113.381/0.394 ms

----------------------------------------------
Pings (10gbps.io Miami):
PING 195.181.162.195 (195.181.162.195) 56(84) bytes of data.
64 bytes from 195.181.162.195: icmp_seq=1 ttl=55 time=55.8 ms
64 bytes from 195.181.162.195: icmp_seq=2 ttl=55 time=56.0 ms
64 bytes from 195.181.162.195: icmp_seq=3 ttl=55 time=55.9 ms

--- 195.181.162.195 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 55.874/55.950/56.060/0.208 ms

    Code (Text):
    
-------------------------------------------
Running mtr tests...
-------------------------------------------

----------------------------------------------
mtr --report -w 200 --report-cycles=10 cachefly.cachefly.net
Start: Mon Feb 26 13:40:44 2018
HOST: hostname          Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                       0.0%    10   11.8  16.0  11.8  21.7   3.9
  2.|-- 147.75.36.4                   0.0%    10   11.9  55.4  11.8 301.0  92.7
  3.|-- 0.ae12.bbr1.ewr1.packet.net   0.0%    10    0.2   0.4   0.2   2.7   0.7
  4.|-- ???                          100.0    10    0.0   0.0   0.0   0.0   0.0
  5.|-- nyk-b6-link.telia.net         0.0%    10    1.0   1.8   1.0   6.4   1.7
  6.|-- nyk-b5-link.telia.net         0.0%    10    2.8   1.5   1.0   2.8   0.7
  7.|-- ???                          100.0    10    0.0   0.0   0.0   0.0   0.0
  8.|-- vip1.G-anycast1.cachefly.net  0.0%    10    1.0   1.0   1.0   1.2   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.atlanta.linode.com
Start: Mon Feb 26 13:40:59 2018
HOST: hostname                  Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                               0.0%    10   10.7  13.8   0.9  88.4  26.4
  2.|-- 2604:1380:2:79::                      0.0%    10    1.5   6.0   1.5  10.7   2.7
  3.|-- 2604:1380:2:20::                      0.0%    10    0.3   0.4   0.2   1.1   0.0
  4.|-- 2001:518:6900:1::5                    0.0%    10    0.7   2.5   0.3  20.7   6.4
  5.|-- 2001:518:2001:15::1                   0.0%    10    0.2   1.4   0.2  12.1   3.7
  6.|-- 2001:518:6001:6::2                    0.0%    10    1.1   2.1   1.0  12.0   3.4
  7.|-- nyk-b3-link.telia.net                 0.0%    10    1.2   1.4   1.1   2.8   0.5
  8.|-- nyk-bb4-v6.telia.net                  0.0%    10   21.5  21.2  21.1  21.5   0.0
  9.|-- ash-bb4-v6.telia.net                  0.0%    10   21.9  21.8  21.6  22.0   0.0
 10.|-- atl-b22-v6.telia.net                  0.0%    10   31.3  25.5  24.1  31.3   2.3
 11.|-- linode-ic-321031-atl-b22.c.telia.net  0.0%    10   21.8  22.1  21.7  23.2   0.3
 12.|-- 2600:3c02:4444:4::2                   0.0%    10   21.9  22.0  21.8  22.5   0.0
 13.|-- 2600:3c02::4b                         0.0%    10   21.5  21.5  21.4  21.6   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.dallas.linode.com
Start: Mon Feb 26 13:41:16 2018
HOST: hostname           Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                        0.0%    10   19.4  19.8   9.5  29.4   6.6
  2.|-- 2604:1380:2:79::               0.0%    10    3.2   5.5   0.5  11.0   3.2
  3.|-- 2604:1380:2:20::               0.0%    10    0.2   3.3   0.2  31.2   9.8
  4.|-- 2001:518:6900:1::5             0.0%    10    0.5   4.5   0.4  40.7  12.7
  5.|-- 2001:518:2001:16::1            0.0%    10    0.2   0.5   0.2   1.9   0.3
  6.|-- 2001:518:6001:7::2             0.0%    10    1.1   1.1   1.1   1.1   0.0
  7.|-- ae2-63.cr0-nyc2.ip6.gtt.net    0.0%    10   70.1  70.1  70.0  70.2   0.0
  8.|-- 2001:668:0:2:ffff:0:5995:8f3d  0.0%    10   80.4  80.3  80.2  80.6   0.0
  9.|-- 2001:668:0:3:ffff:2:0:26a      0.0%    10   42.2  42.2  42.1  42.4   0.0
 10.|-- 2600:3c00:2222:13::2           0.0%    10   43.0  42.8  42.6  43.1   0.0
 11.|-- speedtest.dallas.linode.com    0.0%    10   44.9  44.9  44.8  45.0   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 mirror.us.leaseweb.net
Start: Mon Feb 26 13:41:32 2018
HOST: hostname           Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                        0.0%    10   23.8  46.5  11.8 119.5  36.9
  2.|-- 2604:1380:2:78::               0.0%    10    0.5   4.5   0.5  10.0   3.2
  3.|-- 2604:1380:2:2::1               0.0%    10    0.2   2.1   0.2  19.8   6.2
  4.|-- 2001:518:6900:1::1             0.0%    10    0.7   0.6   0.4   0.7   0.0
  5.|-- 2001:518:2001:14::1            0.0%    10    0.9   0.5   0.2   2.1   0.6
  6.|-- 2001:518:6001:7::2             0.0%    10    1.1   1.2   1.0   1.7   0.0
  7.|-- ae2-63.cr0-nyc2.ip6.gtt.net    0.0%    10   80.3  71.1  70.0  80.3   3.2
  8.|-- 2001:668:0:2:ffff:0:d5fe:d6aa  0.0%    10   76.5  77.1  76.3  82.9   2.0
  9.|-- 2001:668:0:3:ffff:0:45ae:326   0.0%    10   52.2  52.2  52.1  52.3   0.0
 10.|-- 2604:9a00::3d                  0.0%    10   33.7  33.8  33.7  34.0   0.0
 11.|-- 2604:9a00:2010:a0b8::5         0.0%    10   33.9  34.0  33.9  34.0   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.sea01.softlayer.com
Start: Mon Feb 26 13:41:49 2018
HOST: hostname                                               Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                                            0.0%    10   16.8  22.6   6.6  62.4  14.9
  2.|-- 2604:1380:2:79::                                                   0.0%    10   10.7   5.7   0.9  10.7   2.5
  3.|-- 2604:1380:2:20::                                                   0.0%    10    0.2   0.3   0.2   1.3   0.0
  4.|-- de-cix.nyc.us.ibm.com                                              0.0%    10    1.3   1.4   1.1   2.6   0.3
  5.|-- 6320.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10    2.2   2.2   1.7   4.5   0.7
  6.|-- a320.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   22.9  22.8  22.7  23.2   0.0
  7.|-- 5220.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   23.0  23.2  22.9  23.5   0.0
  8.|-- ee20.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   64.5  64.6  63.8  65.4   0.0
  9.|-- 1630.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   62.6  62.7  62.5  63.1   0.0
 10.|-- po3.bbr01.sr01.sea01.networklayer.com                              0.0%    10   62.3  62.5  62.3  63.6   0.3
 11.|-- po1.fcr01.sr01.sea01.networklayer.com                              0.0%    10   63.6  63.4  63.1  63.7   0.0
 12.|-- softlayer.com                                                      0.0%    10   62.8  62.8  62.8  62.8   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.sjc01.softlayer.com
Start: Mon Feb 26 13:42:06 2018
HOST: hostname                                               Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                                            0.0%    10   24.5  30.9  11.0 102.3  25.6
  2.|-- 2604:1380:2:79::                                                   0.0%    10    7.3   5.9   0.7  10.6   2.8
  3.|-- 2604:1380:2:20::                                                   0.0%    10    0.5   0.2   0.2   0.5   0.0
  4.|-- de-cix.nyc.us.ibm.com                                              0.0%    10    1.2   1.2   1.2   1.4   0.0
  5.|-- 6320.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10    1.9   2.0   1.7   2.3   0.0
  6.|-- a320.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   26.0  22.9  22.5  26.0   1.1
  7.|-- 5220.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   23.2  23.0  22.7  23.5   0.0
  8.|-- 8320.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   44.7  45.0  44.7  45.2   0.0
  9.|-- 3620.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   68.0  67.5  67.0  68.9   0.3
 10.|-- 3520.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   69.1  70.7  69.1  78.2   2.7
 11.|-- 9700.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   66.6  66.8  66.6  67.1   0.0
 12.|-- 2607:f0d0:2100:1::3                                               10.0%    10   68.6  69.7  68.6  71.4   1.1
 13.|-- softlayer.com                                                      0.0%    10   71.0  71.0  70.9  71.0   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.wdc01.softlayer.com
Start: Mon Feb 26 13:42:21 2018
HOST: hostname                                               Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                                           10.0%    10    1.0   6.2   0.7  15.6   5.0
  2.|-- 2604:1380:2:78::                                                   0.0%    10   11.1   6.3   0.5  11.3   3.4
  3.|-- 2604:1380:2:5::1                                                   0.0%    10    0.3   0.2   0.2   0.3   0.0
  4.|-- de-cix.nyc.us.ibm.com                                              0.0%    10    1.2   1.4   1.1   3.5   0.7
  5.|-- 2320.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10    2.0   3.2   1.7  10.6   2.9
  6.|-- 2830.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10    7.1   7.6   7.1   9.8   0.7
  7.|-- e730.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   11.4   7.9   7.4  11.4   1.2
  8.|-- bbr01.eq01.wdc02.networklayer.com                                  0.0%    10    7.2   7.3   7.0   7.8   0.0
  9.|-- po2.dar01.sr01.wdc01.networklayer.com                              0.0%    10    8.6   7.8   7.4   9.3   0.5
 10.|-- po1.fcr01.sr01.wdc01.networklayer.com                              0.0%    10    7.8   7.7   7.6   7.9   0.0
 11.|-- softlayer.com                                                      0.0%    10    7.8  12.3   7.8  20.2   4.7

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.tokyo.linode.com
Start: Mon Feb 26 13:42:37 2018
HOST: hostname          Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                       0.0%    10   10.7   5.7   1.2  10.7   3.1
  2.|-- 2604:1380:2:78::              0.0%    10    6.2   6.8   1.5   9.8   2.3
  3.|-- 2604:1380:2:5::1              0.0%    10    0.3   1.4   0.2  11.9   3.6
  4.|-- 10ge3-9.core1.nyc6.he.net     0.0%    10    1.2   1.7   1.1   6.4   1.5
  5.|-- peering.kddi.com              0.0%    10   80.2  79.5  76.6  80.4   1.3
  6.|-- 6nytjbb001.int-gw.kddi.ne.jp  0.0%    10   80.0  80.4  79.8  83.0   0.9
  7.|-- 6lacGCS001.int-gw.kddi.ne.jp  0.0%    10   65.6  67.5  65.5  72.7   3.2
  8.|-- 2001:268:fb80:300::1          0.0%    10  238.2 239.5 238.2 240.4   0.3
  9.|-- 2001:268:fa00:282::2          0.0%    10  237.7 239.0 237.7 249.9   3.8
 10.|-- 6cm-fcu204.int-gw.kddi.ne.jp  0.0%    10  238.5 244.0 238.5 251.5   4.6
 11.|-- 2001:268:f702:6d::2           0.0%    10  242.9 243.0 242.9 243.7   0.0
 12.|-- speedtest.tokyo.linode.com    0.0%    10  238.6 238.8 238.6 240.0   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.sng01.softlayer.com
Start: Mon Feb 26 13:42:56 2018
HOST: hostname                                               Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                                            0.0%    10    4.0   6.2   3.1   9.5   2.1
  2.|-- 2604:1380:2:78::                                                   0.0%    10    4.5   4.0   0.2   8.7   2.4
  3.|-- 2604:1380:2:2::1                                                   0.0%    10    0.5   3.6   0.2  32.7  10.2
  4.|-- 2604:1380:2:1::                                                    0.0%    10    0.4   0.4   0.2   1.0   0.0
  5.|-- de-cix.nyc.us.ibm.com                                              0.0%    10    1.2   1.3   1.2   2.4   0.3
  6.|-- 6320.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10    1.9   2.0   1.8   2.4   0.0
  7.|-- a320.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   22.8  23.3  22.8  25.6   0.7
  8.|-- 5220.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   22.9  23.3  22.9  24.8   0.3
  9.|-- 8320.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   47.5  47.6  47.4  47.9   0.0
 10.|-- a120.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   48.0  48.3  47.4  53.7   1.9
 11.|-- c520.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10   75.2  75.1  72.8  75.7   0.6
 12.|-- 8b40.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10  215.1 215.5 214.6 217.5   0.8
 13.|-- 7940.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10  240.6 242.2 240.4 254.7   4.4
 14.|-- b840.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10  240.5 240.5 240.4 240.9   0.0
 15.|-- db00.0000.0000.0000.2000.2000.0d0f.7062.ip6.static.sl-reverse.com  0.0%    10  241.2 240.5 240.3 241.2   0.0
 16.|-- 3.00.0000.ip4.static.sl-reverse.com                               30.0%    10  240.4 241.2 240.4 245.8   1.9
 17.|-- 2.00.0000.ip4.static.sl-reverse.com                                0.0%    10  240.9 240.8 240.7 240.9   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 speedtest.london.linode.com
Start: Mon Feb 26 13:43:12 2018
HOST: hostname                 Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                              0.0%    10   30.4   8.0   0.3  30.4   8.5
  2.|-- 2604:1380:2:78::                     0.0%    10    5.8   6.8   2.9  11.2   2.6
  3.|-- 2604:1380:2:2::1                     0.0%    10    0.2   0.7   0.2   5.2   1.5
  4.|-- 2001:518:6900:1::1                   0.0%    10    0.6   5.7   0.4  52.3  16.4
  5.|-- 2001:518:2001:13::1                  0.0%    10    0.2   2.4   0.2  12.7   3.8
  6.|-- 2001:518:6001:6::2                   0.0%    10    5.8   6.6   1.0  12.0   4.0
  7.|-- nyk-b3-link.telia.net                0.0%    10    1.2   2.4   1.1  13.3   3.8
  8.|-- nyk-bb4-v6.telia.net                 0.0%    10   70.8  70.8  70.7  71.2   0.0
  9.|-- 2001:2000:3019:7b::1                 0.0%    10   70.8  71.8  70.8  77.3   2.0
 10.|-- ldn-bb2-v6.telia.net                 0.0%    10  109.4  79.3  70.6 109.4  13.5
 11.|-- ldn-b4-v6.telia.net                  0.0%    10   70.7  70.7  70.6  70.8   0.0
 12.|-- linode-ic-318542-ldn-b3.c.telia.net  0.0%    10   72.9  71.8  71.4  72.9   0.5
 13.|-- 2a01:7e00:7777:4::2                  0.0%    10   71.5  71.5  71.3  71.6   0.0
 14.|-- speedtest.london.linode.com          0.0%    10   71.1  71.0  71.0  71.1   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 mirror.i3d.net
Start: Mon Feb 26 13:43:29 2018
HOST: hostname              Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                           0.0%    10   21.4  21.8   6.3  28.7   6.5
  2.|-- 2604:1380:2:78::                  0.0%    10    4.3   3.9   0.4   9.8   3.3
  3.|-- 2604:1380:2:2::1                  0.0%    10    0.3   2.2   0.2  19.4   6.0
  4.|-- 2001:518:6900:1::1                0.0%    10    0.4   3.1   0.4  25.9   8.0
  5.|-- 2001:518:2001:14::1               0.0%    10    0.2   0.4   0.2   1.9   0.5
  6.|-- 2001:518:1001:7::2                0.0%    10    0.9   2.9   0.9   6.8   2.1
  7.|-- 10ge.eqx-ashburn.misi3d.com       0.0%    10    6.3   6.5   6.1   9.2   0.7
  8.|-- 10ge.lr0-iadr0.th-n.lon.i3d.net   0.0%    10   78.6  79.1  77.8  89.9   3.7
  9.|-- 30ge.cr1-lr0.smartdc.rtm.i3d.net  0.0%    10   84.5  84.5  84.4  85.6   0.3
 10.|-- 80ge.br2-cr1.smartdc.rtd.i3d.net  0.0%    10   84.4  96.7  84.3 146.7  25.4
 11.|-- hosted-by.i3d.net                 0.0%    10   84.4  84.8  84.4  87.8   0.8

----------------------------------------------
mtr --report -w 200 --report-cycles=10 syd-au-ping.vultr.com
Start: Mon Feb 26 13:43:45 2018
HOST: hostname                         Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                      0.0%    10   38.6  33.9  26.8  45.1   5.7
  2.|-- 147.75.36.4                                  0.0%    10  492.6 167.4  12.0 1010. 331.5
  3.|-- 0.ae22.bbr2.ewr1.packet.net                  0.0%    10    0.3   0.3   0.2   0.4   0.0
  4.|-- 10ge3-9.core1.nyc6.he.net                    0.0%    10    6.9   2.6   1.3   8.3   2.6
  5.|-- 100ge13-1.core1.nyc4.he.net                  0.0%    10    1.4   5.8   1.4  17.9   4.8
  6.|-- 100ge8-1.core1.sjc2.he.net                   0.0%    10   63.0  63.1  62.9  63.6   0.0
  7.|-- 10ge4-4.core1.sjc1.he.net                    0.0%    10   64.1  63.7  63.1  67.1   1.1
  8.|-- vocus.gigabitethernet2-13.core1.sjc1.he.net  0.0%    10   68.4  68.5  68.4  68.7   0.0
  9.|-- bundle-153.cor02.sjc01.ca.vocus.net          0.0%    10  218.5 218.2 218.1 218.5   0.0
 10.|-- BE-201.cor02.syd04.nsw.VOCUS.net.au          0.0%    10  218.1 218.2 218.0 218.5   0.0
 11.|-- bundle-101.bdr01.syd03.nsw.vocus.net.au      0.0%    10  219.0 218.7 218.5 219.0   0.0
 12.|-- static-73.7.255.49.in-addr.VOCUS.net.au      0.0%    10  217.8 217.8 217.8 217.9   0.0
 13.|-- xe-1-0-1.gw102.sy3.ap.equinix.com            0.0%    10  213.7 213.7 213.7 213.8   0.0
 14.|-- 180.189.25.6                                30.0%    10  231.4 231.7 231.4 232.9   0.4
 15.|-- 103-43-72-34.choopa.net                      0.0%    10  232.1 232.1 232.0 232.3   0.0
 16.|-- ???                                         100.0    10    0.0   0.0   0.0   0.0   0.0
 17.|-- 108.61.212.117.vultr.com                     0.0%    10  222.8 222.8 222.6 222.9   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 hnd-jp-ping.vultr.com
Start: Mon Feb 26 13:44:03 2018
HOST: hostname                        Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                     0.0%    10   11.3  14.3  11.3  19.8   2.9
  2.|-- 147.75.36.4                                 0.0%    10   12.5 202.5  12.5 1031. 334.8
  3.|-- 0.ae22.bbr2.ewr1.packet.net                 0.0%    10    0.2   0.1   0.1   0.2   0.0
  4.|-- 0.xe-0-0-0.csr1.ewr2.packet.net             0.0%    10  114.7  16.3   1.2 114.7  34.8
  6.|-- 0.et-5-0-5.BR2.NYC4.ALTER.NET               0.0%    10    1.5   3.4   1.4  20.0   5.8
  7.|-- verizon.com.customer.alter.net              0.0%    10    1.8   1.7   1.5   2.1   0.0
  8.|-- ae-3.r24.nycmny01.us.bb.gin.ntt.net         0.0%    10    3.2   2.5   2.3   3.3   0.0
  9.|-- ae-2.r20.sttlwa01.us.bb.gin.ntt.net         0.0%    10   69.6  72.8  69.5  94.4   7.9
 10.|-- ae-13.r30.tokyjp05.jp.bb.gin.ntt.net       10.0%    10  177.4 174.0 173.0 177.4   1.3
 11.|-- ae-2.r02.tokyjp05.jp.bb.gin.ntt.net         0.0%    10  181.2 181.0 181.0 181.2   0.0
 12.|-- ce-0-14-0-1.r02.tokyjp05.jp.ce.gin.ntt.net  0.0%    10  206.2 187.4 181.7 206.2   8.6
 13.|-- ???                                        100.0    10    0.0   0.0   0.0   0.0   0.0
 14.|-- vl516-ds1-b5-r2605.tyo1.choopa.net          0.0%    10  181.4 182.2 181.4 188.6   2.2
 15.|-- ???                                        100.0    10    0.0   0.0   0.0   0.0   0.0
 16.|-- 108.61.201.151.vultr.com                    0.0%    10  175.1 175.0 174.9 175.1   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 lax-ca-us-ping.vultr.com
Start: Mon Feb 26 13:44:20 2018
HOST: hostname                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10   20.2  23.7  11.3  90.5  23.7
  2.|-- 147.75.36.2                         0.0%    10   14.8 130.2   9.0 1052. 324.7
  3.|-- 0.ae11.bbr1.ewr1.packet.net         0.0%    10    0.2   2.9   0.1  27.0   8.5
  4.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
  5.|-- nyk-b6-link.telia.net               0.0%    10    1.0   1.3   1.0   1.9   0.0
  6.|-- nyk-bb3-link.telia.net              0.0%    10    1.2   2.3   1.1  11.6   3.2
  7.|-- las-b24-link.telia.net              0.0%    10   88.0  89.3  88.0  94.8   2.2
  8.|-- las-b4-link.telia.net               0.0%    10   87.5  87.6  87.4  88.7   0.3
  9.|-- vultr-ic-318444-las-b4.c.telia.net  0.0%    10   90.1  91.9  87.4  97.9   3.8
 10.|-- vl901-ds1-b5-c705.lax3.choopa.net   0.0%    10   88.5  88.7  88.4  89.6   0.0
 11.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
 12.|-- 108.61.219.200.vultr.com            0.0%    10   87.5  87.5  87.5  87.6   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 wa-us-ping.vultr.com
Start: Mon Feb 26 13:44:37 2018
HOST: hostname                  Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                               0.0%    10   11.3  16.2  11.3  21.3   3.8
  2.|-- 147.75.36.2                           0.0%    10   19.6 115.0  11.8 712.8 229.6
  3.|-- 0.ae11.bbr1.ewr1.packet.net           0.0%    10    1.4   0.3   0.1   1.4   0.3
  4.|-- 0.xe-0-0-1.csr1.ewr2.packet.net       0.0%    10    3.8   5.3   1.4  11.8   3.3
  7.|-- lag-10.ear4.Newark1.Level3.net       30.0%    10    1.2   1.1   1.0   1.2   0.0
  8.|-- ae-2-3613.edge1.Seattle3.Level3.net   0.0%    10   74.8  74.8  74.7  75.2   0.0
  9.|-- CHOOPA-LLC.edge1.Seattle3.Level3.net  0.0%    10   72.0  71.6  71.5  72.0   0.0
 10.|-- ???                                  100.0    10    0.0   0.0   0.0   0.0   0.0
 11.|-- 108.61.194.105.vultr.com              0.0%    10   71.4  71.5  71.4  71.7   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 tx-us-ping.vultr.com
Start: Mon Feb 26 13:44:52 2018
HOST: hostname                 Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                              0.0%    10   11.7  21.4  11.7  63.8  15.8
  2.|-- 147.75.36.2                          0.0%    10   17.2  64.0  12.1 298.2 101.5
  3.|-- 0.ae21.bbr2.ewr1.packet.net          0.0%    10    0.3   0.4   0.2   1.1   0.0
  4.|-- 0.ae0.bbr1.ewr1.packet.net           0.0%    10    3.2   1.4   0.3   7.0   2.0
  5.|-- ???                                 100.0    10    0.0   0.0   0.0   0.0   0.0
  6.|-- nyk-b6-link.telia.net                0.0%    10    1.6   1.1   1.0   1.6   0.0
  7.|-- nyk-bb3-link.telia.net               0.0%    10   39.0  39.5  38.9  42.9   1.1
  8.|-- ash-bb3-link.telia.net               0.0%    10   36.8  37.0  36.3  40.9   1.3
  9.|-- atl-b22-link.telia.net               0.0%    10   20.6  19.2  18.7  20.6   0.5
 10.|-- dls-b22-link.telia.net               0.0%    10   38.9  38.8  38.7  38.9   0.0
 11.|-- vultr-ic-318447-dls-b22.c.telia.net 10.0%    10   39.2  39.1  39.0  39.7   0.0
 12.|-- 108-61-110-6.choopa.net             90.0%    10  158.5 158.5 158.5 158.5   0.0
 13.|-- ???                                 100.0    10    0.0   0.0   0.0   0.0   0.0
 14.|-- 108.61.224.175.vultr.com             0.0%    10   42.7  39.5  39.1  42.7   1.1

----------------------------------------------
mtr --report -w 200 --report-cycles=10 il-us-ping.vultr.com
Start: Mon Feb 26 13:45:09 2018
HOST: hostname                 Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                              0.0%    10   17.3  35.6  12.4 105.2  34.5
  2.|-- 147.75.36.4                          0.0%    10  1086. 295.3  13.1 1086. 457.1
  3.|-- 0.ae12.bbr1.ewr1.packet.net          0.0%    10    0.2   4.0   0.2  37.5  11.7
  4.|-- ???                                 100.0    10    0.0   0.0   0.0   0.0   0.0
  5.|-- nyk-b6-link.telia.net                0.0%    10    1.0   1.0   1.0   1.0   0.0
  6.|-- nyk-bb4-link.telia.net               0.0%    10   24.0  23.8  23.7  24.0   0.0
  7.|-- chi-b21-link.telia.net               0.0%    10   24.6  24.6  24.6  24.7   0.0
  8.|-- vultr-ic-318445-chi-b21.c.telia.net  0.0%    10   27.5  26.8  26.5  27.5   0.0
  9.|-- vl820-ds1-b5-gj178.chi2.choopa.net   0.0%    10   25.0  25.1  24.9  25.3   0.0
 10.|-- ???                                 100.0    10    0.0   0.0   0.0   0.0   0.0
 11.|-- 107.191.51.12.vultr.com              0.0%    10   25.0  25.1  24.9  25.8   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 ga-us-ping.vultr.com
Start: Mon Feb 26 13:45:24 2018
HOST: hostname                 Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                              0.0%    10   14.9  15.1   7.6  25.6   5.5
  2.|-- 147.75.36.2                          0.0%    10   18.0 177.5  13.0 642.2 262.6
  3.|-- 0.ae21.bbr2.ewr1.packet.net          0.0%    10    0.3   0.3   0.2   0.7   0.0
  4.|-- 0.ae0.bbr1.ewr1.packet.net           0.0%    10   10.4   1.5   0.4  10.4   3.1
  5.|-- ???                                 100.0    10    0.0   0.0   0.0   0.0   0.0
  6.|-- nyk-b6-link.telia.net                0.0%    10    1.1   1.2   1.0   2.1   0.0
  7.|-- nyk-bb3-link.telia.net               0.0%    10   18.6  19.5  18.6  24.6   1.8
  8.|-- ash-bb3-link.telia.net               0.0%    10    7.7   8.2   7.4  13.3   1.7
  9.|-- atl-b21-link.telia.net               0.0%    10   18.4  18.6  18.4  19.3   0.0
 10.|-- vultr-ic-319646-atl-b21.c.telia.net  0.0%    10   21.7  18.7  18.3  21.7   1.1
 11.|-- 45-63-44-129.choopa.net              0.0%    10   19.0  18.9  18.8  19.0   0.0
 12.|-- ???                                 100.0    10    0.0   0.0   0.0   0.0   0.0
 13.|-- 108.61.193.166.vultr.com             0.0%    10   18.7  18.8  18.6  19.3   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 fl-us-ping.vultr.com
Start: Mon Feb 26 13:45:40 2018
HOST: hostname                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10   41.2  27.9  17.7  41.2   7.6
  2.|-- 147.75.36.2                         0.0%    10  389.3  96.6  12.3 389.3 131.5
  3.|-- 0.ae21.bbr2.ewr1.packet.net         0.0%    10    0.2   0.3   0.2   0.5   0.0
  4.|-- 0.ae0.bbr1.ewr1.packet.net          0.0%    10    0.9   0.9   0.4   4.6   1.3
  5.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
  6.|-- nyk-b6-link.telia.net               0.0%    10    1.1   1.1   1.0   1.9   0.0
  7.|-- nyk-bb3-link.telia.net              0.0%    10    1.2   4.8   1.1  11.8   4.5
  8.|-- ash-bb3-link.telia.net              0.0%    10    7.3   8.7   7.3  21.3   4.4
  9.|-- mai-b1-link.telia.net               0.0%    10   34.3  34.3  34.2  34.6   0.0
 10.|-- vultr-ic-318945-mai-b1.c.telia.net  0.0%    10   34.6  34.6  34.4  34.8   0.0
 11.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
 12.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
 13.|-- 104.156.244.232                     0.0%    10   34.1  34.0  34.0  34.1   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 nj-us-ping.vultr.com
Start: Mon Feb 26 13:45:57 2018
HOST: hostname                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10   14.1  17.5  13.6  20.4   2.5
  2.|-- 147.75.36.4                         0.0%    10   12.8  74.9  12.8 524.1 159.4
  3.|-- 0.ae12.bbr1.ewr1.packet.net         0.0%    10    0.2   0.7   0.2   5.4   1.5
  4.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
  5.|-- nyk-b6-link.telia.net               0.0%    10    1.0   1.2   1.0   2.5   0.3
  6.|-- nyk-bb3-link.telia.net              0.0%    10    5.1  11.3   1.1  19.4   6.1
  7.|-- nyk-b2-link.telia.net               0.0%    10    1.7   1.7   1.7   1.8   0.0
  8.|-- vultr-ic-318452-nyk-b2.c.telia.net  0.0%    10    2.1   2.0   1.9   2.3   0.0
  9.|-- 66-55-144-149.choopa.net            0.0%    10    3.8   2.6   2.1   4.4   0.7
 10.|-- 108-61-244-174.choopa.net           0.0%    10    2.6   2.8   2.5   3.9   0.3
 11.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
 12.|-- 108.61.149.182.vultr.com            0.0%    10    2.1   2.1   2.0   2.2   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 fra-de-ping.vultr.com
Start: Mon Feb 26 13:46:12 2018
HOST: hostname                 Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                              0.0%    10   14.5  18.3  11.8  31.7   5.8
  2.|-- 147.75.36.2                          0.0%    10  867.7 221.5  12.8 867.7 345.3
  3.|-- 0.ae21.bbr2.ewr1.packet.net          0.0%    10    0.3   1.7   0.2  15.0   4.6
  4.|-- packet.tbr2.wbr.nac.net              0.0%    10    0.5   0.5   0.3   0.7   0.0
  5.|-- 0.lag16.tbr1.wbr.nac.net             0.0%    10    0.2   1.0   0.2   4.5   1.5
  6.|-- 0.e3-4.tbr1.tl9.nac.net              0.0%    10    8.8   4.7   1.1  11.2   4.6
  7.|-- nyk-b3-link.telia.net                0.0%    10    1.2   1.5   1.2   2.5   0.3
  8.|-- nyk-bb4-link.telia.net               0.0%    10    1.8   1.9   1.7   2.2   0.0
  9.|-- prs-bb4-link.telia.net               0.0%    10   88.2  88.5  88.1  91.0   0.8
 10.|-- prs-b2-link.telia.net                0.0%    10   72.4  72.4  72.4  72.6   0.0
 11.|-- prs-bb3-link.telia.net               0.0%    10   81.7  81.7  81.7  81.9   0.0
 12.|-- ffm-bb3-link.telia.net               0.0%    10   87.9  88.0  87.8  88.6   0.0
 13.|-- ffm-b12-link.telia.net               0.0%    10   85.4  86.0  85.3  89.4   1.1
 14.|-- vultr-ic-312911-ffm-b12.c.telia.net  0.0%    10   82.1  93.1  81.8 189.2  33.7
 15.|-- ???                                 100.0    10    0.0   0.0   0.0   0.0   0.0
 16.|-- 108.61.210.117.vultr.com             0.0%    10   81.7  82.0  81.6  82.4   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 ams-nl-ping.vultr.com
Start: Mon Feb 26 13:46:29 2018
HOST: hostname                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10   35.5  30.8  18.8  39.5   5.8
  2.|-- 147.75.36.4                         0.0%    10   20.3 196.9  15.5 1123. 345.3
  3.|-- 0.ae22.bbr2.ewr1.packet.net         0.0%    10    0.3   1.4   0.2  11.4   3.5
  4.|-- packet.tbr2.wbr.nac.net             0.0%    10    0.3   6.8   0.3  53.9  16.9
  5.|-- 0.lag16.tbr1.wbr.nac.net            0.0%    10    0.2   1.6   0.2   6.4   2.2
  6.|-- 0.e3-4.tbr1.tl9.nac.net             0.0%    10    7.4   5.0   1.1  11.2   4.0
  7.|-- nyk-b3-link.telia.net               0.0%    10    1.3   1.2   1.2   1.4   0.0
  8.|-- nyk-bb3-link.telia.net              0.0%    10    1.2   1.3   1.2   1.5   0.0
  9.|-- ldn-bb3-link.telia.net              0.0%    10   71.3  71.2  71.0  72.6   0.5
 10.|-- adm-bb3-link.telia.net              0.0%    10   79.2  79.2  79.0  79.5   0.0
 11.|-- adm-b3-link.telia.net               0.0%    10   79.5  79.4  79.3  79.5   0.0
 12.|-- vultr-ic-313751-adm-b3.c.telia.net 20.0%    10   79.4  79.6  79.4  79.8   0.0
 13.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
 14.|-- 108.61.198.102.vultr.com            0.0%    10   79.6  79.5  79.4  79.7   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 lon-gb-ping.vultr.com
Start: Mon Feb 26 13:46:45 2018
HOST: hostname                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10   30.6  32.6  21.8  58.2  10.1
  2.|-- 147.75.36.2                         0.0%    10  928.0 252.1  11.3 928.0 383.6
  3.|-- 0.ae21.bbr2.ewr1.packet.net         0.0%    10    0.2   3.5   0.2  32.8  10.3
  4.|-- packet.tbr2.wbr.nac.net             0.0%    10  131.3  63.4   0.3 192.5  75.4
  5.|-- 0.lag16.tbr1.wbr.nac.net            0.0%    10    0.2   0.8   0.2   5.8   1.6
  6.|-- 0.e3-4.tbr1.tl9.nac.net             0.0%    10    1.1   1.6   1.1   3.6   0.7
  7.|-- nyk-b3-link.telia.net               0.0%    10    2.2   4.3   1.2  29.9   9.0
  8.|-- nyk-bb3-link.telia.net              0.0%    10    1.2   2.9   1.1  12.4   3.6
  9.|-- ldn-bb3-link.telia.net              0.0%    10   71.0  71.6  71.0  75.5   1.4
 10.|-- ldn-b9-link.telia.net               0.0%    10   71.0  71.1  70.9  72.0   0.0
 11.|-- vultr-ic-312913-ldn-b9.c.telia.net  0.0%    10   72.8  72.8  71.7  78.7   2.1
 12.|-- 10.69.1.90                         10.0%    10   72.6  93.3  72.0 176.4  41.2
 13.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
 14.|-- 108.61.196.101.vultr.com            0.0%    10   71.3  71.2  71.1  71.3   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 par-fr-ping.vultr.com
Start: Mon Feb 26 13:47:01 2018
HOST: hostname                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                             0.0%    10   16.1  14.0   6.2  20.5   3.9
  2.|-- 147.75.36.2                         0.0%    10   19.4  96.7   6.5 514.7 175.1
  3.|-- 0.ae21.bbr2.ewr1.packet.net         0.0%    10    0.3   0.3   0.2   0.3   0.0
  4.|-- packet.tbr2.wbr.nac.net             0.0%    10    0.3   0.4   0.3   0.6   0.0
  5.|-- 0.lag16.tbr1.wbr.nac.net            0.0%    10    0.2   0.3   0.2   0.4   0.0
  6.|-- 0.e3-4.tbr1.tl9.nac.net             0.0%    10    1.2   1.2   1.1   1.7   0.0
  7.|-- nyk-b3-link.telia.net               0.0%    10    1.2   1.8   1.2   7.5   1.9
  8.|-- nyk-bb4-link.telia.net              0.0%    10    2.0   1.9   1.7   2.1   0.0
  9.|-- prs-bb4-link.telia.net              0.0%    10   73.9  74.3  72.2  77.5   1.9
 10.|-- prs-b7-link.telia.net               0.0%    10   72.3  72.3  72.2  72.4   0.0
 11.|-- vultr-ic-312910-prs-b7.c.telia.net 20.0%    10   85.4  86.2  85.4  91.5   2.0
 12.|-- ???                                100.0    10    0.0   0.0   0.0   0.0   0.0
 13.|-- 108.61.209.127.vultr.com            0.0%    10   85.4  85.4  85.4  85.5   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 VersaWeb Las Vegas
Start: Mon Feb 26 13:47:17 2018
HOST: hostname                    Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                 0.0%    10   21.5  16.3  11.5  21.5   3.8
  2.|-- 147.75.36.2                             0.0%    10   21.4 212.5  15.2 1004. 382.3
  3.|-- 0.ae11.bbr1.ewr1.packet.net             0.0%    10    0.2   0.5   0.2   3.1   0.7
  4.|-- 10gigabitethernet1-2.core1.nyc6.he.net  0.0%    10    6.9   7.7   1.4  12.7   3.8
  5.|-- 100ge13-1.core1.nyc4.he.net             0.0%    10    5.7   4.3   1.4  11.1   2.9
  6.|-- 100ge16-1.core1.ash1.he.net             0.0%    10   14.2  19.3   6.5  92.1  25.8
  7.|-- 100ge14-2.core1.mci3.he.net             0.0%    10   32.8  32.7  30.6  39.4   2.6
  8.|-- 100ge12-1.core1.den1.he.net             0.0%    10   42.6  44.1  42.6  53.0   3.2
  9.|-- 100ge10-2.core1.slc1.he.net             0.0%    10   61.4  59.2  54.2  64.4   3.7
 10.|-- 10ge1-1.core1.las1.he.net               0.0%    10   70.3  67.6  62.9  70.5   3.3
 11.|-- 172.18.0.22                             0.0%    10   76.2  78.6  76.1  87.0   4.1
 12.|-- 192.228.109.121                         0.0%    10   76.3  77.9  76.2  86.9   3.5
 13.|-- 192.228.109.86                          0.0%    10   76.5  76.5  76.4  76.6   0.0
 14.|-- lg.las1.ultravps.net                    0.0%    10   76.2  76.2  76.1  76.3   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 VersaWeb Seattle
Start: Mon Feb 26 13:47:35 2018
HOST: hostname                       Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                    0.0%    10   11.9  25.2  11.9  97.4  25.6
  2.|-- 147.75.36.4                                0.0%    10   53.8 295.1  12.8 1136. 448.1
  3.|-- 0.ae22.bbr2.ewr1.packet.net                0.0%    10    0.2   0.3   0.2   1.0   0.0
  4.|-- packet.tbr2.wbr.nac.net                    0.0%    10    0.4   2.7   0.4  12.3   4.5
  5.|-- 0.lag16.tbr1.wbr.nac.net                   0.0%    10    0.4   0.3   0.2   0.5   0.0
  6.|-- 0.e1-3.tbr1.ewr.nac.net                    0.0%    10    1.0   1.0   0.9   1.0   0.0
  7.|-- te0-0-0-24.rcr21.ewr01.atlas.cogentco.com  0.0%    10    1.3   1.4   1.3   1.6   0.0
  8.|-- be3200.rcr24.jfk01.atlas.cogentco.com      0.0%    10    1.7   1.7   1.7   1.8   0.0
  9.|-- be2897.ccr42.jfk02.atlas.cogentco.com      0.0%    10    1.5   1.7   1.5   1.8   0.0
 10.|-- be2890.ccr22.cle04.atlas.cogentco.com      0.0%    10   18.2  18.5  18.2  18.7   0.0
 11.|-- be2718.ccr42.ord01.atlas.cogentco.com      0.0%    10   19.7  19.9  19.7  20.1   0.0
 12.|-- be2832.ccr22.mci01.atlas.cogentco.com      0.0%    10   47.9  48.1  47.8  48.3   0.0
 13.|-- be3036.ccr22.den01.atlas.cogentco.com      0.0%    10   59.9  59.7  59.6  59.9   0.0
 14.|-- be3038.ccr21.slc01.atlas.cogentco.com      0.0%    10   72.5  72.4  72.1  72.6   0.0
 15.|-- be2085.ccr21.sea02.atlas.cogentco.com      0.0%    10   66.9  66.8  66.7  66.9   0.0
 16.|-- 38.142.48.250                              0.0%    10   66.4  66.5  66.4  67.8   0.3
 17.|-- 76.164.234.1                               0.0%    10   66.5  66.5  66.5  66.6   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 OVH Canada
Start: Mon Feb 26 13:47:51 2018
HOST: hostname    Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                 0.0%    10    7.1   7.8   2.3  11.0   2.9
  2.|-- 2604:1380:2:79::        0.0%    10    2.0   5.0   2.0  11.4   2.6
  3.|-- 2604:1380:2:20::        0.0%    10    0.3   3.8   0.2  35.6  11.2
  4.|-- de-cix.nyc.ovh.com      0.0%    10    2.0   2.0   1.7   2.2   0.0
  5.|-- 2607:5300::1c6          0.0%    10   10.5  10.5  10.0  10.8   0.0
  6.|-- 2607:5300::1c3          0.0%    10    8.9   8.9   8.9   9.2   0.0
  7.|-- 2001:41d0:0:50::6:80d   0.0%    10    8.9   8.9   8.9   9.0   0.0
  8.|-- 2001:41d0:0:50::2:404d  0.0%    10    9.9   9.6   9.3  10.2   0.0
  9.|-- 2607:5300:60:44a5::1    0.0%    10    9.1   9.1   8.9   9.5   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io Frankfurt:
Start: Mon Feb 26 13:48:07 2018
HOST: hostname              Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                           0.0%    10   34.0  32.2  15.6  38.2   6.3
  2.|-- 147.75.36.2                       0.0%    10   12.6 144.7  12.4 710.5 272.6
  3.|-- 0.ae21.bbr2.ewr1.packet.net       0.0%    10    0.4   2.3   0.2  20.9   6.5
  4.|-- ae4-1.nyk10.core-backbone.com     0.0%    10    1.2   1.1   1.1   1.3   0.0
  5.|-- ae2-2025.fra30.core-backbone.com  0.0%    10   81.0  81.7  80.7  89.0   2.5
  6.|-- core-backbone.cdn77.com           0.0%    10   93.2  93.2  93.1  93.2   0.0
  7.|-- frankfurt-20.cdn77.com            0.0%    10   93.5  93.2  93.1  93.5   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io London:
Start: Mon Feb 26 13:48:23 2018
HOST: hostname               Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                            0.0%    10   14.1  14.6   6.6  21.9   4.9
  2.|-- 147.75.36.4                        0.0%    10   29.7 227.8  12.2 1102. 412.1
  3.|-- 0.ae12.bbr1.ewr1.packet.net        0.0%    10    0.1   3.0   0.1  28.0   8.8
  4.|-- 0.ae0.bbr2.ewr1.packet.net         0.0%    10    0.4   2.0   0.1  18.6   5.8
  5.|-- ae4-1.nyk10.core-backbone.com      0.0%    10    1.1   2.8   1.1  17.7   5.2
  6.|-- ae10-2092.lon10.core-backbone.com  0.0%    10   70.6  74.1  70.3 101.7   9.8
  7.|-- core-backbone.cdn77.com            0.0%    10   93.8  93.7  93.7  93.8   0.0
  8.|-- london-20.cdn77.com                0.0%    10   94.6  94.6  94.5  94.7   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io Paris:
Start: Mon Feb 26 13:48:40 2018
HOST: hostname                 Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                              0.0%    10   15.8  27.0  14.7  63.0  17.9
  2.|-- 147.75.36.2                          0.0%    10   13.6 154.9  12.3 1068. 328.5
  3.|-- 0.ae21.bbr2.ewr1.packet.net          0.0%    10    0.3   0.3   0.2   0.4   0.0
  4.|-- 0.xe-0-0-0.csr1.ewr2.packet.net      0.0%    10    2.3  26.1   2.3  94.5  33.1
  6.|-- 0.ae8.BR2.NYC4.ALTER.NET             0.0%    10    3.1   5.3   2.7  25.9   7.2
  7.|-- verizon.com.customer.alter.net       0.0%    10    2.2   2.3   2.2   2.5   0.0
  8.|-- ae-3.r24.nycmny01.us.bb.gin.ntt.net  0.0%    10    4.0   5.0   3.0  10.8   2.9
  9.|-- ae-9.r24.londen12.uk.bb.gin.ntt.net  0.0%    10   74.8  75.4  74.8  77.6   0.7
 10.|-- ae-1.r25.londen12.uk.bb.gin.ntt.net  0.0%    10   75.1  75.5  75.1  78.7   1.0
 11.|-- ae-2.r04.parsfr01.fr.bb.gin.ntt.net  0.0%    10   92.4  92.6  92.4  92.9   0.0
 12.|-- ae-5.r03.parsfr02.fr.bb.gin.ntt.net  0.0%    10   92.4  92.5  92.4  92.9   0.0
 13.|-- ae-8.r02.parsfr02.fr.bb.gin.ntt.net  0.0%    10   80.1  79.9  79.7  80.1   0.0
 14.|-- 81.25.206.50                         0.0%    10  101.3 101.3 101.2 101.4   0.0
 15.|-- unn-185-93-2-193.10gbps.io           0.0%    10  101.3 101.4 101.3 101.5   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io Singapore:
Start: Mon Feb 26 13:48:57 2018
HOST: hostname                       Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                                    0.0%    10   12.6  16.4   9.0  30.8   6.1
  2.|-- 147.75.36.2                                0.0%    10  741.2 232.0  13.7 842.1 349.5
  3.|-- 0.ae21.bbr2.ewr1.packet.net                0.0%    10    0.2   0.3   0.2   0.4   0.0
  4.|-- packet.tbr2.wbr.nac.net                    0.0%    10    0.3  26.9   0.3 147.8  56.0
  5.|-- 0.lag16.tbr1.wbr.nac.net                   0.0%    10    0.2   0.3   0.2   1.2   0.0
  6.|-- 0.e1-3.tbr1.ewr.nac.net                    0.0%    10    3.5   5.0   0.9  11.1   3.7
  7.|-- te0-0-0-24.rcr21.ewr01.atlas.cogentco.com  0.0%    10    1.3   1.3   1.2   1.4   0.0
  8.|-- be3199.rcr23.jfk01.atlas.cogentco.com      0.0%    10    1.8   2.0   1.8   3.4   0.5
  9.|-- be2896.ccr41.jfk02.atlas.cogentco.com      0.0%    10    2.0   1.7   1.5   2.0   0.0
 10.|-- be2806.ccr41.dca01.atlas.cogentco.com      0.0%    10    8.0   7.8   7.5   8.0   0.0
 11.|-- be2112.ccr41.atl01.atlas.cogentco.com      0.0%    10   19.7  19.9  19.7  20.1   0.0
 12.|-- be2687.ccr41.iah01.atlas.cogentco.com      0.0%    10   45.3  45.4  45.2  45.9   0.0
 13.|-- be2927.ccr21.elp01.atlas.cogentco.com      0.0%    10   73.7  73.5  73.3  73.8   0.0
 14.|-- be2929.ccr31.phx01.atlas.cogentco.com      0.0%    10   73.6  73.6  73.4  73.9   0.0
 15.|-- be2931.ccr41.lax01.atlas.cogentco.com      0.0%    10   73.6  73.6  73.5  73.7   0.0
 16.|-- be3271.ccr41.lax04.atlas.cogentco.com      0.0%    10   75.4  75.5  75.4  75.7   0.0
 17.|-- be3406.ccr21.sin01.atlas.cogentco.com      0.0%    10  246.7 246.8 246.6 247.9   0.3
 18.|-- 154.18.2.2                                 0.0%    10  245.0 245.0 245.0 245.3   0.0
 19.|-- hol14.homeoflim.com                        0.0%    10  251.7 251.7 251.5 252.3   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io Stockholm:
Start: Mon Feb 26 13:49:15 2018
HOST: hostname              Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                           0.0%    10   17.0  15.5  11.2  20.0   2.9
  2.|-- 147.75.36.4                       0.0%    10  469.0 136.8   9.3 574.4 211.0
  3.|-- 0.ae12.bbr1.ewr1.packet.net       0.0%    10    0.2   0.2   0.2   0.3   0.0
  4.|-- 0.ae0.bbr2.ewr1.packet.net        0.0%    10    2.1   0.4   0.1   2.1   0.6
  5.|-- ae4-1.nyk10.core-backbone.com     0.0%    10    1.1   1.2   1.1   1.4   0.0
  6.|-- ae1-2081.sth10.core-backbone.com  0.0%    10  101.3 101.3 101.3 101.4   0.0
  7.|-- core-backbone.cdn77.com           0.0%    10  101.1 101.1 101.1 101.1   0.0
  8.|-- stockholm-10.cdn77.com            0.0%    10  108.3 108.3 108.2 108.6   0.0

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io Moscow:
Start: Mon Feb 26 13:49:31 2018
HOST: hostname         Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                      0.0%    10   19.7  17.8  12.4  22.0   3.3
  2.|-- 147.75.36.2                  0.0%    10   20.2 133.2  13.0 654.3 246.2
  3.|-- 0.ae21.bbr2.ewr1.packet.net  0.0%    10    0.3   0.3   0.2   0.4   0.0
  4.|-- de-cix.nyc.retn.net          0.0%    10    1.0   1.1   1.0   1.2   0.0
  5.|-- ae1-3.RT.MR.MSK.RU.retn.net  0.0%    10  114.3 115.8 114.3 120.7   2.4
  6.|-- 87.245.204.20                0.0%    10  113.6 113.6 113.3 115.2   0.5

----------------------------------------------
mtr --report -w 200 --report-cycles=10 10gbps.io Miami:
Start: Mon Feb 26 13:49:46 2018
HOST: hostname                 Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gateway                              0.0%    10   78.8  27.2  11.5  78.8  25.4
  2.|-- 147.75.36.4                          0.0%    10   13.0 140.0  11.4 1056. 325.9
  3.|-- 0.ae22.bbr2.ewr1.packet.net          0.0%    10    0.4   0.5   0.1   3.2   0.8
  4.|-- 0.xe-0-0-0.csr1.ewr2.packet.net      0.0%    10    3.4  12.0   2.8  62.0  17.8
  6.|-- 0.ae8.BR2.NYC4.ALTER.NET             0.0%    10    2.7   2.8   2.7   3.2   0.0
  7.|-- verizon.com.customer.alter.net       0.0%    10    2.7   2.4   2.2   2.7   0.0
  8.|-- ae-3.r25.nycmny01.us.bb.gin.ntt.net  0.0%    10    2.3   3.1   2.3   8.1   1.6
  9.|-- ae-9.r22.asbnva02.us.bb.gin.ntt.net  0.0%    10    9.5  10.5   8.7  14.2   2.0
 10.|-- ae-0.r23.asbnva02.us.bb.gin.ntt.net  0.0%    10    8.7   8.6   8.5   8.7   0.0
 11.|-- ae-1.r20.miamfl02.us.bb.gin.ntt.net  0.0%    10   38.7  38.8  38.7  39.3   0.0
 12.|-- ae-1.r04.miamfl02.us.bb.gin.ntt.net  0.0%    10   38.8  38.9  38.7  39.2   0.0
 13.|-- 165.254.21.26                        0.0%    10   38.0  38.0  37.9  38.0   0.0
 14.|-- unn-195-181-162-195.cdn77.com        0.0%    10   56.2  56.0  55.9  56.2   0.0

    Code (Text):
    -------------------------------------------
Check system entropy pool availability
-------------------------------------------

entropy_avail: 4067
entropy_avail: 4067
entropy_avail: 4067
entropy_avail: 4067

    Code (Text):
    -------------------------------------------
OpenSSL System Benchmark
-------------------------------------------

OpenSSL 1.0.2k-fips  26 Jan 2017
-------------------------------------------
openssl speed -multi 48 rsa4096 rsa2048 ecdsap256 sha256 sha1 md5 rc4 aes-256-cbc aes-128-cbc
:0.014341:0.000225 from 5
613.33:112702805.33 from 17
6974464.00:151745877.33:154208938.67 from 23
OpenSSL 1.0.2k-fips  26 Jan 2017
built on: reproducible build, date unspecified
options:bn(64,64) md2(int) rc4(8x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
md5            1713991.89k  5321480.32k 12165950.72k 18738554.88k 21742185.13k
sha1           2632574.01k  9114193.66k 24777751.21k 42884053.67k 51963423.40k
rc4           14219831.48k 17600669.03k 18195224.17k 18252467.96k 18290272.94k
aes-128 cbc    2913189.89k  3122197.89k  3215172.01k  7280857.43k  7405236.09k
aes-256 cbc    2123871.85k  2238767.57k  2278030.00k  5348877.99k  5405005.17k
sha256         5504880.33k 15290409.75k 34443615.15k 51130241.37k 59325699.41k
                  sign    verify    sign/s verify/s
rsa 2048 bits 0.000042s 0.000001s  24069.5 796612.1
rsa 4096 bits 0.000299s 0.000005s   3346.8 215510.7
                              sign    verify    sign/s verify/s
 256 bit ecdsa (nistp256)   0.0000s   0.0000s 553741.1 231184.8
-------------------------------------------
openssl speed -evp aes256 -multi 48
16:64:256:1024:8192 from 31
OpenSSL 1.0.2k-fips  26 Jan 2017
built on: reproducible build, date unspecified
options:bn(64,64) md2(int) rc4(8x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
evp           23725037.36k 29072785.82k 31162345.90k 31983885.65k 32225599.49k
-------------------------------------------
openssl speed -evp aes128 -multi 48
OpenSSL 1.0.2k-fips  26 Jan 2017
built on: reproducible build, date unspecified
options:bn(64,64) md2(int) rc4(8x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
evp           28439367.64k 39542990.27k 42292041.47k 43768720.73k 44169289.73k

    Code (Text):
    -------------------------------------------
Running mysqlslap
-------------------------------------------

mysqlslap --auto-generate-sql --auto-generate-sql-add-autoincrement --auto-generate-sql-secondary-indexes=5 --number-int-cols=5 --number-char-cols=5 --number-of-queries=25000 --auto-generate-sql-unique-query-number=40 --auto-generate-sql-unique-write-number=40 --auto-generate-sql-write-number=1000 --concurrency=64 --iterations=10 --engine=myisam
Benchmark
        Running for engine myisam
        Average number of seconds to run all queries: 0.906 seconds
        Minimum number of seconds to run all queries: 0.889 seconds
        Maximum number of seconds to run all queries: 0.914 seconds
        Number of clients running queries: 64
        Average number of queries per client: 390

Dropping the database is potentially a very bad thing to do.
Any data stored in the database will be destroyed.

Do you really want to drop the 'test' database [y/N] Database "test" dropped

    Code (Text):
    -------------------------------------------
System PHP Info
-------------------------------------------

CPU: 48 x AMD EPYC 7401P 24-Core Processor
              total        used        free      shared  buff/cache   available
Mem:          64146        3794       59507          14         844       59658
Low:          64146        4638       59507
High:             0           0           0
Swap:          1949           0        1949
----------------------------------------------
PHP 5.6.33 (cli) (built: Feb 26 2018 08:24:24)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
    with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies
----------------------------------------------
Configuration File (php.ini) Path: /usr/local/lib
Loaded Configuration File:         /usr/local/lib/php.ini
Scan for additional .ini files in: /etc/centminmod/php.d
Additional .ini files parsed:      /etc/centminmod/php.d/a_customphp.ini,
/etc/centminmod/php.d/curlcainfo.ini,
/etc/centminmod/php.d/geoip.ini,
/etc/centminmod/php.d/igbinary.ini,
/etc/centminmod/php.d/imagick.ini,
/etc/centminmod/php.d/mailparse.ini,
/etc/centminmod/php.d/memcache.ini,
/etc/centminmod/php.d/memcached.ini,
/etc/centminmod/php.d/redis.ini,
/etc/centminmod/php.d/zendopcache.ini

----------------------------------------------
[PHP Modules]
bcmath
bz2
calendar
Core
ctype
curl
date
dom
enchant
ereg
exif
filter
ftp
gd
geoip
gettext
gmp
hash
iconv
igbinary
imagick
imap
intl
json
ldap
libxml
mailparse
mbstring
mcrypt
memcache
memcached
mhash
mysql
mysqli
mysqlnd
openssl
pcntl
pcre
PDO
pdo_mysql
pdo_sqlite
Phar
posix
pspell
readline
redis
Reflection
session
shmop
SimpleXML
snmp
soap
sockets
SPL
sqlite3
standard
sysvmsg
sysvsem
sysvshm
tidy
tokenizer
xml
xmlreader
xmlrpc
xmlwriter
xsl
Zend OPcache
zip
zlib

[Zend Modules]
Zend OPcache

----------------------------------------------
-------------------------------------------
Run PHP test Zend/bench.php
-------------------------------------------

simple             0.104
simplecall         0.101
simpleucall        0.107
simpleudcall       0.111
mandel             0.258
mandel2            0.325
ackermann(7)       0.115
ary(50000)         0.015
ary2(50000)        0.014
ary3(2000)         0.133
fibo(30)           0.384
hash1(50000)       0.027
hash2(500)         0.026
heapsort(20000)    0.078
matrix(20)         0.091
nestedloop(12)     0.191
sieve(30)          0.079
strcat(200000)     0.011
------------------------
Total              2.170
real: 2.23s user: 2.18s sys: 0.03s cpu: 99% maxmem: 43828 KB cswaits: 8

simple             0.103
simplecall         0.102
simpleucall        0.106
simpleudcall       0.109
mandel             0.256
mandel2            0.303
ackermann(7)       0.115
ary(50000)         0.015
ary2(50000)        0.014
ary3(2000)         0.133
fibo(30)           0.383
hash1(50000)       0.027
hash2(500)         0.026
heapsort(20000)    0.078
matrix(20)         0.087
nestedloop(12)     0.189
sieve(30)          0.079
strcat(200000)     0.011
------------------------
Total              2.137
real: 2.17s user: 2.15s sys: 0.01s cpu: 99% maxmem: 44000 KB cswaits: 3

simple             0.102
simplecall         0.101
simpleucall        0.107
simpleudcall       0.109
mandel             0.258
mandel2            0.322
ackermann(7)       0.115
ary(50000)         0.015
ary2(50000)        0.014
ary3(2000)         0.132
fibo(30)           0.388
hash1(50000)       0.027
hash2(500)         0.027
heapsort(20000)    0.078
matrix(20)         0.087
nestedloop(12)     0.191
sieve(30)          0.079
strcat(200000)     0.011
------------------------
Total              2.164
real: 2.20s user: 2.18s sys: 0.02s cpu: 99% maxmem: 43992 KB cswaits: 3

bench.php results from 3 runs
2.170
2.137
2.164

bench.php avg: 2.1570
Avg: real: 2.20s user: 2.17s sys: 0.02s cpu: 99.00% maxmem: 43940.00KB cswaits: 4.67
created results log at /home/phpbench_logs/bench_260218-133419.log
server PHP info log at /home/phpbench_logs/bench_phpinfo_260218-133419.log

-------------------------------------------
Run PHP test Zend/micro_bench.php
-------------------------------------------

empty_loop         0.151
func()             0.345    0.194
undef_func()       0.344    0.193
int_func()         0.284    0.133
$x = self::$x      0.253    0.102
self::$x = 0       0.233    0.083
isset(self::$x)    0.243    0.093
empty(self::$x)    0.268    0.118
$x = Foo::$x       0.243    0.092
Foo::$x = 0        0.225    0.074
isset(Foo::$x)     0.214    0.063
empty(Foo::$x)     0.226    0.075
self::f()          0.385    0.235
Foo::f()           0.326    0.175
$x = $this->x      0.221    0.070
$this->x = 0       0.282    0.131
$this->x += 2      0.233    0.082
++$this->x         0.227    0.076
--$this->x         0.234    0.083
$this->x++         0.240    0.089
$this->x--         0.266    0.115
isset($this->x)    0.325    0.174
empty($this->x)    0.246    0.095
$this->f()         0.424    0.273
$x = Foo::TEST     0.247    0.096
new Foo()          0.660    0.509
$x = TEST          0.193    0.042
$x = $_GET         0.208    0.057
$x = $GLOBALS['v'] 0.283    0.132
$x = $hash['v']    0.244    0.093
$x = $str[0]       0.320    0.169
$x = $a ?: null    0.219    0.069
$x = $f ?: tmp     0.302    0.151
$x = $f ? $f : $a  0.222    0.071
$x = $f ? $f : tmp 0.304    0.153
------------------------
Total              9.641
real: 9.69s user: 9.66s sys: 0.03s cpu: 99% maxmem: 32044 KB cswaits: 4

empty_loop         0.150
func()             0.330    0.180
undef_func()       0.325    0.175
int_func()         0.280    0.130
$x = self::$x      0.256    0.105
self::$x = 0       0.277    0.127
isset(self::$x)    0.255    0.104
empty(self::$x)    0.262    0.112
$x = Foo::$x       0.245    0.095
Foo::$x = 0        0.245    0.095
isset(Foo::$x)     0.228    0.078
empty(Foo::$x)     0.232    0.082
self::f()          0.370    0.220
Foo::f()           0.331    0.181
$x = $this->x      0.220    0.070
$this->x = 0       0.279    0.129
$this->x += 2      0.232    0.082
++$this->x         0.247    0.097
--$this->x         0.239    0.089
$this->x++         0.254    0.103
$this->x--         0.271    0.121
isset($this->x)    0.335    0.185
empty($this->x)    0.242    0.092
$this->f()         0.418    0.268
$x = Foo::TEST     0.243    0.093
new Foo()          0.654    0.504
$x = TEST          0.208    0.057
$x = $_GET         0.207    0.056
$x = $GLOBALS['v'] 0.284    0.134
$x = $hash['v']    0.244    0.094
$x = $str[0]       0.318    0.168
$x = $a ?: null    0.221    0.071
$x = $f ?: tmp     0.301    0.151
$x = $f ? $f : $a  0.220    0.070
$x = $f ? $f : tmp 0.303    0.153
------------------------
Total              9.725
real: 9.76s user: 9.75s sys: 0.01s cpu: 99% maxmem: 31896 KB cswaits: 3

empty_loop         0.151
func()             0.325    0.174
undef_func()       0.330    0.179
int_func()         0.299    0.148
$x = self::$x      0.235    0.084
self::$x = 0       0.257    0.106
isset(self::$x)    0.234    0.083
empty(self::$x)    0.250    0.100
$x = Foo::$x       0.225    0.074
Foo::$x = 0        0.227    0.076
isset(Foo::$x)     0.215    0.064
empty(Foo::$x)     0.227    0.076
self::f()          0.369    0.219
Foo::f()           0.326    0.175
$x = $this->x      0.221    0.071
$this->x = 0       0.282    0.132
$this->x += 2      0.232    0.081
++$this->x         0.234    0.083
--$this->x         0.234    0.083
$this->x++         0.253    0.102
$this->x--         0.268    0.117
isset($this->x)    0.333    0.182
empty($this->x)    0.252    0.101
$this->f()         0.422    0.272
$x = Foo::TEST     0.243    0.093
new Foo()          0.666    0.515
$x = TEST          0.209    0.058
$x = $_GET         0.205    0.054
$x = $GLOBALS['v'] 0.283    0.133
$x = $hash['v']    0.242    0.092
$x = $str[0]       0.317    0.166
$x = $a ?: null    0.219    0.069
$x = $f ?: tmp     0.298    0.148
$x = $f ? $f : $a  0.236    0.085
$x = $f ? $f : tmp 0.303    0.153
------------------------
Total              9.623
real: 9.66s user: 9.65s sys: 0.00s cpu: 99% maxmem: 31864 KB cswaits: 4

micro_bench.php results from 3 runs
9.641
9.725
9.623

micro_bench.php avg: 9.6630
Avg: real: 9.70s user: 9.69s sys: 0.01s cpu: 99.00% maxmem: 31934.67KB cswaits: 3.67
created results log at /home/phpbench_logs/bench_micro_260218-133419.log
server PHP info log at /home/phpbench_logs/bench_phpinfo_260218-133419.log

    Code (Text):
    ========================================================================
   BYTE UNIX Benchmarks (Version 5.1.3)

   System: hostname: GNU/Linux
   OS: GNU/Linux -- 4.15.5-1.el7.elrepo.x86_64 -- #1 SMP Thu Feb 22 22:52:54 EST 2018
   Machine: x86_64 (x86_64)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 1: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 2: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 3: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 4: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 5: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 6: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 7: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 8: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 9: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 10: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 11: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 12: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 13: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 14: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 15: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 16: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 17: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 18: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 19: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 20: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 21: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 22: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 23: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 24: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 25: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 26: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 27: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 28: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 29: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 30: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 31: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 32: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 33: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 34: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 35: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 36: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 37: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 38: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 39: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 40: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 41: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 42: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 43: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 44: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 45: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 46: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   CPU 47: AMD EPYC 7401P 24-Core Processor (3992.4 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, AMD virtualization, SYSCALL/SYSRET
   13:54:57 up 53 min,  3 users,  load average: 14.95, 17.56, 8.01; runlevel 3

    Code (Text):
    ------------------------------------------------------------------------
Benchmark Run: Mon Feb 26 2018 13:54:57 - 14:19:09
48 CPUs in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       35135751.8 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4238.7 MWIPS (11.3 s, 7 samples)
Execl Throughput                               5577.1 lps   (30.0 s, 2 samples)
Pipe Throughput                             1604579.9 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 239817.5 lps   (10.0 s, 7 samples)
Process Creation                               9672.3 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   7283.8 lpm   (60.0 s, 2 samples)
Shell Scripts (16 concurrent)                  2403.6 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   3783.5 lpm   (60.0 s, 2 samples)
System Call Overhead                        2126181.0 lps   (10.0 s, 7 samples)

System Benchmarks Partial Index              BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   35135751.8   3010.8
Double-Precision Whetstone                       55.0       4238.7    770.7
Execl Throughput                                 43.0       5577.1   1297.0
Pipe Throughput                               12440.0    1604579.9   1289.9
Pipe-based Context Switching                   4000.0     239817.5    599.5
Process Creation                                126.0       9672.3    767.6
Shell Scripts (1 concurrent)                     42.4       7283.8   1717.9
Shell Scripts (16 concurrent)                     ---       2403.6      ---
Shell Scripts (8 concurrent)                      6.0       3783.5   6305.8
System Call Overhead                          15000.0    2126181.0   1417.5
                                                                   ========
System Benchmarks Index Score (Partial Only)                         1444.8

    Code (Text):
    ------------------------------------------------------------------------
Benchmark Run: Mon Feb 26 2018 14:19:09 - 14:43:15
48 CPUs in system; running 48 parallel copies of tests

Dhrystone 2 using register variables      885066254.8 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                   166998.3 MWIPS (9.9 s, 7 samples)
Execl Throughput                              26091.6 lps   (29.8 s, 2 samples)
Pipe Throughput                            43195823.9 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                6086783.7 lps   (10.0 s, 7 samples)
Process Creation                              80514.4 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                 108057.5 lpm   (60.0 s, 2 samples)
Shell Scripts (16 concurrent)                  7447.9 lpm   (60.1 s, 2 samples)
Shell Scripts (8 concurrent)                  14832.3 lpm   (60.1 s, 2 samples)
System Call Overhead                        8000223.5 lps   (10.0 s, 7 samples)

System Benchmarks Partial Index              BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0  885066254.8  75841.2
Double-Precision Whetstone                       55.0     166998.3  30363.3
Execl Throughput                                 43.0      26091.6   6067.8
Pipe Throughput                               12440.0   43195823.9  34723.3
Pipe-based Context Switching                   4000.0    6086783.7  15217.0
Process Creation                                126.0      80514.4   6390.0
Shell Scripts (1 concurrent)                     42.4     108057.5  25485.2
Shell Scripts (16 concurrent)                     ---       7447.9      ---
Shell Scripts (8 concurrent)                      6.0      14832.3  24720.5
System Call Overhead                          15000.0    8000223.5   5333.5
                                                                   ========
System Benchmarks Index Score (Partial Only)                        17557.2

    Code (Text):
    -------------------------------------------
centminmodbench.sh completed
-------------------------------------------

centminmodbench.sh Total Run Time: 4118.678124307 seconds
     
    Last edited: Feb 27, 2018
  6. Feb 27, 2018 #6
    eva2000

    eva2000 Administrator Staff Member

    53,280
    12,117
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,655
    Local Time:
    2:23 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    Packet.net AMD EPYC 7401P + 4.15.6-1 Kernel Revisited



    Seeing as there were some improvements using ELrepo's 4.15.5-1 Linux Kernel, I did a fresh reinstall of CentOS 7.4 OS and updated to latest ELrepo mainline Linux Kernel. Seems in between the reinstall of OS, ELrepo released a newer 4.15.6-1 Linux Kernel so updated to that and did a server reboot before doing a new Centmin Mod 123.09beta01 run :)

    Code (Text):
    uname -r
4.15.6-1.el7.elrepo.x86_64

    Centmin Mod 123.09beta01 install command
    Code (Text):
    yum -y update; curl -O https://centminmod.com/betainstaller.sh && chmod 0700 betainstaller.sh && bash betainstaller.sh

    AMD EPYC 7401P Centmin Mod 123.09beta01 Install Times with 4.15.6-1 Linux Kernel which looks much improved due YUM related downloads and also PHP-FPM compile times.
    Code (Text):
    ---------------------------------------------------------------------------
Total Curl Installer YUM or DNF Time: 110.3599 seconds
Total YUM Time: 36.359334737 seconds
Total YUM or DNF + Source Download Time: 57.1042
Total Nginx First Time Install Time: 196.2899
Total PHP First Time Install Time: 124.1210
Download Zip From Github Time: 1.2566
Total Time Other eg. source compiles: 280.8104
Total Centmin Mod Install Time: 658.3255
---------------------------------------------------------------------------
Total Install Time (curl yum + cm install + zip download): 769.9420 seconds
---------------------------------------------------------------------------

    This compared to previous AMD EPYC 7401P Centmin Mod 123.09beta01 Install Times
    Code (Text):
    ---------------------------------------------------------------------------
Total Curl Installer YUM or DNF Time: 340.5943 seconds
Total YUM Time: 60.142379961 seconds
Total YUM or DNF + Source Download Time: 86.9888
Total Nginx First Time Install Time: 182.0339
Total PHP First Time Install Time: 183.7518
Download Zip From Github Time: 1.6182
Total Time Other eg. source compiles: 418.1967
Total Centmin Mod Install Time: 870.9712
---------------------------------------------------------------------------
Total Install Time (curl yum + cm install + zip download): 1213.1837 seconds
---------------------------------------------------------------------------

    compared to Intel Xeon E3-1578Lv5 Skylake Centmin Mod 123.09beta01 Install Times
    Code (Text):
    ---------------------------------------------------------------------------
Total Curl Installer YUM or DNF Time: 91.2912 seconds
Total YUM Time: 64.039117358 seconds
Total YUM or DNF + Source Download Time: 87.3015
Total Nginx First Time Install Time: 135.2867
Total PHP First Time Install Time: 141.0516
Download Zip From Github Time: 1.8695
Total Time Other eg. source compiles: 264.2750
Total Centmin Mod Install Time: 627.9147
---------------------------------------------------------------------------
Total Install Time (curl yum + cm install + zip download): 721.0754 seconds
---------------------------------------------------------------------------


    GCC 8.0.1 Compile & RPM Build Times



    Centmin Mod LEMP Stack's Nginx web server supports multiple GCC/Clang version compilers from GCC 4.8, 5.3, 6.3, 7.2 and 8.x and Clang 3.4, 4, 5, and 6. Currently re-testing custom GCC 8.0.1 compilations and RPM builds and it seems there are huge improvements with 4.15.6-1 kernel in place dropping binutil 2.30 + GCC 8.0.1 compile and RPM build times from ~2684 seconds to ~1939 seconds - ~38% faster so just ~32 minutes !

    Just a reminder these are Profile Guided Optimization based GCC builds so basically need to recompile twice for make. PGO boosted PHP 7 performance as would PGO built GCC 8.0.1 for resulting built binaries that utilise GCC 8.0.1 PGO enabled compiler :)
    Code (Text):
    RPMs Built
/svr-setup/binutils-gcc8-2.30-1.el7.x86_64.rpm
/svr-setup/gcc8-pgo-8.0-1.el7.x86_64.rpm

ls -lah /svr-setup | egrep 'gcc8-pgo-8.0|binutils-gcc8-2.30'
-rw-r--r--  1 root      root       5.2M Feb 26 15:40 binutils-gcc8-2.30-1.el7.x86_64.rpm
-rw-r--r--  1 root      root        39M Feb 26 16:10 gcc8-pgo-8.0-1.el7.x86_64.rpm

Installed Packages
Name        : binutils-gcc8
Arch        : x86_64
Version     : 2.30
Release     : 1.el7
Size        : 48 M
Repo        : installed
Summary     : binutils-gcc8 for centminmod.com LEMP stack installs
URL         : https://centminmod.com
License     : unknown
Description : binutils-gcc8 for centminmod.com LEMP stacks

* Mon Feb 26 2018 George Liu <centminmod.com> 2.30
- Binutils 2.30 for centminmod.com LEMP stack installs

    Code (Text):
    Installed Packages
Name        : gcc8-pgo
Arch        : x86_64
Version     : 8.0
Release     : 1.el7
Size        : 287 M
Repo        : installed
Summary     : gcc8-pgo for centminmod.com LEMP stack installs
URL         : https://centminmod.com
License     : unknown
Description : gcc8-pgo for centminmod.com LEMP stacks

* Mon Feb 26 2018 George Liu <centminmod.com> 8
- GCC 8 for centminmod.com LEMP stack installs

*************************************************
Compile GCC Completed
log: /root/centminlogs/tools-gcc-install-pgo_260218-153809.log
*************************************************

Total Binutils + GCC Install Time: 1939.402776571 seconds


    CPU load average didn't go beyond 43.96 according to sar stats
    Code (Text):
    03:35:01 PM         0       655      0.64      1.82      1.37         0
03:40:01 PM         2       670      4.25      1.83      1.38         0
03:45:01 PM         1       673      5.15      4.89      2.80         0
03:50:01 PM        52       767     43.96     18.94      8.48         0
03:55:01 PM         1       662      2.99     12.93      9.42         0
04:00:01 PM        53       771     33.53     15.89     10.89         0
04:05:01 PM         3       674     16.43     14.97     11.63         0
04:10:01 PM         1       658      1.39      6.78      9.03         0
04:15:01 PM         0       649      0.08      2.52      6.55         0

04:15:01 PM   runq-sz  plist-sz   ldavg-1   ldavg-5  ldavg-15   blocked
04:20:01 PM         0       651      0.00      0.91      4.73         0
04:25:01 PM         0       650      0.09      0.41      3.44         0
04:30:01 PM         0       651      0.00      0.16      2.49         0
04:35:02 PM         0       651      0.08      0.08      1.80         0
04:40:01 PM         0       649      0.00      0.02      1.29         0
04:45:01 PM         0       650      0.01      0.01      0.93         0
04:50:01 PM         0       649      0.00      0.00      0.66         0
04:55:01 PM         0       654      0.16      0.04      0.49         0
Average:            3       604      3.94      3.11      2.37         0

    sar memory stats for same duration
    Code (Text):
    03:35:01 PM  62480068   3206172      4.88      3208     87172  19021956     28.10   1915948     48452         8
03:40:01 PM  59514596   6171644      9.40     39828   1461940  20560624     30.38   3984236    877228     99612
03:45:01 PM  57476376   8209864     12.50     94796   4544748  19185836     28.35   3981352   2638072    124488
03:50:01 PM  50222836  15463404     23.54     96960   4959464  26218668     38.74  10941888   2929068      2788
03:55:01 PM  56154188   9532052     14.51     97628   5092688  19995360     29.54   4950652   3026384       104
04:00:01 PM  48303432  17382808     26.46    101408   6661720  26548720     39.23  11534824   4169488    158160
04:05:01 PM  52561692  13124548     19.98    105004   9100884  19340412     28.57   5776880   5631816     20644
04:10:01 PM  49631492  16054748     24.44    107720  11475204  20453276     30.22   8248624   6005844         0
04:15:01 PM  50673364  15012876     22.86    111284  11181188  19103652     28.23   6964604   6272264        32

04:15:01 PM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact   kbdirty
04:20:01 PM  50670508  15015732     22.86    111500  11186772  19107328     28.23   6969392   6273624        32
04:25:01 PM  50443288  15242952     23.21    191876  11254744  19105032     28.23   7086220   6305292        24
04:30:01 PM  50460636  15225604     23.18    192000  11254796  19082940     28.19   7086472   6305348        20
04:35:02 PM  50478676  15207564     23.15    192152  11254784  19072260     28.18   7086352   6305320         8
04:40:01 PM  50482276  15203964     23.15    192292  11254828  19060084     28.16   7086780   6305332        20
04:45:01 PM  50483616  15202624     23.14    192412  11254808  19065324     28.17   7086680   6305312        36
04:50:01 PM  50481012  15205228     23.15    192608  11255044  19061636     28.16   7090352   6302084         8
04:55:01 PM  50463996  15222244     23.17    193336  11259452  19092144     28.21   7107500   6306344        80
05:00:01 PM  50469136  15217104     23.17    193444  11259468  19091884     28.21   7110648   6303540         8
Average:     57059893   8626347     13.13    126564   5686601   9141103     13.51   3164933   3884380     23466


    Packet.net Bill



    Troubleshooting and retesting on AMD EPYC 7401P server, prolonged the server running time so the Packet.net bill for above testing was higher than I was expecting. But the beauty of Packet.net is the hourly billing as there is no way could afford to commit for a full month for the above 3 server configurations ! I look forward to more AMD EPYC testing to properly tune Centmin Mod for AMD EPYC cpus and not just Intel cpus. All the above testing data will be thoroughly analysed and incorporated into Centmin Mod developments :)

    If you like this benchmark review and find it useful, please consider a small donation to support Centmin Mod ;)

    type1e-billing-02.png
     
    Last edited: Feb 27, 2018
  7. Feb 27, 2018 #7
    eva2000

    eva2000 Administrator Staff Member

    53,280
    12,117
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,655
    Local Time:
    2:23 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    Packet.net AMD EPYC 7401P Hardware Details



    Lets dig into the hardware specifics.

    Seems the server is a Dell PowerEdge R6415 with 01/05/2018 dated bios but version number is newer 1.1.3 ? with 8x8GB Micron ECC DDR4-2666Mhz memory and supports a max of 16x memory slots for max 2TB of memory.

    According to Dell PowerEdge R6415 downloads page, there is a newer 1.1.3 February 13, 2018 for code optimization and improved stability & performance than the existing Jan 5th, 2018 bios. But the Jan 5th 2018 bios is marked version 1.1.3 ? There's also a 1.09 bios which addresses CVE-2017-5715 Spectre vulnerability which hasn't been applied yet. So performance might change between existing bios and 1.09 and 1.1.3 bios updates.

    Dell EMC Server PowerEdge BIOS R6415/R7415 Version 1.1.3 Driver Details | Dell US

    and 1.09 January 22, 2018 bios Dell EMC Server PowerEdge BIOS R6415/R7415 Version 1.0.9 Driver Details | Dell US

    dmidecode details

    Code (Text):
    # dmidecode 3.0
Getting SMBIOS data from sysfs.
SMBIOS 3.0.0 present.

Handle 0x0000, DMI type 0, 26 bytes
BIOS Information
        Vendor: Dell Inc.
        Version: 1.1.3
        Release Date: 01/05/2018
        Address: 0xF0000
        Runtime Size: 64 kB
        ROM Size: 16384 kB

    Code (Text):
    Handle 0x0100, DMI type 1, 27 bytes
System Information
        Manufacturer: Dell Inc.
        Product Name: PowerEdge R6415
        Version: Not Specified
        Serial Number: J1X*****
        UUID: 4C4C4544-0031-5810-8035-CAC04F434D32
        Wake-up Type: Power Switch
        SKU Number: SKU=NotProvided;ModelName=PowerEdge R6415
        Family: PowerEdge

    AMD EPYC 7401P
    Code (Text):
    andle 0x0400, DMI type 4, 48 bytes
Processor Information
        Socket Designation: CPU1
        Type: Central Processor
        Family: <OUT OF SPEC>
        Manufacturer: AMD
        ID: 12 0F 80 00 FF FB 8B 17
        Version: AMD EPYC 7401P 24-Core Processor   
        Voltage: 0.8 V
        External Clock: 9600 MHz
        Max Speed: 3000 MHz
        Current Speed: 2000 MHz
        Status: Populated, Enabled
        Upgrade: <OUT OF SPEC>
        L1 Cache Handle: 0x0700
        L2 Cache Handle: 0x0701
        L3 Cache Handle: 0x0702
        Serial Number: Not Specified
        Asset Tag: Not Specified
        Part Number: Not Specified
        Core Count: 24
        Core Enabled: 24
        Thread Count: 48
        Characteristics:
                64-bit capable
                Multi-Core
                Hardware Thread
                Execute Protection
                Enhanced Virtualization

    8x8GB Micron ECC DDR4-2666Mhz rated memory clocked at DDR4-2400Mhz with part number 9ASF1G72PZ-2G6D1
    Code (Text):
    Handle 0x1000, DMI type 16, 23 bytes
Physical Memory Array
        Location: System Board Or Motherboard
        Use: System Memory
        Error Correction Type: Multi-bit ECC
        Maximum Capacity: 2 TB
        Error Information Handle: Not Provided
        Number Of Devices: 16

Handle 0x1100, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 8192 MB
        Form Factor: DIMM
        Set: 1
        Locator: A1
        Bank Locator: Not Specified
        Type: DDR4
        Type Detail: Synchronous Registered (Buffered)
        Speed: 2666 MHz
        Manufacturer: 802C8632802C
        Serial Number: 191DE773
        Asset Tag: 0F1741A0
        Part Number: 9ASF1G72PZ-2G6D1
        Rank: 1
        Configured Clock Speed: 2400 MHz
        Minimum Voltage: 1.2 V
        Maximum Voltage: 1.2 V
        Configured Voltage: 1.2 V

Handle 0x1101, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 8192 MB
        Form Factor: DIMM
        Set: 1
        Locator: A2
        Bank Locator: Not Specified
        Type: DDR4
        Type Detail: Synchronous Registered (Buffered)
        Speed: 2666 MHz
        Manufacturer: 802C8632802C
        Serial Number: 191DEABC
        Asset Tag: 0F1741A0
        Part Number: 9ASF1G72PZ-2G6D1
        Rank: 1
        Configured Clock Speed: 2400 MHz
        Minimum Voltage: 1.2 V
        Maximum Voltage: 1.2 V
        Configured Voltage: 1.2 V

Handle 0x1102, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 8192 MB
        Form Factor: DIMM
        Set: 1
        Locator: A3
        Bank Locator: Not Specified
        Type: DDR4
        Type Detail: Synchronous Registered (Buffered)
        Speed: 2666 MHz
        Manufacturer: 802C8632802C
        Serial Number: 191DDFBD
        Asset Tag: 0F1741A0
        Part Number: 9ASF1G72PZ-2G6D1
        Rank: 1
        Configured Clock Speed: 2400 MHz
        Minimum Voltage: 1.2 V
        Maximum Voltage: 1.2 V
        Configured Voltage: 1.2 V

Handle 0x1103, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 8192 MB
        Form Factor: DIMM
        Set: 1
        Locator: A4
        Bank Locator: Not Specified
        Type: DDR4
        Type Detail: Synchronous Registered (Buffered)
        Speed: 2666 MHz
        Manufacturer: 802C8632802C
        Serial Number: 191DE88E
        Asset Tag: 0F1741A0
        Part Number: 9ASF1G72PZ-2G6D1
        Rank: 1
        Configured Clock Speed: 2400 MHz
        Minimum Voltage: 1.2 V
        Maximum Voltage: 1.2 V
        Configured Voltage: 1.2 V

Handle 0x1104, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 8192 MB
        Form Factor: DIMM
        Set: 1
        Locator: A5
        Bank Locator: Not Specified
        Type: DDR4
        Type Detail: Synchronous Registered (Buffered)
        Speed: 2666 MHz
        Manufacturer: 802C8632802C
        Serial Number: 191DE7E8
        Asset Tag: 0F1741A0
        Part Number: 9ASF1G72PZ-2G6D1
        Rank: 1
        Configured Clock Speed: 2400 MHz
        Minimum Voltage: 1.2 V
        Maximum Voltage: 1.2 V
        Configured Voltage: 1.2 V

Handle 0x1105, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 8192 MB
        Form Factor: DIMM
        Set: 1
        Locator: A6
        Bank Locator: Not Specified
        Type: DDR4
        Type Detail: Synchronous Registered (Buffered)
        Speed: 2666 MHz
        Manufacturer: 802C8632802C
        Serial Number: 191DE7F5
        Asset Tag: 0F1741A0
        Part Number: 9ASF1G72PZ-2G6D1
        Rank: 1
        Configured Clock Speed: 2400 MHz
        Minimum Voltage: 1.2 V
        Maximum Voltage: 1.2 V
        Configured Voltage: 1.2 V

Handle 0x1106, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 8192 MB
        Form Factor: DIMM
        Set: 1
        Locator: A7
        Bank Locator: Not Specified
        Type: DDR4
        Type Detail: Synchronous Registered (Buffered)
        Speed: 2666 MHz
        Manufacturer: 802C8632802C
        Serial Number: 191DE7B6
        Asset Tag: 0F1741A0
        Part Number: 9ASF1G72PZ-2G6D1
        Rank: 1
        Configured Clock Speed: 2400 MHz
        Minimum Voltage: 1.2 V
        Maximum Voltage: 1.2 V
        Configured Voltage: 1.2 V

Handle 0x1107, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 8192 MB
        Form Factor: DIMM
        Set: 1
        Locator: A8
        Bank Locator: Not Specified
        Type: DDR4
        Type Detail: Synchronous Registered (Buffered)
        Speed: 2666 MHz
        Manufacturer: 802C8632802C
        Serial Number: 191DE7C1
        Asset Tag: 0F1741A0
        Part Number: 9ASF1G72PZ-2G6D1
        Rank: 1
        Configured Clock Speed: 2400 MHz
        Minimum Voltage: 1.2 V
        Maximum Voltage: 1.2 V
        Configured Voltage: 1.2 V

Handle 0x1108, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: Unknown
        Data Width: Unknown
        Size: No Module Installed
        Form Factor: Unknown
        Set: 2
        Locator: A9
        Bank Locator: Not Specified
        Type: Unknown
        Type Detail: None
        Speed: Unknown
        Manufacturer: Not Specified
        Serial Number: Not Specified
        Asset Tag: Not Specified
        Part Number: Not Specified
        Rank: Unknown
        Configured Clock Speed: Unknown
        Minimum Voltage: Unknown
        Maximum Voltage: Unknown
        Configured Voltage: Unknown

Handle 0x1109, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: Unknown
        Data Width: Unknown
        Size: No Module Installed
        Form Factor: Unknown
        Set: 2
        Locator: A10
        Bank Locator: Not Specified
        Type: Unknown
        Type Detail: None
        Speed: Unknown
        Manufacturer: Not Specified
        Serial Number: Not Specified
        Asset Tag: Not Specified
        Part Number: Not Specified
        Rank: Unknown
        Configured Clock Speed: Unknown
        Minimum Voltage: Unknown
        Maximum Voltage: Unknown
        Configured Voltage: Unknown

Handle 0x110A, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: Unknown
        Data Width: Unknown
        Size: No Module Installed
        Form Factor: Unknown
        Set: 2
        Locator: A11
        Bank Locator: Not Specified
        Type: Unknown
        Type Detail: None
        Speed: Unknown
        Manufacturer: Not Specified
        Serial Number: Not Specified
        Asset Tag: Not Specified
        Part Number: Not Specified
        Rank: Unknown
        Configured Clock Speed: Unknown
        Minimum Voltage: Unknown
        Maximum Voltage: Unknown
        Configured Voltage: Unknown

Handle 0x110B, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: Unknown
        Data Width: Unknown
        Size: No Module Installed
        Form Factor: Unknown
        Set: 2
        Locator: A12
        Bank Locator: Not Specified
        Type: Unknown
        Type Detail: None
        Speed: Unknown
        Manufacturer: Not Specified
        Serial Number: Not Specified
        Asset Tag: Not Specified
        Part Number: Not Specified
        Rank: Unknown
        Configured Clock Speed: Unknown
        Minimum Voltage: Unknown
        Maximum Voltage: Unknown
        Configured Voltage: Unknown

Handle 0x110C, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: Unknown
        Data Width: Unknown
        Size: No Module Installed
        Form Factor: Unknown
        Set: 2
        Locator: A13
        Bank Locator: Not Specified
        Type: Unknown
        Type Detail: None
        Speed: Unknown
        Manufacturer: Not Specified
        Serial Number: Not Specified
        Asset Tag: Not Specified
        Part Number: Not Specified
        Rank: Unknown
        Configured Clock Speed: Unknown
        Minimum Voltage: Unknown
        Maximum Voltage: Unknown
        Configured Voltage: Unknown

Handle 0x110D, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: Unknown
        Data Width: Unknown
        Size: No Module Installed
        Form Factor: Unknown
        Set: 2
        Locator: A14
        Bank Locator: Not Specified
        Type: Unknown
        Type Detail: None
        Speed: Unknown
        Manufacturer: Not Specified
        Serial Number: Not Specified
        Asset Tag: Not Specified
        Part Number: Not Specified
        Rank: Unknown
        Configured Clock Speed: Unknown
        Minimum Voltage: Unknown
        Maximum Voltage: Unknown
        Configured Voltage: Unknown

Handle 0x110E, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: Unknown
        Data Width: Unknown
        Size: No Module Installed
        Form Factor: Unknown
        Set: 2
        Locator: A15
        Bank Locator: Not Specified
        Type: Unknown
        Type Detail: None
        Speed: Unknown
        Manufacturer: Not Specified
        Serial Number: Not Specified
        Asset Tag: Not Specified
        Part Number: Not Specified
        Rank: Unknown
        Configured Clock Speed: Unknown
        Minimum Voltage: Unknown
        Maximum Voltage: Unknown
        Configured Voltage: Unknown

Handle 0x110F, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: Unknown
        Data Width: Unknown
        Size: No Module Installed
        Form Factor: Unknown
        Set: 2
        Locator: A16
        Bank Locator: Not Specified
        Type: Unknown
        Type Detail: None
        Speed: Unknown
        Manufacturer: Not Specified
        Serial Number: Not Specified
        Asset Tag: Not Specified
        Part Number: Not Specified
        Rank: Unknown
        Configured Clock Speed: Unknown
        Minimum Voltage: Unknown
        Maximum Voltage: Unknown
        Configured Voltage: Unknown

    SSD part numbers for 120GB and 480GB SSD. The 120GB SSD appears to be Intel DC S3520 Series (Downieville) M.2 SSD and 480GB SSD appears to be Intel DC S3520 Series (Downieville) 2.5" SSD.
    Code (Text):
    lshw -short -C disk
H/W path              Device     Class          Description
===========================================================
/0/12e/3.1/0/0        /dev/sdc   disk           120GB SSDSCKJB120G7R
/0/12e/3.1/0/1        /dev/sdd   disk           120GB SSDSCKJB120G7R
/0/135/1.1/0/0.0.0    /dev/sda   disk           480GB SSDSC2BB480G7R
/0/135/1.1/0/0.1.0    /dev/sdb   disk           480GB SSDSC2BB480G7R

    Except there is no official Intel 120GB DC S3520 part as only a 150GB version exists so it could be a custom SSD manufactured by Dell. The 150GB Intel DCS3520 specs match with above shown disk speeds of around 170MB/s

    Intel® Product Specification Comparison

    epyc-ssd-compare.png

    Disk rated speeds aren't going to be able to fully utilised the 2x10Gbps bonded 20Gbps network that this AMD EPYC 7401P server sports unfortunately :(
     
    Last edited: Feb 28, 2018
  8. Feb 28, 2018 #8
    eva2000

    eva2000 Administrator Staff Member

    53,280
    12,117
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,655
    Local Time:
    2:23 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    Packet.net AMD EPYC 7401P c2.medium + Centmin Mod 123.09beta01 Install - Feb 28, 2018



    I've updated Centmin Mod 123.09beta01 code to now optimise Nginx and PHP-FPM compilation routines for AMD EPYC cpus if they are detected just like what is done when Intel cpus are detected. This will increase the initial install times for Nginx and PHP-FPM but in theory result in better performing compiled Nginx and PHP-FPM binaries.

    I have also updated CentOS 7.4 to default to 4.15.6-1 Elrepo Linux Kernel first before Centmin Mod 123.09beta01 install as well done some manual tuning to the system to see what the install time is like now.
    Code (Text):
    uname -r
4.15.6-1.el7.elrepo.x86_64

    Centmin Mod 123.09beta01 install command
    Code (Text):
    yum -y update; curl -O https://centminmod.com/betainstaller.sh && chmod 0700 betainstaller.sh && bash betainstaller.sh

    Centmin Mod 123.09beta01 install times
    Code (Text):
    ---------------------------------------------------------------------------
Total Curl Installer YUM or DNF Time: 128.5828 seconds
Total YUM Time: 38.440464465 seconds
Total YUM or DNF + Source Download Time: 59.0134
Total Nginx First Time Install Time: 176.0012
Total PHP First Time Install Time: 123.5181
Download Zip From Github Time: 1.3709
Total Time Other eg. source compiles: 341.5887
Total Centmin Mod Install Time: 700.1213
---------------------------------------------------------------------------
Total Install Time (curl yum + cm install + zip download): 830.0750 seconds
---------------------------------------------------------------------------

    Resulting Nginx compiled configuration

    AMD EPYC 7401P CPU Frequency Scaling Turbo Boost



    Interestingly, looking at turbostat output the AMD EPYC 7401P cpu frequency only fluctuated between 2344Mhz to 2550Mhz when under full 48 cpu thread cpu load. The AMD EPYC 7401P is rated for 2.0Ghz based, 2.8Ghz all cpus turbo boost and single cpu turbo boost of 3.0Ghz. So cpu frequency scaling not working properly ? Will have to investigate.

    top stats during load
    Code (Text):
    top -bn1 | head -n 52
top - 21:46:33 up 57 min,  3 users,  load average: 48.29, 24.43, 11.38
Tasks: 665 total,  47 running, 363 sleeping,   0 stopped,   0 zombie
%Cpu0  :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu1  :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu2  :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu3  :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu4  : 95.8 us,  4.2 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu5  :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu6  : 20.0 us, 20.0 sy,  0.0 ni, 60.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu7  :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu8  :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu9  :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu10 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu11 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu12 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu13 : 96.0 us,  4.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu14 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu15 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu16 : 64.0 us, 16.0 sy,  0.0 ni, 20.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu17 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu18 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu19 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu20 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu21 : 96.0 us,  4.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu22 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu23 : 96.0 us,  4.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu24 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu25 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu26 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu27 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu28 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu29 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu30 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu31 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu32 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu33 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu34 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu35 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu36 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu37 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu38 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu39 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu40 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu41 : 95.8 us,  4.2 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu42 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu43 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu44 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu45 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu46 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu47 :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem : 65620288 total, 48240376 free,  9984520 used,  7395392 buff/cache
KiB Swap:  1996796 total,  1996796 free,        0 used. 54709792 avail Mem

    turbostat
    Code (Text):
    Core    CPU     Avg_MHz Busy%   Bzy_MHz TSC_MHz IRQ
-       -       2321    100.00  2321    1996    248478
0       0       2317    100.00  2317    1996    5121
0       24      2317    100.00  2317    1996    5124
1       8       2322    100.00  2322    1996    5146
1       32      2322    100.00  2322    1996    5771
2       16      2321    100.00  2321    1996    5066
2       40      2321    100.00  2321    1996    5122
4       4       2322    100.00  2322    1996    5020
4       28      2322    100.00  2322    1996    5430
5       12      2321    100.00  2321    1996    5083
5       36      2321    100.00  2321    1996    5288
6       20      2322    100.00  2322    1996    5043
6       44      2322    100.00  2322    1996    5029
8       1       2322    100.00  2322    1996    5118
8       25      2322    100.00  2322    1996    5200
9       9       2322    100.00  2322    1996    5547
9       33      2322    100.00  2322    1996    5079
10      17      2322    100.00  2322    1996    5257
10      41      2322    100.00  2322    1996    5122
12      5       2311    100.00  2311    1996    5049
12      29      2311    100.00  2311    1996    5188
13      13      2322    100.00  2322    1996    5464
13      37      2322    100.00  2322    1996    5032
14      21      2322    100.00  2322    1996    5055
14      45      2322    100.00  2322    1996    5020
16      2       2321    100.00  2321    1996    5061
16      26      2321    100.00  2321    1996    5041
17      10      2322    100.00  2322    1996    5641
17      34      2322    100.00  2322    1996    5114
18      18      2317    100.00  2317    1996    5188
18      42      2317    100.00  2317    1996    5170
20      6       2321    100.00  2321    1996    5094
20      30      2321    100.00  2321    1996    5106
21      14      2322    100.00  2322    1996    5078
21      38      2322    100.00  2322    1996    5763
22      22      2322    100.00  2322    1996    5244
22      46      2322    100.00  2322    1996    5043
24      3       2322    100.00  2322    1996    5167
24      27      2322    100.00  2322    1996    5027
25      11      2322    100.00  2322    1996    5168
25      35      2322    100.00  2322    1996    5076
26      19      2322    100.00  2322    1996    5051
26      43      2322    100.00  2322    1996    5065
28      7       2322    100.00  2322    1996    5079
28      31      2322    100.00  2322    1996    5254
29      15      2322    100.00  2322    1996    5066
29      39      2322    100.00  2322    1996    5051
30      23      2322    100.00  2322    1996    5045
30      47      2322    100.00  2322    1996    5512

    Code (Text):
    Core    CPU     Avg_MHz Busy%   Bzy_MHz TSC_MHz IRQ
-       -       2531    100.00  2531    1996    69073
0       0       2532    100.00  2532    1996    1809
0       24      2532    100.00  2532    1996    1467
1       8       2532    100.00  2532    1996    1504
1       32      2532    100.00  2532    1996    1501
2       16      2531    100.00  2531    1996    1469
2       40      2531    100.00  2531    1996    1386
4       4       2532    100.00  2532    1996    1438
4       28      2532    100.00  2532    1996    1276
5       12      2531    100.00  2531    1996    1304
5       36      2531    100.00  2531    1996    1375
6       20      2532    100.00  2532    1996    1299
6       44      2532    100.00  2532    1996    1301
8       1       2532    100.00  2532    1996    1476
8       25      2532    100.00  2532    1996    1350
9       9       2532    100.00  2532    1996    1390
9       33      2532    100.00  2532    1996    1201
10      17      2532    100.00  2532    1996    1442
10      41      2532    100.00  2532    1996    1401
12      5       2528    100.00  2528    1996    1421
12      29      2528    100.00  2528    1996    1290
13      13      2532    100.00  2532    1996    1348
13      37      2532    100.00  2532    1996    1339
14      21      2532    100.00  2532    1996    1380
14      45      2532    100.00  2532    1996    1377
16      2       2531    100.00  2531    1996    1551
16      26      2531    100.00  2531    1996    1405
17      10      2532    100.00  2532    1996    1318
17      34      2532    100.00  2532    1996    1345
18      18      2530    100.00  2530    1996    1860
18      42      2530    100.00  2530    1996    1413
20      6       2531    100.00  2531    1996    3458
20      30      2531    100.00  2531    1996    1371
21      14      2532    100.00  2532    1996    1316
21      38      2532    100.00  2532    1996    1481
22      22      2532    100.00  2532    1996    1399
22      46      2532    100.00  2532    1996    1373
24      3       2532    100.00  2532    1996    1518
24      27      2532    100.00  2532    1996    1244
25      11      2532    100.00  2532    1996    1614
25      35      2532    100.00  2532    1996    1172
26      19      2532    100.00  2532    1996    1349
26      43      2532    100.00  2532    1996    1307
28      7       2532    100.00  2532    1996    1302
28      31      2532    100.00  2532    1996    1256
29      15      2532    100.00  2532    1996    1797
29      39      2532    100.00  2532    1996    1295
30      23      2532    100.00  2532    1996    1202
30      47      2532    100.00  2532    1996    1183


    Reinstalled CentOS 7.4 with default 3.10 kernel and I see at 24 cpu thread load that cpu jumps to 2700+Mhz and 48 cpu thread drops to 2560Mhz

    Code (Text):
            Core    CPU     Avg_MHz Busy%   Bzy_MHz TSC_MHz IRQ
        -       -       1363    50.05   2724    1996    128036
        0       0       2725    100.00  2725    1996    5436
        0       24      0       0.00    2702    1996    12
        1       8       2724    100.00  2724    1996    5039
        1       32      0       0.00    2694    1996    7
        2       16      2723    100.00  2723    1996    5038
        2       40      0       0.01    2699    1996    12
        4       4       2724    100.00  2724    1996    5036
        4       28      9       0.33    2709    1996    1125
        5       12      0       0.00    2682    1996    10
        5       36      2725    100.00  2725    1996    5442
        6       20      5       0.17    2730    1996    20
        6       44      2724    100.00  2724    1996    5041
        8       1       2725    100.00  2725    1996    5036
        8       25      0       0.00    2720    1996    6
        9       9       0       0.00    2707    1996    6
        9       33      2725    100.00  2725    1996    5034
        10      17      2725    100.00  2725    1996    5036
        10      41      0       0.00    2713    1996    9
        12      5       41      1.54    2689    1996    4953
        12      29      2724    100.00  2724    1996    5036
        13      13      0       0.00    2689    1996    6
        13      37      2724    100.00  2724    1996    5036
        14      21      0       0.00    2711    1996    6
        14      45      2725    100.00  2725    1996    5036
        16      2       2724    100.00  2724    1996    5036
        16      26      0       0.00    2702    1996    6
        17      10      2725    100.00  2725    1996    5036
        17      34      0       0.00    2722    1996    6
        18      18      2722    100.00  2722    1996    5036
        18      42      0       0.00    2665    1996    8
        20      6       0       0.00    2721    1996    6
        20      30      2725    100.00  2725    1996    5036
        21      14      1       0.03    2705    1996    91
        21      38      2725    100.00  2725    1996    5036
        22      22      0       0.00    2723    1996    6
        22      46      2724    100.00  2724    1996    5036
        24      3       2724    100.00  2724    1996    5036
        24      27      0       0.00    2685    1996    6
        25      11      2725    100.00  2725    1996    5036
        25      35      0       0.00    2719    1996    6
        26      19      2724    100.00  2724    1996    5036
        26      43      0       0.00    2694    1996    6
        28      7       2725    100.00  2725    1996    5036
        28      31      0       0.00    2706    1996    6
        29      15      2725    100.00  2725    1996    5036
        29      39      0       0.00    2726    1996    6
        30      23      2725    100.00  2725    1996    5036
        30      47      6       0.21    2724    1996    33

    Code (Text):
    turbostat
        Core    CPU     Avg_MHz Busy%   Bzy_MHz TSC_MHz IRQ
        -       -       2561    99.96   2562    1996    245435
        0       0       2560    99.99   2560    1994    5298
        0       24      2560    99.98   2560    1994    5089
        1       8       2557    99.93   2559    1994    5192
        1       32      2562    99.94   2564    1998    5170
        2       16      2563    99.99   2563    1998    5088
        2       40      2563    100.00  2563    1998    5090
        4       4       2563    99.97   2564    1998    5090
        4       28      2563    99.96   2564    1998    5091
        5       12      2563    99.93   2565    1998    5091
        5       36      2564    99.96   2565    1998    5405
        6       20      2560    99.96   2561    1995    5092
        6       44      2560    99.96   2561    1995    5096
        8       1       2561    99.97   2562    1995    5089
        8       25      2562    100.00  2562    1995    5069
        9       9       2560    99.96   2561    1995    5086
        9       33      2560    99.93   2561    1995    5090
        10      17      2560    99.95   2562    1995    5088
        10      41      2560    99.95   2562    1995    5090
        12      5       2558    99.94   2560    1995    5119
        12      29      2558    99.93   2560    1995    5129
        13      13      2559    99.96   2560    1995    5089
        13      37      2560    99.99   2560    1995    5088
        14      21      2560    99.94   2562    1996    5167
        14      45      2561    99.98   2562    1996    5208
        16      2       2560    99.94   2562    1996    5091
        16      26      2561    99.94   2562    1996    5089
        17      10      2561    99.97   2562    1996    5090
        17      34      2561    99.95   2562    1996    5088
        18      18      2558    99.97   2558    1996    5089
        18      42      2558    99.98   2558    1996    5091
        20      6       2561    99.95   2562    1996    5089
        20      30      2561    99.94   2562    1996    5085
        21      14      2561    99.93   2562    1996    5128
        21      38      2562    99.98   2562    1996    5160
        22      22      2561    99.97   2561    1996    5089
        22      46      2560    99.93   2561    1996    5087
        24      3       2560    99.98   2561    1996    5152
        24      27      2559    99.93   2561    1996    5121
        25      11      2561    99.97   2562    1996    5087
        25      35      2561    99.95   2562    1996    5089
        26      19      2560    99.98   2560    1996    5088
        26      43      2559    99.95   2560    1996    5089
        28      7       2563    99.99   2563    1996    5087
        28      31      2563    99.99   2563    1996    5089
        29      15      2563    99.99   2563    1996    5088
        29      39      2561    99.93   2563    1996    5088
        30      23      2563    100.00  2563    1996    5087
        30      47      2563    100.00  2563    1996    5090

    and 1 cpu thread load hits the max 3.0Ghz turbo speed
    Code (Text):
            Core    CPU     Avg_MHz Busy%   Bzy_MHz TSC_MHz IRQ
        -       -       65      2.16    2994    1996    7555
        0       0       3       0.11    2893    1996    516
        0       24      0       0.00    2717    1996    7
        1       8       0       0.01    2653    1996    16
        1       32      0       0.00    2690    1996    7
        2       16      0       0.01    2802    1996    13
        2       40      0       0.00    2567    1996    12
        4       4       91      3.03    2993    1996    269
        4       28      0       0.00    2630    1996    7
        5       12      0       0.00    2648    1996    11
        5       36      2       0.07    2891    1996    506
        6       20      0       0.00    2602    1996    7
        6       44      3       0.11    2745    1996    812
        8       1       0       0.00    2746    1996    7
        8       25      0       0.00    2722    1996    7
        9       9       0       0.00    2724    1996    7
        9       33      0       0.01    2703    1996    16
        10      17      0       0.00    2712    1996    7
        10      41      0       0.00    2714    1996    7
        12      5       0       0.00    2540    1996    8
        12      29      0       0.00    2594    1996    7
        13      13      0       0.00    2523    1996    7
        13      37      0       0.00    2574    1996    7
        14      21      0       0.00    2528    1996    7
        14      45      0       0.00    2590    1996    7
        16      2       0       0.00    2752    1996    5
        16      26      0       0.00    2707    1996    7
        17      10      0       0.00    2687    1996    7
        17      34      0       0.00    2716    1996    7
        18      18      0       0.00    2667    1996    7
        18      42      4       0.14    2959    1996    21
        20      6       0       0.00    2579    1996    7
        20      30      0       0.00    2648    1996    7
        21      14      1       0.02    2590    1996    66
        21      38      0       0.01    2667    1996    31
        22      22      0       0.00    2576    1996    7
        22      46      0       0.00    2643    1996    7
        24      3       0       0.00    2782    1996    5
        24      27      0       0.00    2716    1996    7
        25      11      0       0.00    2783    1996    8
        25      35      0       0.00    2684    1996    8
        26      19      0       0.00    2586    1996    7
        26      43      0       0.00    2651    1996    7
        28      7       0       0.00    2593    1996    7
        28      31      0       0.00    2653    1996    7
        29      15      0       0.00    2886    1996    7
        29      39      2994    100.00  2994    1996    5012
        30      23      0       0.00    2581    1996    7
        30      47      3       0.11    2970    1996    17

    So looks like it's something to do with either my 4.15 Kernel update or additional tweaks I did manually.

    Looks like it was due to my own tweaking of kernel boot command paramters - the culprit was setting idle=poll ! But that only gets me to 2.55-2.65Ghz max frequency and not 2.80Ghz rated max for AMD 7401P.

    Removing that idle=poll and reboot server shows turbostat

    at idle
    Code (Text):
    Core    CPU     Avg_MHz Busy%   Bzy_MHz TSC_MHz IRQ     C1      C2      C1%     C2%
-       -       0       0.01    2797    1996    978     374     588     1.65    98.34
0       0       2       0.06    2763    1996    182     10      167     0.11    99.84
0       24      0       0.00    2822    1996    12      7       4       0.09    99.91
1       8       1       0.02    2595    1996    110     11      96      3.15    96.83
1       32      0       0.00    2735    1996    12      7       4       0.08    99.91
2       16      0       0.00    2637    1996    14      10      5       0.10    99.89
2       40      0       0.00    2722    1996    12      8       4       0.10    99.89
4       4       0       0.00    2686    1996    14      8       4       0.12    99.88
4       28      1       0.02    2906    1996    110     20      86      24.98   75.00
5       12      0       0.00    2732    1996    12      6       5       0.08    99.92
5       36      0       0.01    2758    1996    69      18      60      47.45   52.54
6       20      0       0.00    2659    1996    17      13      4       0.11    99.88
6       44      0       0.00    2743    1996    12      5       6       0.07    99.92
8       1       0       0.00    2775    1996    12      8       4       0.11    99.89
8       25      0       0.00    2874    1996    11      7       3       0.09    99.91
9       9       0       0.00    2770    1996    11      7       4       0.09    99.91
9       33      0       0.00    2880    1996    11      7       3       0.09    99.91
10      17      0       0.00    2672    1996    13      5       7       0.07    99.93
10      41      0       0.00    2907    1996    10      5       4       0.07    99.93
12      5       0       0.00    2634    1996    12      9       3       0.12    99.88
12      29      0       0.00    2662    1996    11      7       3       0.08    99.92
13      13      0       0.00    2581    1996    11      8       4       0.10    99.90
13      37      0       0.00    2737    1996    9       5       3       0.08    99.92
14      21      0       0.00    2660    1996    12      8       4       0.09    99.90
14      45      0       0.00    2743    1996    9       5       3       0.08    99.92
16      2       0       0.01    2794    1996    15      10      5       0.11    99.89
16      26      0       0.00    2790    1996    10      6       3       0.07    99.93
17      10      0       0.00    2728    1996    11      5       4       0.07    99.93
17      34      0       0.00    2851    1996    10      6       3       0.07    99.93
18      18      0       0.00    2658    1996    12      5       5       0.07    99.93
18      42      0       0.00    2899    1996    9       4       4       0.05    99.95
20      6       0       0.00    2636    1996    12      5       4       0.08    99.92
20      30      0       0.00    2728    1996    10      4       4       0.04    99.96
21      14      0       0.01    2733    1996    15      6       9       0.08    99.92
21      38      0       0.00    2794    1996    8       4       3       0.06    99.94
22      22      0       0.00    2621    1996    10      7       3       0.07    99.92
22      46      0       0.00    2735    1996    8       5       2       0.08    99.92
24      3       0       0.00    2800    1996    10      8       2       0.09    99.91
24      27      0       0.00    2714    1996    14      5       8       0.07    99.92
25      11      0       0.00    2760    1996    9       6       3       0.07    99.93
25      35      0       0.00    2841    1996    9       6       2       0.07    99.93
26      19      0       0.00    2780    1996    11      6       4       0.07    99.93
26      43      0       0.00    2860    1996    8       4       3       0.05    99.95
28      7       0       0.00    2659    1996    10      7       3       0.06    99.94
28      31      0       0.01    2729    1996    11      4       5       0.03    99.97
29      15      0       0.00    2723    1996    13      10      4       0.14    99.86
29      39      0       0.00    2810    1996    15      29      2       0.14    99.86
30      23      0       0.01    2741    1996    14      15      4       0.13    99.86
30      47      2       0.08    2964    1996    16      3       9       0.05    99.87

    1 cpu load @ 2994Mhz
    Code (Text):
    Core    CPU     Avg_MHz Busy%   Bzy_MHz TSC_MHz IRQ     C1      C2      C1%     C2%
-       -       63      2.09    2993    1996    6182    454     681     2.26    95.64
0       0       2       0.07    2675    1996    191     12      173     0.16    99.77
0       24      0       0.00    2770    1996    18      9       8       0.12    99.87
1       8       1       0.02    2592    1996    113     23      91      9.89    90.09
1       32      0       0.00    2831    1996    16      9       7       0.12    99.88
2       16      0       0.00    2552    1996    17      10      7       0.14    99.86
2       40      0       0.00    2698    1996    16      10      5       0.14    99.86
4       4       0       0.00    2662    1996    16      11      5       0.15    99.84
4       28      1       0.02    2853    1996    123     26      85      47.88   52.10
5       12      0       0.00    2764    1996    17      10      6       0.13    99.87
5       36      0       0.01    2719    1996    74      19      64      46.07   53.92
6       20      0       0.00    2596    1996    17      11      6       0.15    99.85
6       44      0       0.00    2707    1996    16      8       7       0.11    99.89
8       1       200     6.69    2994    1996    354     9       7       0.13    93.19
8       25      4       0.12    2991    1996    25      10      7       0.13    99.75
9       9       0       0.01    2725    1996    23      10      12      0.14    99.85
9       33      0       0.00    2837    1996    15      8       6       0.10    99.90
10      17      0       0.00    2662    1996    17      7       9       0.08    99.92
10      41      0       0.00    2822    1996    15      7       7       0.10    99.90
12      5       0       0.00    2658    1996    16      12      4       0.15    99.85
12      29      0       0.00    2694    1996    14      8       5       0.12    99.89
13      13      0       0.00    2654    1996    15      13      4       0.15    99.85
13      37      0       0.00    2674    1996    14      7       6       0.09    99.91
14      21      0       0.00    2544    1996    15      6       8       0.09    99.91
14      45      0       0.00    2726    1996    14      6       7       0.07    99.93
16      2       2789    93.17   2994    1996    4676    0       2       0.00    6.83
16      26      0       0.00    2958    1996    15      7       6       0.10    99.89
17      10      0       0.00    2701    1996    15      11      5       0.14    99.86
17      34      0       0.00    2831    1996    13      7       5       0.10    99.90
18      18      0       0.00    2648    1996    15      7       7       0.09    99.90
18      42      0       0.00    2789    1996    13      7       5       0.09    99.90
20      6       0       0.00    2626    1996    15      8       6       0.11    99.89
20      30      0       0.00    2742    1996    13      7       5       0.09    99.91
21      14      0       0.01    2661    1996    19      6       11      0.08    99.91
21      38      0       0.00    2696    1996    13      7       5       0.09    99.91
22      22      0       0.00    2614    1996    14      8       6       0.10    99.89
22      46      0       0.00    2697    1996    13      7       5       0.08    99.91
24      3       3       0.09    2958    1996    27      14      8       0.21    99.70
24      27      0       0.00    2666    1996    17      6       9       0.10    99.90
25      11      0       0.00    2744    1996    14      9       5       0.12    99.88
25      35      0       0.00    2840    1996    12      6       5       0.08    99.92
26      19      0       0.00    2714    1996    14      7       6       0.09    99.90
26      43      0       0.00    2830    1996    12      6       5       0.07    99.92
28      7       0       0.00    2667    1996    13      8       4       0.11    99.89
28      31      0       0.00    2733    1996    12      6       5       0.07    99.93
29      15      0       0.01    2713    1996    13      22      4       0.10    99.89
29      39      0       0.01    2780    1996    12      9       3       0.10    99.89
30      23      0       0.01    2684    1996    14      15      5       0.10    99.89
30      47      3       0.10    2914    1996    17      3       8       0.03    99.87

    all 24 cpu threads load ~@ 2.63Ghz
    Code (Text):
    Core    CPU     Avg_MHz Busy%   Bzy_MHz TSC_MHz IRQ     C1      C2      C1%     C2%
-       -       1327    50.48   2629    1996    126226  2260    2936    8.60    40.92
0       0       34      1.28    2628    1996    4229    1686    2506    42.57   56.21
0       24      2629    99.99   2629    1996    5052    10      0       0.01    0.00
1       8       0       0.02    2624    1996    104     17      88      14.97   85.02
1       32      2629    99.99   2629    1996    5053    10      0       0.01    0.00
2       16      306     11.66   2628    1996    16      6       6       0.09    88.25
2       40      2629    99.99   2629    1996    5051    14      0       0.01    0.00
4       4       2629    99.99   2629    1996    5052    15      0       0.01    0.00
4       28      1       0.03    2627    1996    150     38      102     58.41   41.56
5       12      0       0.01    2629    1996    35      8       24      1.22    98.77
5       36      2629    99.98   2629    1996    5053    10      0       0.02    0.00
6       20      2629    99.99   2629    1996    5054    10      0       0.01    0.00
6       44      0       0.01    2627    1996    72      20      63      60.97   39.01
8       1       2629    99.99   2629    1996    5053    14      0       0.01    0.00
8       25      0       0.00    2622    1996    14      5       6       0.06    99.94
9       9       2629    99.98   2629    1996    5053    10      0       0.02    0.00
9       33      0       0.00    2632    1996    17      8       6       77.33   22.67
10      17      2629    99.99   2629    1996    5054    10      0       0.01    0.00
10      41      0       0.00    2634    1996    16      6       7       0.08    99.92
12      5       2629    99.99   2629    1996    5053    10      0       0.01    0.00
12      29      0       0.00    2630    1996    15      6       6       0.08    99.92
13      13      2629    99.99   2629    1996    5054    10      0       0.01    0.00
13      37      0       0.00    2629    1996    14      5       6       0.06    99.94
14      21      2629    99.99   2629    1996    5054    10      0       0.01    0.00
14      45      0       0.00    2622    1996    17      7       7       77.78   22.21
16      2       2629    99.98   2629    1996    5051    10      0       0.02    0.00
16      26      0       0.00    2622    1996    16      8       5       0.12    99.88
17      10      268     10.19   2629    1996    21      9       7       0.13    89.68
17      34      2629    99.99   2629    1996    5054    15      0       0.01    0.00
18      18      2629    99.99   2629    1996    5053    14      0       0.01    0.00
18      42      0       0.00    2619    1996    14      5       6       0.06    99.94
20      6       2629    99.99   2629    1996    5053    14      0       0.01    0.00
20      30      0       0.01    2624    1996    22      6       12      0.10    99.90
21      14      2629    99.99   2629    1996    5054    14      0       0.01    0.00
21      38      0       0.00    2621    1996    16      6       7       0.08    99.92
22      22      2629    99.99   2629    1996    5051    10      0       0.01    0.00
22      46      0       0.00    2617    1996    15      6       6       0.08    99.92
24      3       2629    99.99   2629    1996    5053    10      0       0.01    0.00
24      27      0       0.00    2622    1996    18      6       9       0.10    99.90
25      11      3       0.13    2620    1996    38      14      15      0.23    99.64
25      35      2629    99.99   2629    1996    5054    14      0       0.01    0.00
26      19      2629    99.99   2629    1996    5054    10      0       0.01    0.00
26      43      0       0.00    2631    1996    20      8       9       0.12    99.88
28      7       2629    99.99   2629    1996    5053    10      0       0.01    0.00
28      31      0       0.01    2624    1996    24      90      3       0.21    99.78
29      15      2629    99.99   2629    1996    5054    10      0       0.01    0.00
29      39      0       0.02    2624    1996    39      19      17      77.45   22.53
30      23      2629    100.00  2629    1996    5035    0       0       0.00    0.00
30      47      3       0.12    2623    1996    29      7       13      0.11    99.78

    all 48 cpu threads load ~@2.55Ghz
    Code (Text):
    Core    CPU     Avg_MHz Busy%   Bzy_MHz TSC_MHz IRQ     C1      C2      C1%     C2%
-       -       2536    99.98   2537    1996    242691  491     0       0.02    0.00
0       0       2537    100.00  2537    1996    5109    0       0       0.00    0.00
0       24      2536    99.97   2537    1996    5050    10      0       0.03    0.00
1       8       2536    99.98   2537    1996    5064    13      0       0.02    0.00
1       32      2536    99.97   2537    1996    5049    10      0       0.03    0.00
2       16      2536    99.98   2537    1996    5055    11      0       0.02    0.00
2       40      2536    99.97   2537    1996    5053    10      0       0.02    0.00
4       4       2536    99.97   2537    1996    5054    10      0       0.03    0.00
4       28      2536    99.98   2537    1996    5195    11      0       0.02    0.00
5       12      2536    99.98   2537    1996    5051    10      0       0.02    0.00
5       36      2536    99.97   2537    1996    5049    10      0       0.03    0.00
6       20      2536    99.98   2537    1996    5051    14      0       0.02    0.00
6       44      2536    99.97   2537    1996    5052    10      0       0.03    0.00
8       1       2536    99.98   2537    1996    5052    15      0       0.02    0.00
8       25      2536    99.98   2537    1996    5052    10      0       0.02    0.00
9       9       2536    99.98   2537    1996    5052    10      0       0.02    0.00
9       33      2536    99.98   2537    1996    5053    10      0       0.02    0.00
10      17      2536    99.97   2537    1996    5049    10      0       0.03    0.00
10      41      2536    99.97   2537    1996    5052    10      0       0.03    0.00
12      5       2536    99.97   2537    1996    5053    10      0       0.03    0.00
12      29      2536    99.98   2537    1996    5052    10      0       0.02    0.00
13      13      2536    99.98   2537    1996    5051    10      0       0.02    0.00
13      37      2536    99.97   2537    1996    5052    10      0       0.03    0.00
14      21      2536    99.98   2537    1996    5051    10      0       0.02    0.00
14      45      2536    99.97   2537    1996    5052    10      0       0.03    0.00
16      2       2536    99.97   2537    1996    5053    10      0       0.03    0.00
16      26      2536    99.98   2537    1996    5051    11      0       0.02    0.00
17      10      2536    99.97   2537    1996    5051    10      0       0.03    0.00
17      34      2536    99.97   2537    1996    5053    10      0       0.03    0.00
18      18      2536    99.98   2537    1996    5051    10      0       0.02    0.00
18      42      2536    99.97   2537    1996    5052    11      0       0.03    0.00
20      6       2536    99.98   2537    1996    5053    10      0       0.02    0.00
20      30      2536    99.98   2537    1996    5051    10      0       0.02    0.00
21      14      2536    99.98   2537    1996    5051    10      0       0.02    0.00
21      38      2536    99.97   2537    1996    5050    10      0       0.03    0.00
22      22      2536    99.98   2537    1996    5054    10      0       0.02    0.00
22      46      2536    99.98   2537    1996    5050    10      0       0.02    0.00
24      3       2536    99.97   2537    1996    5050    10      0       0.03    0.00
24      27      2536    99.97   2537    1996    5050    10      0       0.03    0.00
25      11      2536    99.97   2537    1996    5052    10      0       0.03    0.00
25      35      2536    99.98   2537    1996    5053    10      0       0.02    0.00
26      19      2536    99.98   2537    1996    5051    10      0       0.02    0.00
26      43      2536    99.98   2537    1996    5052    10      0       0.02    0.00
28      7       2536    99.97   2537    1996    5053    11      0       0.03    0.00
28      31      2536    99.98   2537    1996    5051    10      0       0.02    0.00
29      15      2536    99.98   2537    1996    5051    10      0       0.02    0.00
29      39      2536    99.98   2537    1996    5052    10      0       0.02    0.00
30      23      2536    99.97   2537    1996    5052    11      0       0.03    0.00
30      47      2536    99.98   2537    1996    5051    13      0       0.02    0.00


    So again it was my own manual tweaks I needed to remove for another kernel command boot parameter skew_tick=1 LOL

    24 cpu thread load now gives 2760Mhz clock speed
    Code (Text):
    Core    CPU     Avg_MHz Busy%   Bzy_MHz TSC_MHz IRQ     C1      C2      C1%     C2%
-       -       1007    36.46   2761    2000    95200   5272    2920    26.23   37.44
0       0       1978    71.50   2767    2004    3714    81      24      18.70   9.92
0       24      0       0.01    2751    2004    92      83      17      20.88   79.52
1       8       577     20.81   2774    2004    405     69      19      28.89   50.61
1       32      1982    71.63   2766    2004    3712    83      26      21.03   7.45
2       16      1987    71.84   2765    2004    3709    86      1       28.27   0.00
2       40      1       0.03    2722    2004    89      44      45      15.83   84.51
4       4       1987    71.85   2766    2003    3717    65      33      18.54   9.71
4       28      1       0.05    2759    2003    192     80      100     72.80   27.50
5       12      1986    71.82   2765    2003    3711    82      4       28.27   0.00
5       36      1       0.04    2766    2003    271     70      205     10.00   90.29
6       20      1985    71.81   2765    2003    3718    89      3       28.27   0.00
6       44      1       0.03    2753    2003    205     97      105     45.95   54.34
8       1       1985    71.80   2765    2003    3708    86      5       28.28   0.00
8       25      0       0.01    2743    2002    106     86      16      20.98   79.30
9       9       0       0.01    2739    2002    86      74      11      20.88   79.42
9       33      1985    71.82   2764    2002    3726    88      15      20.94   7.33
10      17      1984    71.78   2764    2002    3709    76      11      28.29   0.02
10      41      0       0.02    2772    2002    117     86      23      21.08   79.18
12      5       24      0.86    2738    2002    3381    1391    1969    55.93   43.51
12      29      1983    71.78   2762    2001    3709    88      2       28.28   0.00
13      13      1971    71.38   2762    2001    3691    84      1       28.68   0.00
13      37      0       0.01    2765    2001    93      82      13      20.97   79.26
14      21      285     10.19   2796    2000    602     72      7       41.08   48.93
14      45      1981    71.75   2762    2000    3703    78      6       28.30   0.01
16      2       1934    70.09   2760    2000    3712    128     50      29.02   0.95
16      26      0       0.01    2751    2000    91      78      13      20.93   79.25
17      10      1709    62.01   2755    2000    3248    99      12      28.57   9.49
17      34      0       0.01    2694    2000    99      83      16      21.03   79.14
18      18      1979    71.75   2758    1999    3711    90      1       28.30   0.00
18      42      0       0.01    2720    1999    98      82      16      21.01   79.14
20      6       1871    67.87   2757    1999    3518    82      2       32.17   0.00
20      30      0       0.01    2776    1999    98      80      17      21.00   79.14
21      14      1979    71.73   2759    1999    3709    83      2       28.30   0.00
21      38      0       0.01    2769    1999    96      81      15      21.00   79.12
22      22      1979    71.73   2759    1998    3710    84      1       28.30   0.00
22      46      0       0.01    2776    1998    95      79      16      20.96   79.14
24      3       1978    71.72   2758    1998    3709    82      2       28.31   0.00
24      27      0       0.01    2733    1998    94      81      13      21.00   79.08
25      11      1978    71.71   2758    1998    3710    87      1       28.31   0.00
25      35      0       0.01    2787    1998    93      78      15      20.95   79.12
26      19      2301    83.32   2762    1997    3712    20      4       16.69   0.00
26      43      0       0.01    2752    1997    92      80      13      20.97   79.08
28      7       1977    71.70   2757    1997    3709    82      2       28.31   0.00
28      31      0       0.02    2791    1997    92      81      12      20.98   79.04
29      15      1976    71.69   2756    1997    3710    87      3       21.89   6.42
29      39      3       0.11    2790    1997    118     163     16      20.88   79.04
30      23      1975    71.68   2756    1996    3710    85      1       28.32   0.00
30      47      4       0.13    2787    1996    100     77      16      20.88   79.00

    48 cpu threads gives 2580-2603Mhz cpu clock speed - is this the max or should it be 2.8Ghz ?
    Code (Text):
    Core    CPU     Avg_MHz Busy%   Bzy_MHz TSC_MHz IRQ     C1      C2      C1%     C2%
-       -       1773    68.42   2591    1988    167510  772     730     0.55    30.90
0       0       1773    68.72   2580    1979    3506    26      27      0.18    30.84
0       24      1722    66.84   2577    1980    3414    14      14      0.13    32.76
1       8       1754    68.00   2579    1980    3473    18      12      0.11    31.63
1       32      1710    66.36   2577    1980    3397    9       22      0.08    33.28
2       16      1731    67.15   2578    1981    3421    12      7       0.09    32.50
2       40      1698    65.90   2578    1981    3362    9       11      0.08    33.76
4       4       1772    68.61   2583    1981    3539    16      52      5.42    25.74
4       28      1809    69.87   2589    1982    3636    9       48      4.26    25.66
5       12      1798    69.50   2587    1982    3545    8       15      0.07    30.21
5       36      1794    69.38   2586    1983    3571    13      45      0.11    30.30
6       20      1769    68.45   2584    1983    3489    16      6       0.09    31.25
6       44      1808    69.81   2590    1983    3641    29      36      10.42   19.57
8       1       1757    67.98   2584    1984    3476    17      19      0.11    31.70
8       25      1793    69.28   2588    1984    3528    12      12      0.09    30.44
9       9       1756    67.93   2585    1984    3461    9       12      0.07    31.81
9       33      1791    69.18   2589    1985    3522    11      10      0.09    30.56
10      17      1793    69.25   2589    1985    3537    18      16      0.22    30.36
10      41      1770    68.42   2587    1985    3487    11      13      0.13    31.28
12      5       1814    70.02   2591    1986    3568    16      15      0.16    29.67
12      29      1783    68.91   2587    1986    3512    21      11      0.21    30.72
13      13      1770    68.39   2589    1987    3478    19      6       0.16    31.30
13      37      1782    68.80   2590    1987    3504    12      12      0.14    30.92
14      21      1765    68.17   2589    1987    3473    18      9       0.18    31.52
14      45      1780    68.70   2591    1988    3501    10      15      0.11    31.05
16      2       1745    67.44   2588    1988    3448    25      20      0.25    32.18
16      26      1740    67.24   2588    1988    3424    10      13      0.11    32.52
17      10      1806    69.55   2597    1989    3533    20      5       0.15    30.19
17      34      1716    66.30   2588    1989    3380    9       17      0.11    33.47
18      18      1810    69.73   2596    1989    3543    20      7       0.12    30.04
18      42      1707    65.98   2587    1990    3365    15      15      0.19    33.73
20      6       1778    68.57   2592    1990    3484    20      6       0.15    31.18
20      30      1747    67.46   2590    1990    3432    11      11      0.13    32.32
21      14      1760    67.88   2592    1991    3453    21      9       0.19    31.84
21      38      1786    68.82   2596    1991    3500    11      13      0.12    30.97
22      22      1772    68.31   2594    1991    3474    15      10      0.15    31.47
22      46      1807    69.49   2600    1992    3531    15      11      0.14    30.30
24      3       1777    68.47   2596    1992    3482    14      13      0.12    31.34
24      27      1749    67.44   2593    1993    3427    10      11      0.10    32.40
25      11      1766    68.04   2596    1993    3462    17      13      0.19    31.72
25      35      1713    66.05   2594    1993    3372    15      21      0.21    33.69
26      19      1777    68.41   2597    1994    3477    20      10      0.19    31.36
26      43      1819    69.88   2604    1994    3549    13      14      0.11    29.98
28      7       1815    69.72   2604    1994    3551    21      14      0.24    30.01
28      31      1803    69.31   2602    1995    3521    21      10      0.17    30.50
29      15      1799    69.14   2602    1995    3509    20      6       0.16    30.68
29      39      1801    69.20   2602    1995    3533    49      12      0.16    30.62
30      23      1802    69.27   2602    1996    3516    19      8       0.14    30.58
30      47      1793    68.87   2603    1996    3503    8       16      0.08    31.05


    So lets try a new Centmin Mod 123.09beta01 install
    Code (Text):
    yum -y update; curl -O https://centminmod.com/betainstaller.sh && chmod 0700 betainstaller.sh && bash betainstaller.sh

    Centmin Mod 123.09beta01 install times at ~733 seconds which for AMD cpu compiler flag optimised build is decent considering non AMD cpu compiler flag optimised build previous took ~769 seconds
    Code (Text):
    ---------------------------------------------------------------------------
Total Curl Installer YUM or DNF Time: 152.7325 seconds
Total YUM Time: 35.876224841 seconds
Total YUM or DNF + Source Download Time: 54.1354
Total Nginx First Time Install Time: 138.7361
Total PHP First Time Install Time: 103.9416
Download Zip From Github Time: 1.0804
Total Time Other eg. source compiles: 282.6103
Total Centmin Mod Install Time: 579.4234
---------------------------------------------------------------------------
Total Install Time (curl yum + cm install + zip download): 733.2363 seconds
---------------------------------------------------------------------------

    Looking good, checking cpu frequency during Nginx compilation stage of Centmin Mod installation shows ~2780Mhz cpu frequency :)
    Code (Text):
    grep MHz /proc/cpuinfo
cpu MHz         : 2773.788
cpu MHz         : 2778.998
cpu MHz         : 2745.497
cpu MHz         : 2765.689
cpu MHz         : 2770.491
cpu MHz         : 2724.596
cpu MHz         : 2772.604
cpu MHz         : 2784.042
cpu MHz         : 2782.290
cpu MHz         : 2780.214
cpu MHz         : 2779.382
cpu MHz         : 2783.001
cpu MHz         : 2768.134
cpu MHz         : 2767.165
cpu MHz         : 2780.245
cpu MHz         : 2791.233
cpu MHz         : 2754.883
cpu MHz         : 2782.203
cpu MHz         : 2747.773
cpu MHz         : 2772.235
cpu MHz         : 2769.830
cpu MHz         : 2778.107
cpu MHz         : 2781.585
cpu MHz         : 2777.746
cpu MHz         : 2778.892
cpu MHz         : 2779.206
cpu MHz         : 2755.335
cpu MHz         : 2772.050
cpu MHz         : 2780.281
cpu MHz         : 2750.842
cpu MHz         : 2774.551
cpu MHz         : 2782.898
cpu MHz         : 2784.215
cpu MHz         : 2784.794
cpu MHz         : 2780.141
cpu MHz         : 2784.718
cpu MHz         : 2774.572
cpu MHz         : 2773.928
cpu MHz         : 2777.358
cpu MHz         : 2792.444
cpu MHz         : 2752.807
cpu MHz         : 2785.860
cpu MHz         : 2750.918
cpu MHz         : 2773.167
cpu MHz         : 2774.841
cpu MHz         : 2776.860
cpu MHz         : 2786.369
cpu MHz         : 2775.252


    GCC 8.0.1 Compile & RPM Build Times



    Centmin Mod LEMP Stack's Nginx web server supports multiple GCC/Clang version compilers from GCC 4.8, 5.3, 6.3, 7.2 and 8.x and Clang 3.4, 4, 5, and 6. Retesting binutils 2.30 + GCC 8.0.1 with Profile Guided Optimizations (PGO) custom RPM build times. Took ~2074 seconds to build RPMs.
    Code (Text):
    /opt/gcc-8-20180225/bin/gcc -v
Using built-in specs.
COLLECT_GCC=/opt/gcc-8-20180225/bin/gcc
COLLECT_LTO_WRAPPER=/opt/gcc-8-20180225/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --prefix=/opt/gcc-8-20180225 --disable-multilib --enable-bootstrap --enable-plugin --with-gcc-major-version-only --enable-shared --disable-nls --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-install-libiberty --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-linker-hash-style=gnu --enable-languages=c,c++ --enable-initfini-array --disable-libgcj --enable-gnu-indirect-function --with-tune=generic --build=x86_64-redhat-linux --enable-lto --enable-gold
Thread model: posix
gcc version 8.0.1 20180225 (experimental) (GCC)

RPMs Built
/svr-setup/binutils-gcc8-2.30-1.el7.x86_64.rpm
/svr-setup/gcc8-pgo-8.0-1.el7.x86_64.rpm

ls -lah /svr-setup | egrep 'gcc8-pgo-8.0|binutils-gcc8-2.30'
-rw-r--r--  1 root      root       5.2M Feb 28 00:46 binutils-gcc8-2.30-1.el7.x86_64.rpm
-rw-r--r--  1 root      root        39M Feb 28 01:17 gcc8-pgo-8.0-1.el7.x86_64.rpm

Installed Packages
Name        : binutils-gcc8
Arch        : x86_64
Version     : 2.30
Release     : 1.el7
Size        : 48 M
Repo        : installed
Summary     : binutils-gcc8 for centminmod.com LEMP stack installs
URL         : https://centminmod.com
License     : unknown
Description : binutils-gcc8 for centminmod.com LEMP stacks

* Wed Feb 28 2018 George Liu <centminmod.com> 2.30
- Binutils 2.30 for centminmod.com LEMP stack installs

    Code (Text):
    Installed Packages
Name        : gcc8-pgo
Arch        : x86_64
Version     : 8.0
Release     : 1.el7
Size        : 287 M
Repo        : installed
Summary     : gcc8-pgo for centminmod.com LEMP stack installs
URL         : https://centminmod.com
License     : unknown
Description : gcc8-pgo for centminmod.com LEMP stacks

* Wed Feb 28 2018 George Liu <centminmod.com> 8
- GCC 8 for centminmod.com LEMP stack installs

*************************************************
Compile GCC Completed
log: /root/centminlogs/tools-gcc-install-pgo_280218-004337.log
*************************************************

Total Binutils + GCC Install Time: 2074.655782081 seconds


    AMD EPYC 7401P Turbo Frequencies



    Looking at EPYC 7401P - AMD the Turbo frequencies are as follows
     
    Last edited: Feb 28, 2018
  9. Feb 28, 2018 #9
    eva2000

    eva2000 Administrator Staff Member

    53,280
    12,117
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,655
    Local Time:
    2:23 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    Packet.net AMD EPYC 7401P - Dell PowerEdge R6415 Bios



    A quick peak into Dell PowerEdge R6415 Bios for the AMD EPYC 7401P server :)

    packetnet-epyc-bios-setup-01.png packetnet-epyc-bios-setup-02.png packetnet-epyc-bios-setup-03.png packetnet-epyc-bios-setup-04.png packetnet-epyc-bios-setup-05.png packetnet-epyc-bios-setup-06.png packetnet-epyc-bios-setup-07.png packetnet-epyc-bios-setup-08.png packetnet-epyc-bios-setup-09.png packetnet-epyc-bios-setup-10.png
     
Previous Thread Next Thread
Loading...
Thread Status:
Not open for further replies.

.