Forums - Linode KVM Migration

The forums have finally be migrated from 4GB Linode Xen VPS to 4GB Linode KVM VPS for better performance. Also moved from Intel Xeon E5-2680v2 cpus to the newer Intel Xeon E5-2680v3 Haswell cpus with much better encryption/SSL performance

Actual Linode Xen to KVM migration time was 7 mins for data + 2 mins for migration clean up = 9 minutes total

Forums can still get faster as I am still on MariaDB 5.5. The update to MariaDB 10.0.x will be next

AWS Route53 DNS failover to backup VPS 503 maintenance server worked nicely too. Probably wasn't as fast with 30 seconds regular health checks versus 10 seconds fast health checks.

Code (Text):

Notification: System is DOWN
IP: 173.230.147.144
System Desc: cm 503 failover
Port: 443
Protocol: TCP
2015-08-23 21:39:13.269 UTC
ID:
Previously failed: 0 times.

Code (Text):

Notification: System is UP
IP: 173.230.147.144
System Desc: cm 503 failover
Port: 443
Protocol: TCP
2015-08-23 21:52:59.694 UTC
Previously failed: 3 times.
ID:

Code (Text):

cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 63
model name      : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping        : 2
microcode       : 0x1
cpu MHz         : 2499.994
cache size      : 30720 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer xsave avx f16c rdrand hypervisor lahf_lm abm fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt
bugs            :
bogomips        : 5001.32
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 63
model name      : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping        : 2
microcode       : 0x1
cpu MHz         : 2499.994
cache size      : 30720 KB
physical id     : 1
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 1
initial apicid  : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer xsave avx f16c rdrand hypervisor lahf_lm abm fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt
bugs            :
bogomips        : 5001.32
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 2
vendor_id       : GenuineIntel
cpu family      : 6
model           : 63
model name      : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping        : 2
microcode       : 0x1
cpu MHz         : 2499.994
cache size      : 30720 KB
physical id     : 2
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 2
initial apicid  : 2
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer xsave avx f16c rdrand hypervisor lahf_lm abm fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt
bugs            :
bogomips        : 5001.32
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 63
model name      : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping        : 2
microcode       : 0x1
cpu MHz         : 2499.994
cache size      : 30720 KB
physical id     : 3
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 3
initial apicid  : 3
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer xsave avx f16c rdrand hypervisor lahf_lm abm fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt
bugs            :
bogomips        : 5001.32
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

 

Very quick benchmarks for Xeon E5-2680v3

Linode Disk I/O

Code (Text):

dd if=/dev/zero of=sb-io-test bs=128k count=1k conv=fdatasync
1024+0 records in
1024+0 records out
134217728 bytes (134 MB) copied, 0.264556 s, 507 MB/s
 

Code (Text):

dd if=/dev/zero of=sb-io-test bs=8k count=16k conv=fdatasync
16384+0 records in
16384+0 records out
134217728 bytes (134 MB) copied, 0.186124 s, 721 MB/s
 

Code (Text):

dd if=/dev/zero of=sb-io-test bs=128k count=1k oflag=dsync
1024+0 records in
1024+0 records out
134217728 bytes (134 MB) copied, 0.770734 s, 174 MB/s
 

Code (Text):

dd if=/dev/zero of=sb-io-test bs=8k count=16k oflag=dsync
16384+0 records in
16384+0 records out
134217728 bytes (134 MB) copied, 7.02928 s, 19.1 MB/s
 

System OpenSSL

Code (Text):

openssl speed rsa4096 rsa2048 ecdsap256 sha256 sha1 md5 rc4 aes-256-cbc aes-128-cbc -multi

OpenSSL 1.0.1e-fips 11 Feb 2013
built on: Thu Jul 23 19:06:35 UTC 2015
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
md5             181272.19k   546248.43k  1188019.11k  1682422.44k  1917927.42k
sha1            206100.39k   578866.33k  1189582.59k  1780727.47k  2241680.73k
rc4            1336735.43k  2020682.07k  2364263.59k  2296203.95k  2396484.95k
aes-128 cbc     379622.43k   425944.28k   442604.37k   912943.79k   938920.62k
aes-256 cbc     277859.16k   294068.10k   304622.17k   679045.80k   694804.48k
sha256          153822.84k   343844.35k   587159.04k   690615.64k   745474.73k
                  sign    verify    sign/s verify/s
rsa 2048 bits 0.000390s 0.000012s   2562.3  82908.2
rsa 4096 bits 0.002740s 0.000043s    365.0  23088.9
                              sign    verify    sign/s verify/s
 256 bit ecdsa (nistp256)   0.0000s   0.0001s  27546.2   7564.4
 

Code (Text):

openssl speed -evp aes256 -multi 4

OpenSSL 1.0.1e-fips 11 Feb 2013
built on: Thu Jul 23 19:06:35 UTC 2015
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
evp             608953.13k   628667.52k   640411.48k   715339.78k   714495.32k
 

Code (Text):

openssl speed -evp aes128 -multi 4

OpenSSL 1.0.1e-fips 11 Feb 2013
built on: Thu Jul 23 19:06:35 UTC 2015
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
evp             893910.24k  1002266.73k   999148.20k  1035551.74k  1041249.62k
 

Nginx LibreSSL

Code (Text):

/opt/libressl/bin/openssl speed rsa4096 rsa2048 ecdsap256 sha256 sha1 md5 rc4 aes-256-cbc aes-128-cbc -multi 4

LibreSSL 2.2.2
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: information not available
md5             213483.14k   594613.65k  1253302.27k  1719120.21k  1928544.26k
sha1            250248.93k   665154.97k  1236922.62k  1855896.58k  2131995.31k
rc4            1214674.86k  1997689.98k  2346846.72k  2375308.63k  2416372.39k
aes-128 cbc     385637.06k   425217.69k   437167.36k   938727.42k   874283.01k
aes-256 cbc     277579.57k   304266.15k   306230.10k   549419.01k   667461.33k
sha256          160346.17k   332748.33k   575845.38k   707859.80k   713239.21k
                  sign    verify    sign/s verify/s
rsa 2048 bits 0.000394s 0.000012s   2540.0  84219.9
rsa 4096 bits 0.002728s 0.000044s    366.6  22827.3
                              sign    verify    sign/s verify/s
 256 bit ecdsa (nistp256)   0.0000s   0.0002s  25608.6   6507.2
 

Code (Text):

/opt/libressl/bin/openssl speed -evp aes256 -multi 4

LibreSSL 2.2.2
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: information not available
evp             644557.90k   728889.17k   737381.21k   747056.81k   753374.55k
 

Code (Text):

/opt/libressl/bin/openssl speed -evp aes128 -multi 4

LibreSSL 2.2.2
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: information not available
evp             602759.09k   730203.80k   975702.36k   931622.57k   984626.52k
 

 

hmm the above hardware assisted AES-NI (openssl speed -evp aes128/256) numbers look slow for Xeon E5-2680v3 vs Xeon E5-2680v2 I have. About 1/2 the performance compared to Linode Singapore 4GB Xeon E5-2680v3 tests.

Digging deep seems Linode E5-2680v3 has the aes = AES-NI cpu flag disabled on while on older Linode E5-2680v2 aes and thus AES-NI enabled. To confirm my suspicions, on my E5-2680v2 Linode I tested AES-NI enabled and disabled (OPENSSL_ia32cap="~0x200000200000000") and disabled is 1/2 the results of enabled which is what my E5-2680v3 with AES-NI disabled looks like with 1/2 the results/performance.

E5-2680v2

disabled AES-NI

Code (Text):

OPENSSL_ia32cap="~0x200000200000000" openssl speed -evp aes128 -multi 4

OpenSSL 1.0.1e-fips 11 Feb 2013
built on: Thu Jul 23 19:06:35 UTC 2015
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
evp             874571.83k   989803.80k  1006499.84k  1072211.63k  1010963.80k
 

enabled AES-NI on E5-2680v2

Code (Text):

openssl speed -evp aes128 -multi 4

OpenSSL 1.0.1e-fips 11 Feb 2013
built on: Thu Jul 23 19:06:35 UTC 2015
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
evp            2010482.39k  2147274.18k  2176253.61k  2184024.41k  2188099.58k
 

This Fremont Linode Xeon E5-2680v3

Code (Text):

openssl speed -evp aes128 -multi 4

OpenSSL 1.0.1e-fips 11 Feb 2013
built on: Thu Jul 23 19:06:35 UTC 2015
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
evp             893910.24k  1002266.73k   999148.20k  1035551.74k  1041249.62k
 

The old benchmarked Singapore Linode Xeon E5-2680v3 twice as fast ! Unfortunately, I didn't jot down the cpu flags on that instance

Code (Text):

openssl speed -evp aes128 -multi 4
OpenSSL 1.0.1e-fips 11 Feb 2013
built on: Thu Jul 23 19:06:35 UTC 2015
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
evp            2019892.87k  2145538.99k  2181736.53k  2188356.95k  2195144.70k

Intel Xeon E5-2680v2 with aes

Code (Text):

processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 62
model name      : Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz
stepping        : 4
microcode       : 0x1
cpu MHz         : 2799.996
cache size      : 25600 KB
physical id     : 3
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 3
initial apicid  : 3
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl eagerfpu pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm fsgsbase tsc_adjust smep erms xsaveopt
bugs            :
bogomips        : 5602.32
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

Intel Xeon E5-2680v3 without aes

Code (Text):

processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 63
model name      : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping        : 2
microcode       : 0x1
cpu MHz         : 2499.994
cache size      : 30720 KB
physical id     : 3
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 3
initial apicid  : 3
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer xsave avx f16c rdrand hypervisor lahf_lm abm fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt
bugs            :
bogomips        : 5001.32
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

So double check non hardware assisted openssl performance

Per Clock Numbers for RSA 2048bit

• E5-2680v2 @2.80Ghz for rsa 2048bits sign/s = 2567.3/2.80 = 918.89/Ghz
• E5-2680v2 @2.80Ghz for rsa 2048bits verify/s = 82617.3/2.80 = 29,506.18/Ghz
• E5-2680v3 @2.50Ghz for rsa 2048bits sign/s = 2494.7/2.50 = 997.88/Ghz ~8.596% faster clock for clock
• E5-2680v3 @2.50Ghz for rsa 2048bits verify/s = 82174.0/2.50 = 32,869.60/Ghz ~11.399% faster clock for clock

Per Clock Numbers for 256 bit ecdsa (nistp256)

• E5-2680v2 @2.80Ghz for 256 bit ecdsa (nistp256) sign/s = 27729.9/2.80 = 8,903.54/Ghz
• E5-2680v2 @2.80Ghz for 256 bit ecdsa (nistp256) verify/s = 6760.2/2.80 = 2,414.36/Ghz
• E5-2680v3 @2.50Ghz for 256 bit ecdsa (nistp256) sign/s = 25917.8/2.50 = 10,367.12/Ghz ~16.438% faster clock for clock
• E5-2680v3 @2.50Ghz for 256 bit ecdsa (nistp256) verify/s = 6405.3/2.50 = 2,562.12/Ghz ~6.12% faster clock for clock

Intel Xeon E5-2680v2

Code (Text):

/opt/libressl/bin/openssl speed rsa4096 rsa2048 ecdsap256 sha256 sha1 md5 rc4 aes-256-cbc aes-128-cbc -multi 4

built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: information not available
hmac(md5)       210078.18k   588272.55k  1244640.74k  1748776.62k  2040837.46k
rmd160          236242.94k   664017.96k  1439561.81k  1973316.27k  2142803.29k
des cbc        1338620.56k  2080736.85k  2380065.88k  2492318.04k  2509821.27k
aes-192 cbc     382207.38k   418996.52k   428394.50k   940881.24k   953196.54k
camellia-128 cbc   289790.92k   308905.86k   312896.51k   692934.66k   694386.69k
sha512          168621.55k   364070.93k   610627.07k   746643.80k   799885.99k
                  sign    verify    sign/s verify/s
rsa 2048 bits 0.000390s 0.000012s   2567.3  82617.3
rsa 4096 bits 0.002782s 0.000044s    359.4  22700.9
                              sign    verify    sign/s verify/s
256 bit ecdsa (nistp256)   0.0000s   0.0001s  27729.9   6760.2
 

Intel Xeon E5-2680v3

Code (Text):

/opt/libressl/bin/openssl speed rsa4096 rsa2048 ecdsap256 sha256 sha1 md5 rc4 aes-256-cbc aes-128-cbc -multi 4

built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: information not available
md5             174053.69k   528170.07k  1116595.88k  1712696.66k  1813757.95k
sha1            235140.94k   588240.13k  1151584.43k  1567105.75k  2023964.67k
rc4            1167645.38k  1874711.38k  2098178.82k  2306176.00k  2157270.36k
aes-128 cbc     360273.38k   401504.58k   413118.55k   894954.15k   870899.71k
aes-256 cbc     267898.60k   289197.44k   297658.45k   648947.03k   645780.82k
sha256          141769.35k   316525.53k   540710.92k   672106.50k   699247.27k
                  sign    verify    sign/s verify/s
rsa 2048 bits 0.000401s 0.000012s   2494.7  82174.0
rsa 4096 bits 0.002813s 0.000045s    355.5  22272.6
                              sign    verify    sign/s verify/s
256 bit ecdsa (nistp256)   0.0000s   0.0002s  25917.8   6405.3
 

Looks like Xeon E5-2680v3 is slightly slower than older Xeon E5-2680v2 due to the clock speeds 2.50Ghz vs 2.80Ghz and without AES-NI hardware assisted performance on Xeon E5-2680v3 the aes128/256 numbers are are down too

Submitted a ticket to Linode so they're investigating now

Edit: Seems i mixed up the numbers at Benchmarks - DigitalOcean vs Linode vs Vultr vs Dediserve - London vs Singapore VPS benchmarks | Centmin Mod Community Singapore E5-2680v3 KVM column was switched with London E5-2680v2 KVM so looks like aes wasn't on the Singapore E5-2680v3

I'm still hoping aes can be enabled on E5-2680v3 KVM

 

More benchmarks with LibreSSL 2.2.x supported chacha20_poly1305 ciphers if you have it configured for your in your SSL certificates cipher preference for Centmin Mod Nginx



On Linode E5-2680v2 @2.80Ghz KVM

Code (Text):

/opt/libressl/bin/openssl speed -evp chacha -multi 4

LibreSSL 2.2.2
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: information not available
sha256          658692.08k   865413.18k   870826.67k  1125912.58k  1071887.70k

On Linode E5-2680v3 @2.50Ghz KVM you see the Haswell-EP E5 v3 clock for clock is faster even with 0.30Ghz slower processor clock speed it nearly matches the E5-2680v2.

• E5-2680v2 @2.80Ghz = 1071887.70/2.80 = 382,817k per Ghz
• E5-2680v3 @2.50Ghz = 1031132.50/2.50 = 412,453k per Ghz = ~7.74% faster clock for clock

Code (Text):

/opt/libressl/bin/openssl speed -evp chacha -multi 4

LibreSSL 2.2.2
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: information not available
evp             621252.70k   978952.62k  1026608.38k  1062226.60k  1031132.50k
 

Xeon E3-1245v2 @3.40Ghz Compare

To put it in perspective. This is SoYouStart E3-1245v2 Ivy Bridge server with 32GB and cpu clocked @3.40Ghz

for 4 threads

Code (Text):

/opt/libressl/bin/openssl speed -evp chacha -multi 4

LibreSSL 2.2.2
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: information not available
sha256          789225.70k  1297801.77k  1396448.17k  1420022.78k  1433127.59k

Code (Text):

openssl speed -evp aes128 -multi 4

OpenSSL 1.0.1e-fips 11 Feb 2013
built on: Thu Jul 23 19:06:35 UTC 2015
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
evp            2464379.08k  2621100.48k  2663480.92k  2674247.68k  2684134.44k
 

for all 8 cpu threads

Code (Text):

/opt/libressl/bin/openssl speed -evp chacha -multi 8

LibreSSL 2.2.2
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: information not available
sha256          824958.46k  1385365.89k  1524375.30k  1649299.46k  1647613.27k
 

Code (Text):

openssl speed -evp aes128 -multi 8

OpenSSL 1.0.1e-fips 11 Feb 2013
built on: Thu Jul 23 19:06:35 UTC 2015
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
evp            3635260.68k  4785832.17k  5178178.13k  5299851.95k  5326316.11k
 

Xeon E3-1230v2 @3.30Ghz

TheSkySpace OpenVZ VPS 4 cpu cores

for 4 threads

Code (Text):

/opt/libressl/bin/openssl speed -evp chacha -multi 4

LibreSSL 2.2.2
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: information not available
evp             738729.03k   837507.52k  1174202.45k  1249839.10k  1252786.18k

Code (Text):

openssl speed -evp aes128 -multi 4

OpenSSL 1.0.1e-fips 11 Feb 2013
built on: Mon Jun 29 12:45:07 UTC 2015
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
evp            2228085.98k  2469053.59k  2549819.39k  2577706.67k  2583388.16k

Xeon E5-2643v3 @3.4Ghz OpenVZ

Wable OpenVZ 2 cpu thread VPS using Xeon E5-2643v3 @3.40Ghz

for 2 threads

Code (Text):

/opt/libressl/bin/openssl speed -evp chacha -multi 2

LibreSSL 2.2.2
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: information not available
evp             422045.00k   697088.60k   771803.73k   790486.70k   803400.36k

Code (Text):

openssl speed -evp aes128 -multi 2

OpenSSL 1.0.1e-fips 11 Feb 2013
built on: Mon Jun 29 12:45:07 UTC 2015
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
evp             641021.59k   700798.27k   708422.83k   719146.33k   719366.83k

Dual Xeon E5-2660v1 Compare

Dediserve Cloud VPS Xen based Dual Xeon E5-2660v1 Sandy Bridge-EP

for 4 threads

Code (Text):

/opt/libressl/bin/openssl speed -evp chacha -multi 4

LibreSSL 2.2.2
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: information not available
evp             542480.59k   836499.41k   882573.40k   888890.37k   896197.93k
 

Code (Text):

openssl speed -evp aes128 -multi 4

OpenSSL 1.0.1e-fips 11 Feb 2013
built on: Mon Jun 15 18:29:40 UTC 2015
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
evp            1881735.82k  2042498.47k  2089364.57k  2080492.20k  2083613.35k
 

for 8 threads

Code (Text):

/opt/libressl/bin/openssl speed -evp chacha -multi 8

LibreSSL 2.2.2
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: information not available
evp            1074710.82k  1673635.97k  1607687.77k  1662743.94k  1535311.87k
 

Code (Text):

openssl speed -evp aes128 -multi 8

OpenSSL 1.0.1e-fips 11 Feb 2013
built on: Mon Jun 15 18:29:40 UTC 2015
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
evp            3627881.75k  4068366.34k  4157716.99k  4193532.25k  4202730.84k
 

for 32 threads

Code (Text):

/opt/libressl/bin/openssl speed -evp chacha -multi 32

LibreSSL 2.2.2
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: information not available
evp            1911174.46k  2747128.31k  3081352.20k  3001818.42k  3123119.56k
 

Code (Text):

openssl speed -evp aes128 -multi 32

OpenSSL 1.0.1e-fips 11 Feb 2013
built on: Mon Jun 15 18:29:40 UTC 2015
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
evp            8414335.75k 10830233.06k 12614132.44k 13173155.15k 11522091.65k
 

 

Linode has started working on enabling AES-NI support in KVM Linodes with E5-2680v3 across all of it's datacenters. They were kind enough to get me migrated to a Fremont, Linode KVM 4GB VPS with Xeon E5-2680v3 with aes cpu flag and AES-NI support enabled

Code (Text):

processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 63
model name      : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping        : 2
microcode       : 0x1
cpu MHz         : 2499.996
cache size      : 30720 KB
physical id     : 3
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 3
initial apicid  : 3
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt
bugs            :
bogomips        : 5001.32
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

Migration only took 11 mins 14 seconds !

Code (Text):

dd if=/dev/zero of=sb-io-test bs=128k count=1k conv=fdatasync
1024+0 records in
1024+0 records out
134217728 bytes (134 MB) copied, 0.215259 s, 624 MB/s
 

Code (Text):

dd if=/dev/zero of=sb-io-test bs=8k count=16k conv=fdatasync
16384+0 records in
16384+0 records out
134217728 bytes (134 MB) copied, 0.174479 s, 769 MB/s
 

Code (Text):

dd if=/dev/zero of=sb-io-test bs=128k count=1k oflag=dsync
1024+0 records in
1024+0 records out
134217728 bytes (134 MB) copied, 1.05194 s, 128 MB/s
 

Code (Text):

dd if=/dev/zero of=sb-io-test bs=8k count=16k oflag=dsync
16384+0 records in
16384+0 records out
134217728 bytes (134 MB) copied, 8.21284 s, 16.3 MB/s
 

Code (Text):

openssl speed rsa4096 rsa2048 ecdsap256 sha256 sha1 md5 rc4 aes-256-cbc aes-128-cbc -multi 4

OpenSSL 1.0.1e-fips 11 Feb 2013
built on: Thu Jul 23 19:06:35 UTC 2015
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
md5             146684.07k   487155.09k   838853.38k  1399870.12k  1794842.60k
sha1            146335.33k   441458.97k  1061914.37k  1505390.93k  1834241.85k
rc4            1118019.09k  1474061.46k  1627649.19k  1696505.51k  2122274.13k
aes-128 cbc     357156.50k   409691.65k   402043.05k   867563.52k   887392.94k
aes-256 cbc     270927.02k   285262.06k   293319.17k   657434.28k   600336.14k
sha256          139370.13k   320929.26k   552451.16k   688924.33k   706032.98k
                  sign    verify    sign/s verify/s
rsa 2048 bits 0.000396s 0.000013s   2523.1  75315.0
rsa 4096 bits 0.003331s 0.000045s    300.2  22225.7
                              sign    verify    sign/s verify/s
256 bit ecdsa (nistp256)   0.0000s   0.0001s  28452.8   7304.7
 

Yup with aes cpu flag properly enabled in E5-2680v3, doubled the AES-NI performance !

Code (Text):

openssl speed -evp aes256 -multi 4

OpenSSL 1.0.1e-fips 11 Feb 2013
built on: Thu Jul 23 19:06:35 UTC 2015
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
evp            1452515.14k  1502223.42k  1574510.17k  1437445.46k  1532283.56k
 

Code (Text):

openssl speed -evp aes128 -multi 4

OpenSSL 1.0.1e-fips 11 Feb 2013
built on: Thu Jul 23 19:06:35 UTC 2015
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
evp            1966487.93k  2042708.89k  2139913.30k  2215458.82k  2099879.94k
 

Code (Text):

/opt/libressl/bin/openssl speed rsa4096 rsa2048 ecdsap256 sha256 sha1 md5 rc4 aes-256-cbc aes-128-cbc -multi 4

LibreSSL 2.2.3
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: information not available
md5             214818.96k   546131.69k  1037765.00k  1362558.98k  1478959.10k
sha1            234327.22k   686784.53k  1329452.54k  1303103.58k  1749073.92k
rc4            1063876.19k  1843306.60k  2152255.06k  2186817.88k  2274858.33k
aes-128 cbc     329793.79k   357214.12k   355300.61k   658999.57k   717018.45k
aes-256 cbc     264278.04k   295578.33k   298894.93k   652134.25k   663814.14k
sha256          153906.39k   326037.82k   538127.96k   687584.94k   709369.86k
                  sign    verify    sign/s verify/s
rsa 2048 bits 0.000403s 0.000015s   2480.0  64532.9
rsa 4096 bits 0.002805s 0.000051s    356.5  19633.7
                              sign    verify    sign/s verify/s
256 bit ecdsa (nistp256)   0.0000s   0.0002s  22221.8   6164.4
 

Again 2x times better AES-NI assisted performance with aes cpu flag enabled

Code (Text):

/opt/libressl/bin/openssl speed -evp aes256 -multi 4

LibreSSL 2.2.3
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: information not available
evp            1243293.03k  1535215.38k  1535450.03k  1515500.89k  1582877.35k
 

Code (Text):

/opt/libressl/bin/openssl speed -evp aes128 -multi 4

LibreSSL 2.2.3
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: information not available
evp            1974199.53k  2014369.62k  2114966.95k  2171014.49k  2199759.53k
 

chacha20 benchmarks also slightly better

Code (Text):

/opt/libressl/bin/openssl speed -evp chacha -multi 4

LibreSSL 2.2.3
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: information not available
evp             641928.77k   986407.94k  1126784.85k  1113495.21k  1171578.88k