SSL Caddy HTTP/2 server & benchmarks

Lets check out Caddy HTTP/2 web server which also integrates Letsencrypt SSL Centmin Mod has plans to integrate OpenLiteSpeed HTTP/2 web server, Apache 2.4 + PHP-FPM and H20 HTTP/2 server. I've already been playing with H2O and OpenLiteSpeed. So I will also evaluate the viability of Caddy integration into Centmin Mod stack

Right now I am using Centmin Mod 123.09beta01le branch's generated free Letsencrypt SSL certificates for both Centmin Mod Nginx and Caddy's HTTP/2 SSL deployment so not using Caddy's Letsencrypt auto setup and deployment.

Caddy Documentation

• Caddy Documentation

Site Links

Set up on my le12.http2ssl.xyz test Letsencrypt server at the following urls and ports. Server is an OpenVZ 1GB VPS with 4 cpu threads using Intel Xeon E3-1230v2 @3.30Ghz

• http://le12.http2ssl.xyz:80 Centmin Mod Nginx 1.9.7 HTTP/1.1
• https://le12.http2ssl.xyz:443 Centmin Mod Nginx 1.9.7 HTTP/2 https
• http://le12.http2ssl.xyz:81 Caddy 0.80 HTTP/1.1
• https://le12.http2ssl.xyz:445 Caddy 0.80 HTTP/2 https

Server Versions

---

Code:

caddy -version
Caddy 0.8

Notes

• OCSP stapling is currently only enabled on Caddy auto generated Letsencrypt SSL certificates [issue]. If you try to define your own SSL certificates for Caddy via tls directive, the OCSP stapling (and Ticket sessions) is disabled.
  
  Code:

  cipherscan le12.http2ssl.xyz:443
  ............................
  Target: le12.http2ssl.xyz:443
  
  prio  ciphersuite                  protocols              pfs                 curves
  1     ECDHE-RSA-CHACHA20-POLY1305  TLSv1.2                ECDH,P-256,256bits  prime256v1
  2     ECDHE-RSA-AES128-GCM-SHA256  TLSv1.2                ECDH,P-256,256bits  prime256v1
  3     ECDHE-RSA-AES256-GCM-SHA384  TLSv1.2                ECDH,P-256,256bits  prime256v1
  4     DHE-RSA-AES128-GCM-SHA256    TLSv1.2                DH,2048bits         None
  5     DHE-RSA-AES256-GCM-SHA384    TLSv1.2                DH,2048bits         None
  6     ECDHE-RSA-AES128-SHA256      TLSv1.2                ECDH,P-256,256bits  prime256v1
  7     ECDHE-RSA-AES128-SHA         TLSv1,TLSv1.1,TLSv1.2  ECDH,P-256,256bits  prime256v1
  8     ECDHE-RSA-AES256-SHA384      TLSv1.2                ECDH,P-256,256bits  prime256v1
  9     ECDHE-RSA-AES256-SHA         TLSv1,TLSv1.1,TLSv1.2  ECDH,P-256,256bits  prime256v1
  10    DHE-RSA-AES128-SHA256        TLSv1.2                DH,2048bits         None
  11    DHE-RSA-AES128-SHA           TLSv1,TLSv1.1,TLSv1.2  DH,2048bits         None
  12    DHE-RSA-AES256-SHA256        TLSv1.2                DH,2048bits         None
  13    DHE-RSA-AES256-SHA           TLSv1,TLSv1.1,TLSv1.2  DH,2048bits         None
  14    AES128-GCM-SHA256            TLSv1.2                None                None
  15    AES256-GCM-SHA384            TLSv1.2                None                None
  16    AES128-SHA256                TLSv1.2                None                None
  17    AES256-SHA256                TLSv1.2                None                None
  18    AES128-SHA                   TLSv1,TLSv1.1,TLSv1.2  None                None
  19    AES256-SHA                   TLSv1,TLSv1.1,TLSv1.2  None                None
  
  Certificate: trusted, 2048 bits, sha256WithRSAEncryption signature
  TLS ticket lifetime hint: 3600
  OCSP stapling: supported
  Cipher ordering: server
  Curves ordering: server - fallback: no
  Server supports secure renegotiation
  Server supported compression methods: NONE
  TLS Tolerance: yes  

  Code:

  cipherscan le12.http2ssl.xyz:445        
  .......................
  Target: le12.http2ssl.xyz:445
  
  prio  ciphersuite                  protocols              pfs                 curves
  1     ECDHE-RSA-AES128-GCM-SHA256  TLSv1.2                ECDH,P-256,256bits  prime256v1,secp384r1,secp521r1
  2     ECDHE-RSA-AES128-SHA         TLSv1,TLSv1.1,TLSv1.2  ECDH,P-256,256bits  prime256v1,secp384r1,secp521r1
  3     ECDHE-RSA-AES256-SHA         TLSv1,TLSv1.1,TLSv1.2  ECDH,P-256,256bits  prime256v1,secp384r1,secp521r1
  4     AES128-SHA                   TLSv1,TLSv1.1,TLSv1.2  None                None
  5     AES256-SHA                   TLSv1,TLSv1.1,TLSv1.2  None                None
  6     ECDHE-RSA-DES-CBC3-SHA       TLSv1,TLSv1.1,TLSv1.2  ECDH,P-256,256bits  prime256v1,secp384r1,secp521r1
  7     DES-CBC3-SHA                 TLSv1,TLSv1.1,TLSv1.2  None                None
  
  Certificate: trusted, 2048 bits, sha256WithRSAEncryption signature
  TLS ticket lifetime hint: None
  OCSP stapling: not supported
  Cipher ordering: server
  Curves ordering: server - fallback: no
  Server supports secure renegotiation
  Server supported compression methods: NONE
  TLS Tolerance: yes

• Default log file format doesn't seem to include a user agent string
  
  Code:

  IP - [07/Dec/2015:00:29:55 +0000] "HEAD / HTTP/1.1" 200 0
  IP - [07/Dec/2015:00:30:22 +0000] "HEAD / HTTP/2.0" 200 0

  Update: there's a hidden user-agent variable too so updated my Caddyfile below.

Curl Headers

For Centmin Mod Nginx

over HTTP/1.1

Code:

curl -I http://le12.http2ssl.xyz:80
HTTP/1.1 200 OK
Server: nginx centminmod
Date: Mon, 07 Dec 2015 00:29:25 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 1832
Last-Modified: Sun, 06 Dec 2015 23:40:18 GMT
Connection: keep-alive
Vary: Accept-Encoding
ETag: "5664c762-728"
Expires: Tue, 08 Dec 2015 00:29:25 GMT
Cache-Control: max-age=86400
Accept-Ranges: bytes

over HTTP/1.1

Code:

curl -I https://le12.http2ssl.xyz:443
HTTP/1.1 200 OK
Server: nginx centminmod
Date: Mon, 07 Dec 2015 00:29:52 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 1832
Last-Modified: Sun, 06 Dec 2015 23:40:18 GMT
Connection: keep-alive
Vary: Accept-Encoding
ETag: "5664c762-728"
Expires: Tue, 08 Dec 2015 00:29:52 GMT
Cache-Control: max-age=86400
Accept-Ranges: bytes

over HTTP/2

Code:

curl --http2 -I https://le12.http2ssl.xyz:443
HTTP/2.0 200
server:nginx
date:Mon, 07 Dec 2015 00:30:19 GMT
content-type:text/html; charset=utf-8
content-length:1832
last-modified:Sun, 06 Dec 2015 23:40:18 GMT
vary:Accept-Encoding
etag:"5664c762-728"
expires:Tue, 08 Dec 2015 00:30:19 GMT
cache-control:max-age=86400
accept-ranges:bytes

For Caddy

over HTTP/1.1

Code:

curl -I http://le12.http2ssl.xyz:81
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=86400
Content-Length: 1832
Content-Type: text/html; charset=utf-8
Last-Modified: Sun, 06 Dec 2015 23:40:18 GMT
Server: Caddy
Date: Mon, 07 Dec 2015 00:29:28 GMT

over HTTP/1.1

Code:

curl -I https://le12.http2ssl.xyz:445
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=86400
Content-Length: 1832
Content-Type: text/html; charset=utf-8
Last-Modified: Sun, 06 Dec 2015 23:40:18 GMT
Server: Caddy
Date: Mon, 07 Dec 2015 00:29:55 GMT

over HTTP/2

Code:

curl --http2 -I https://le12.http2ssl.xyz:445
HTTP/2.0 200
content-length:1832
server:Caddy
cache-control:max-age=86400
last-modified:Sun, 06 Dec 2015 23:40:18 GMT
content-type:text/html; charset=utf-8
accept-ranges:bytes

VPS Specs

Server 1GB OpenVZ using Xeon E3-1230v2

Code:

cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 58
model name      : Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz
stepping        : 9
cpu MHz         : 3300.117
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 0
cpu cores       : 4
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ida arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms
bogomips        : 6600.23
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 58
model name      : Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz
stepping        : 9
cpu MHz         : 3300.117
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 1
cpu cores       : 4
apicid          : 2
initial apicid  : 2
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ida arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms
bogomips        : 6600.23
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 2
vendor_id       : GenuineIntel
cpu family      : 6
model           : 58
model name      : Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz
stepping        : 9
cpu MHz         : 3300.117
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 2
cpu cores       : 4
apicid          : 4
initial apicid  : 4
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ida arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms
bogomips        : 6600.23
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 58
model name      : Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz
stepping        : 9
cpu MHz         : 3300.117
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 3
cpu cores       : 4
apicid          : 6
initial apicid  : 6
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ida arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms
bogomips        : 6600.23
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

 

My Caddyfile at /opt/caddy/Caddyfile

Updated with custom log format that closely resembles Nginx format with added user-agent

Code:

le12.http2ssl.xyz:81 {
    gzip {
        level 6
    }
    browse
    header / {
        Cache-Control "max-age=86400"
    }  
    tls off
    root /home/nginx/domains/le12.http2ssl.xyz/public
    fastcgi / 127.0.0.1:9000 {
        ext   .php
        split .php
        index index.php
    }
    log / /opt/caddy/le12.http2ssl.xyz-access.log "{remote} {when} {method} {uri} {proto} {status} {size} {>User-Agent} {latency}" {
        rotate {
                size 100 # Rotate after 100 MB
                age  14  # Keep log files for 14 days
                keep 10  # Keep at most 10 log files
        }
    }
}

le12.http2ssl.xyz:445 {
    gzip {
        level 6
    }
    browse
    header / {
        Cache-Control "max-age=86400"
    }  
    tls /etc/letsencrypt/live/le12.http2ssl.xyz/fullchain.pem /etc/letsencrypt/live/le12.http2ssl.xyz/privkey.pem {
        protocols tls1.0 tls1.2
        ciphers ECDHE-RSA-AES128-GCM-SHA256 ECDHE-ECDSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-CBC-SHA ECDHE-RSA-AES256-CBC-SHA ECDHE-ECDSA-AES256-CBC-SHA ECDHE-ECDSA-AES128-CBC-SHA RSA-AES128-CBC-SHA RSA-AES256-CBC-SHA ECDHE-RSA-3DES-EDE-CBC-SHA RSA-3DES-EDE-CBC-SHA
    }
    root /home/nginx/domains/le12.http2ssl.xyz/public
    fastcgi / 127.0.0.1:9000 {
        ext   .php
        split .php
        index index.php
    }
    log / /opt/caddy/le12.http2ssl.xyz-access.log "{remote} {when} {method} {uri} {proto} {status} {size} {>User-Agent} {latency}" {
        rotate {
                size 100 # Rotate after 100 MB
                age  14  # Keep log files for 14 days
                keep 10  # Keep at most 10 log files
        }
    }
}

 

testssl tests

Caddy

Code:

testssl le12.http2ssl.xyz:445

Testing protocols (via sockets except TLS 1.2 and SPDY/NPN)

SSLv2      not offered (OK)
SSLv3      not offered (OK)
TLS 1      offered
TLS 1.1    offered
TLS 1.2    offered (OK)
SPDY/NPN   please check manually, server response was ambigious ...

Testing ~standard cipher lists

Null Ciphers                 not offered (OK)
Anonymous NULL Ciphers       not offered (OK)
Anonymous DH Ciphers         not offered (OK)
40 Bit encryption            not offered (OK)
56 Bit encryption            not offered (OK)
Export Ciphers (general)     not offered (OK)
Low (<=64 Bit)               not offered (OK)
DES Ciphers                  not offered (OK)
Medium grade encryption      not offered (OK)
Triple DES Ciphers           offered (NOT ok)
High grade encryption        offered (OK)

Testing (perfect) forward secrecy, (P)FS -- omitting 3DES, RC4 and Null Encryption here

PFS is offered (OK)  ECDHE-RSA-AES256-SHA ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-SHA

Testing server preferences

Has server cipher order?     yes (OK)
Negotiated protocol          TLSv1.2
Negotiated cipher            ECDHE-RSA-AES128-GCM-SHA256, 256 bit ECDH
Cipher order
     TLSv1:     ECDHE-RSA-AES128-SHA ECDHE-RSA-AES256-SHA AES128-SHA AES256-SHA ECDHE-RSA-DES-CBC3-SHA DES-CBC3-SHA
     TLSv1.1:   ECDHE-RSA-AES128-SHA ECDHE-RSA-AES256-SHA AES128-SHA AES256-SHA ECDHE-RSA-DES-CBC3-SHA DES-CBC3-SHA
     TLSv1.2:   ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-SHA ECDHE-RSA-AES256-SHA AES128-SHA AES256-SHA ECDHE-RSA-DES-CBC3-SHA DES-CBC3-SHA
     h2:        ECDHE-RSA-AES128-GCM-SHA256
     h2-14:     ECDHE-RSA-AES128-GCM-SHA256

Testing server defaults (Server Hello)

TLS server extensions (std)  "next protocol" "session ticket" "renegotiation info"
Session Tickets RFC 5077     (none)
SSL Session ID support       yes
Server key size              2048 bit
Signature Algorithm          SHA256 with RSA
Fingerprint / Serial         SHA1 815EB565DAC210C4CECAFE125899374786BFE2D1 / 0195E067A220D2083468AA50F1FAD06977FB
                              SHA256 8804577F5FBC9B2F8A3F71693EE02FA8032B00E58F6EA3F94DA8044728ACE041
Common Name (CN)             "le12.http2ssl.xyz" (works w/o SNI)
subjectAltName (SAN)         "le12.http2ssl.xyz"
Issuer                       "Let's Encrypt Authority X1" ("Let's Encrypt" from "US")
EV cert (experimental)       no
Certificate Expiration       >= 60 days (2015-12-03 17:39 --> 2016-03-02 17:39 +0000)
# of certificates provided   2
Chain of trust (experim.)    "/usr/bin/etc/*.pem" cannot be found / not readable
Certificate Revocation List  --
OCSP URI                     http://ocsp.int-x1.letsencrypt.org/
OCSP stapling                not offered
TLS timestamp                random values, no fingerprinting possible

Testing HTTP header response @ "/"

HTTP Status Code             200 OK
HTTP clock skew              0 sec from localtime
Strict Transport Security    --
Public Key Pinning           --
Server banner                Caddy
Application banner           --
Cookie(s)                    (none issued at "/")
Security headers             --
Reverse Proxy banner         --

Testing vulnerabilities

Heartbleed (CVE-2014-0160)                not vulnerable (OK) (no heartbeat extension)
CCS (CVE-2014-0224)                       not vulnerable (OK)
Secure Renegotiation (CVE-2009-3555)      not vulnerable (OK)
Secure Client-Initiated Renegotiation     not vulnerable (OK)
CRIME, TLS (CVE-2012-4929)                not vulnerable (OK)
BREACH (CVE-2013-3587)                    potentially NOT ok, uses gzip HTTP compression. - only supplied "/" tested
                                           Can be ignored for static pages or if no secrets in the page
POODLE, SSL (CVE-2014-3566)               not vulnerable (OK)
TLS_FALLBACK_SCSV (RFC 7507), experim.    Downgrade attack prevention supported (OK)
FREAK (CVE-2015-0204)                     not vulnerable (OK)
LOGJAM (CVE-2015-4000), experimental      not vulnerable (OK), common primes not checked. See below for any DH ciphers + bit size
BEAST (CVE-2011-3389)                     TLS1: DES-CBC3-SHA AES128-SHA
                                                 AES256-SHA ECDHE-RSA-DES-CBC3-SHA ECDHE-RSA-AES128-SHA
                                                 ECDHE-RSA-AES256-SHA
                                           VULNERABLE -- but also supports higher protocols (possible mitigation): TLSv1.1 TLSv1.2
RC4 (CVE-2013-2566, CVE-2015-2808)        no RC4 ciphers detected (OK)


Testing all 181 locally available ciphers against the server, ordered by encryption strength

Hexcode  Cipher Suite Name (OpenSSL)    KeyExch.   Encryption Bits
-------------------------------------------------------------------------
xc014   ECDHE-RSA-AES256-SHA           ECDH 256   AES        256                                                                                   
x35     AES256-SHA                     RSA        AES        256                                                                                   
xc02f   ECDHE-RSA-AES128-GCM-SHA256    ECDH 256   AESGCM     128                                                                                   
xc013   ECDHE-RSA-AES128-SHA           ECDH 256   AES        128                                                                                   
x2f     AES128-SHA                     RSA        AES        128                                                                                   
xc012   ECDHE-RSA-DES-CBC3-SHA         ECDH 256   3DES       168                                                                                   
x0a     DES-CBC3-SHA                   RSA        3DES       168  

Centmin Mod Nginx

Code:

testssl le12.http2ssl.xyz:443   

Testing protocols (via sockets except TLS 1.2 and SPDY/NPN)

SSLv2      not offered (OK)
SSLv3      not offered (OK)
TLS 1      offered
TLS 1.1    offered
TLS 1.2    offered (OK)
SPDY/NPN   h2, http/1.1 (advertised)

Testing ~standard cipher lists

Null Ciphers                 not offered (OK)
Anonymous NULL Ciphers       not offered (OK)
Anonymous DH Ciphers         not offered (OK)
40 Bit encryption            not offered (OK)
56 Bit encryption            not offered (OK)
Export Ciphers (general)     not offered (OK)
Low (<=64 Bit)               not offered (OK)
DES Ciphers                  not offered (OK)
Medium grade encryption      not offered (OK)
Triple DES Ciphers           not offered (OK)
High grade encryption        offered (OK)

Testing (perfect) forward secrecy, (P)FS -- omitting 3DES, RC4 and Null Encryption here

PFS is offered (OK)  ECDHE-RSA-AES256-GCM-SHA384 DHE-RSA-AES256-GCM-SHA384 DHE-RSA-AES256-SHA256 DHE-RSA-AES256-SHA ECDHE-RSA-CHACHA20-POLY1305 ECDHE-RSA-AES256-SHA384 ECDHE-RSA-AES256-SHA ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-SHA256 DHE-RSA-AES128-GCM-SHA256 DHE-RSA-AES128-SHA256 DHE-RSA-AES128-SHA ECDHE-RSA-AES128-SHA

Testing server preferences

Has server cipher order?     yes (OK)
Negotiated protocol          TLSv1.2
Negotiated cipher            ECDHE-RSA-CHACHA20-POLY1305, 256 bit ECDH
Cipher order
     TLSv1:     ECDHE-RSA-AES128-SHA ECDHE-RSA-AES256-SHA DHE-RSA-AES128-SHA DHE-RSA-AES256-SHA AES128-SHA AES256-SHA
     TLSv1.1:   ECDHE-RSA-AES128-SHA ECDHE-RSA-AES256-SHA DHE-RSA-AES128-SHA DHE-RSA-AES256-SHA AES128-SHA AES256-SHA
     TLSv1.2:   ECDHE-RSA-CHACHA20-POLY1305 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES256-GCM-SHA384 DHE-RSA-AES128-GCM-SHA256 DHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES128-SHA256 ECDHE-RSA-AES128-SHA ECDHE-RSA-AES256-SHA384 ECDHE-RSA-AES256-SHA DHE-RSA-AES128-SHA256 DHE-RSA-AES128-SHA DHE-RSA-AES256-SHA256 DHE-RSA-AES256-SHA AES128-GCM-SHA256 AES256-GCM-SHA384 AES128-SHA256 AES256-SHA256 AES128-SHA AES256-SHA
     h2:        ECDHE-RSA-CHACHA20-POLY1305 DHE-RSA-AES128-GCM-SHA256 DHE-RSA-AES256-GCM-SHA384 DHE-RSA-AES128-SHA256 DHE-RSA-AES128-SHA DHE-RSA-AES256-SHA256 DHE-RSA-AES256-SHA AES128-GCM-SHA256 AES256-GCM-SHA384 AES128-SHA256 AES256-SHA256 AES128-SHA AES256-SHA
     http/1.1:  ECDHE-RSA-CHACHA20-POLY1305 DHE-RSA-AES128-GCM-SHA256 DHE-RSA-AES256-GCM-SHA384 DHE-RSA-AES128-SHA256 DHE-RSA-AES128-SHA DHE-RSA-AES256-SHA256 DHE-RSA-AES256-SHA AES128-GCM-SHA256 AES256-GCM-SHA384 AES128-SHA256 AES256-SHA256 AES128-SHA AES256-SHA

Testing server defaults (Server Hello)

TLS server extensions (std)  "server name" "renegotiation info" "EC point formats" "session ticket" "status request" "next protocol"
Session Tickets RFC 5077     3600 seconds (PFS requires session ticket keys to be rotated <= daily)
SSL Session ID support       yes
Server key size              2048 bit
Signature Algorithm          SHA256 with RSA
Fingerprint / Serial         SHA1 815EB565DAC210C4CECAFE125899374786BFE2D1 / 0195E067A220D2083468AA50F1FAD06977FB
                              SHA256 8804577F5FBC9B2F8A3F71693EE02FA8032B00E58F6EA3F94DA8044728ACE041
Common Name (CN)             "le12.http2ssl.xyz" (CN in response to request w/o SNI: "le10.http2ssl.xyz")
subjectAltName (SAN)         "le12.http2ssl.xyz"
Issuer                       "Let's Encrypt Authority X1" ("Let's Encrypt" from "US")
EV cert (experimental)       no
Certificate Expiration       >= 60 days (2015-12-03 17:39 --> 2016-03-02 17:39 +0000)
# of certificates provided   2
Chain of trust (experim.)    "/usr/bin/etc/*.pem" cannot be found / not readable
Certificate Revocation List  --
OCSP URI                     http://ocsp.int-x1.letsencrypt.org/
OCSP stapling                offered
TLS timestamp                random values, no fingerprinting possible

Testing HTTP header response @ "/"

HTTP Status Code             200 OK
HTTP clock skew              0 sec from localtime
Strict Transport Security    --
Public Key Pinning           --
Server banner                nginx centminmod
Application banner           --
Cookie(s)                    (none issued at "/")
Security headers             --
Reverse Proxy banner         --

Testing vulnerabilities

Heartbleed (CVE-2014-0160)                not vulnerable (OK) (no heartbeat extension)
CCS (CVE-2014-0224)                       not vulnerable (OK)
Secure Renegotiation (CVE-2009-3555)      not vulnerable (OK)
Secure Client-Initiated Renegotiation     not vulnerable (OK)
CRIME, TLS (CVE-2012-4929)                not vulnerable (OK)
BREACH (CVE-2013-3587)                    potentially NOT ok, uses gzip HTTP compression. - only supplied "/" tested
                                           Can be ignored for static pages or if no secrets in the page
POODLE, SSL (CVE-2014-3566)               not vulnerable (OK)
TLS_FALLBACK_SCSV (RFC 7507), experim.    Downgrade attack prevention supported (OK)
FREAK (CVE-2015-0204)                     not vulnerable (OK)
LOGJAM (CVE-2015-4000), experimental      not vulnerable (OK), common primes not checked. See below for any DH ciphers + bit size
BEAST (CVE-2011-3389)                     TLS1: AES128-SHA DHE-RSA-AES128-SHA
                                                 AES256-SHA DHE-RSA-AES256-SHA ECDHE-RSA-AES128-SHA
                                                 ECDHE-RSA-AES256-SHA
                                           VULNERABLE -- but also supports higher protocols (possible mitigation): TLSv1.1 TLSv1.2
RC4 (CVE-2013-2566, CVE-2015-2808)        no RC4 ciphers detected (OK)

Testing all 181 locally available ciphers against the server, ordered by encryption strength

Hexcode  Cipher Suite Name (OpenSSL)    KeyExch.   Encryption Bits
-------------------------------------------------------------------------
xcc13   ECDHE-RSA-CHACHA20-POLY1305    ECDH 256   ChaCha20   256                                                                                   
xc030   ECDHE-RSA-AES256-GCM-SHA384    ECDH 256   AESGCM     256                                                                                   
xc028   ECDHE-RSA-AES256-SHA384        ECDH 256   AES        256                                                                                   
xc014   ECDHE-RSA-AES256-SHA           ECDH 256   AES        256                                                                                   
x9f     DHE-RSA-AES256-GCM-SHA384      DH 2048    AESGCM     256                                                                                   
x6b     DHE-RSA-AES256-SHA256          DH 2048    AES        256                                                                                   
x39     DHE-RSA-AES256-SHA             DH 2048    AES        256                                                                                   
x9d     AES256-GCM-SHA384              RSA        AESGCM     256                                                                                   
x3d     AES256-SHA256                  RSA        AES        256                                                                                   
x35     AES256-SHA                     RSA        AES        256                                                                                   
xc02f   ECDHE-RSA-AES128-GCM-SHA256    ECDH 256   AESGCM     128                                                                                   
xc027   ECDHE-RSA-AES128-SHA256        ECDH 256   AES        128                                                                                   
xc013   ECDHE-RSA-AES128-SHA           ECDH 256   AES        128                                                                                   
x9e     DHE-RSA-AES128-GCM-SHA256      DH 2048    AESGCM     128                                                                                   
x67     DHE-RSA-AES128-SHA256          DH 2048    AES        128                                                                                   
x33     DHE-RSA-AES128-SHA             DH 2048    AES        128                                                                                   
x9c     AES128-GCM-SHA256              RSA        AESGCM     128                                                                                   
x3c     AES128-SHA256                  RSA        AES        128                                                                                   
x2f     AES128-SHA                     RSA        AES        128                                                                                   

 

nghttp2 HTTP/2 client test

nghttp2 test over HTTP/2 connections

Centmin Mod Nginx

Code:

nghttp -nv https://le12.http2ssl.xyz:443
[  0.020] Connected
The negotiated protocol: h2
[  0.057] recv SETTINGS frame <length=18, flags=0x00, stream_id=0>
          (niv=3)
          [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):128]
          [SETTINGS_INITIAL_WINDOW_SIZE(0x04):2147483647]
          [SETTINGS_MAX_FRAME_SIZE(0x05):16777215]
[  0.058] recv WINDOW_UPDATE frame <length=4, flags=0x00, stream_id=0>
          (window_size_increment=2147418112)
[  0.058] send SETTINGS frame <length=12, flags=0x00, stream_id=0>
          (niv=2)
          [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
          [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65535]
[  0.058] send SETTINGS frame <length=0, flags=0x01, stream_id=0>
          ; ACK
          (niv=0)
[  0.058] send PRIORITY frame <length=5, flags=0x00, stream_id=3>
          (dep_stream_id=0, weight=201, exclusive=0)
[  0.058] send PRIORITY frame <length=5, flags=0x00, stream_id=5>
          (dep_stream_id=0, weight=101, exclusive=0)
[  0.058] send PRIORITY frame <length=5, flags=0x00, stream_id=7>
          (dep_stream_id=0, weight=1, exclusive=0)
[  0.058] send PRIORITY frame <length=5, flags=0x00, stream_id=9>
          (dep_stream_id=7, weight=1, exclusive=0)
[  0.058] send PRIORITY frame <length=5, flags=0x00, stream_id=11>
          (dep_stream_id=3, weight=1, exclusive=0)
[  0.058] send HEADERS frame <length=44, flags=0x25, stream_id=13>
          ; END_STREAM | END_HEADERS | PRIORITY
          (padlen=0, dep_stream_id=11, weight=16, exclusive=0)
          ; Open new stream
          :method: GET
          :path: /
          :scheme: https
          :authority: le12.http2ssl.xyz
          accept: */*
          accept-encoding: gzip, deflate
          user-agent: nghttp2/1.5.1-DEV
[  0.073] recv SETTINGS frame <length=0, flags=0x01, stream_id=0>
          ; ACK
          (niv=0)
[  0.074] recv (stream_id=13) :status: 200
[  0.074] recv (stream_id=13) server: nginx
[  0.074] recv (stream_id=13) date: Mon, 07 Dec 2015 00:35:16 GMT
[  0.074] recv (stream_id=13) content-type: text/html; charset=utf-8
[  0.074] recv (stream_id=13) last-modified: Sun, 06 Dec 2015 23:40:18 GMT
[  0.074] recv (stream_id=13) vary: Accept-Encoding
[  0.074] recv (stream_id=13) etag: W/"5664c762-728"
[  0.074] recv (stream_id=13) expires: Tue, 08 Dec 2015 00:35:16 GMT
[  0.074] recv (stream_id=13) cache-control: max-age=86400
[  0.074] recv (stream_id=13) content-encoding: gzip
[  0.074] recv HEADERS frame <length=227, flags=0x04, stream_id=13>
          ; END_HEADERS
          (padlen=0)
          ; First response header
[  0.074] recv DATA frame <length=543, flags=0x01, stream_id=13>
          ; END_STREAM
[  0.074] send GOAWAY frame <length=8, flags=0x00, stream_id=0>
          (last_stream_id=0, error_code=NO_ERROR(0x00), opaque_data(0)=[])

Caddy

Code:

nghttp -nv https://le12.http2ssl.xyz:445
[  0.035] Connected
The negotiated protocol: h2
[  0.072] send SETTINGS frame <length=12, flags=0x00, stream_id=0>
          (niv=2)
          [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
          [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65535]
[  0.072] send PRIORITY frame <length=5, flags=0x00, stream_id=3>
          (dep_stream_id=0, weight=201, exclusive=0)
[  0.072] send PRIORITY frame <length=5, flags=0x00, stream_id=5>
          (dep_stream_id=0, weight=101, exclusive=0)
[  0.072] send PRIORITY frame <length=5, flags=0x00, stream_id=7>
          (dep_stream_id=0, weight=1, exclusive=0)
[  0.072] send PRIORITY frame <length=5, flags=0x00, stream_id=9>
          (dep_stream_id=7, weight=1, exclusive=0)
[  0.072] send PRIORITY frame <length=5, flags=0x00, stream_id=11>
          (dep_stream_id=3, weight=1, exclusive=0)
[  0.072] send HEADERS frame <length=47, flags=0x25, stream_id=13>
          ; END_STREAM | END_HEADERS | PRIORITY
          (padlen=0, dep_stream_id=11, weight=16, exclusive=0)
          ; Open new stream
          :method: GET
          :path: /
          :scheme: https
          :authority: le12.http2ssl.xyz:445
          accept: */*
          accept-encoding: gzip, deflate
          user-agent: nghttp2/1.5.1-DEV
[  0.085] recv SETTINGS frame <length=18, flags=0x00, stream_id=0>
          (niv=3)
          [SETTINGS_MAX_FRAME_SIZE(0x05):1048576]
          [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):250]
          [SETTINGS_MAX_HEADER_LIST_SIZE(0x06):1048896]
[  0.085] recv SETTINGS frame <length=0, flags=0x01, stream_id=0>
          ; ACK
          (niv=0)
[  0.085] send SETTINGS frame <length=0, flags=0x01, stream_id=0>
          ; ACK
          (niv=0)
[  0.086] recv (stream_id=13) :status: 200
[  0.086] recv (stream_id=13) vary: Accept-Encoding
[  0.086] recv (stream_id=13) cache-control: max-age=86400
[  0.086] recv (stream_id=13) last-modified: Sun, 06 Dec 2015 23:40:18 GMT
[  0.086] recv (stream_id=13) content-type: text/html; charset=utf-8
[  0.086] recv (stream_id=13) accept-ranges: bytes
[  0.086] recv (stream_id=13) server: Caddy
[  0.086] recv (stream_id=13) content-encoding: gzip
[  0.086] recv (stream_id=13) content-length: 548
[  0.086] recv HEADERS frame <length=92, flags=0x04, stream_id=13>
          ; END_HEADERS
          (padlen=0)
          ; First response header
[  0.086] recv DATA frame <length=548, flags=0x01, stream_id=13>
          ; END_STREAM
[  0.087] send GOAWAY frame <length=8, flags=0x00, stream_id=0>
          (last_stream_id=0, error_code=NO_ERROR(0x00), opaque_data(0)=[])

nghttp2 statistics

HTTP/2 connection timings

for Centmin Mod Nginx

Code:

nghttp -nas https://le12.http2ssl.xyz:443
***** Statistics *****

Request timing:
  responseEnd: the  time  when  last  byte of  response  was  received
               relative to connectEnd
requestStart: the time  just before  first byte  of request  was sent
               relative  to connectEnd.   If  '*' is  shown, this  was
               pushed by server.
      process: responseEnd - requestStart
         code: HTTP status code
         size: number  of  bytes  received as  response  body  without
               inflation.
          URI: request URI

see http://www.w3.org/TR/resource-timing/#processing-model

sorted by 'complete'

id  responseEnd requestStart  process code size request path
13    +16.95ms       +351us  16.60ms  200  543 /

for Caddy

Code:

nghttp -nas https://le12.http2ssl.xyz:445
***** Statistics *****

Request timing:
  responseEnd: the  time  when  last  byte of  response  was  received
               relative to connectEnd
requestStart: the time  just before  first byte  of request  was sent
               relative  to connectEnd.   If  '*' is  shown, this  was
               pushed by server.
      process: responseEnd - requestStart
         code: HTTP status code
         size: number  of  bytes  received as  response  body  without
               inflation.
          URI: request URI

see http://www.w3.org/TR/resource-timing/#processing-model

sorted by 'complete'

id  responseEnd requestStart  process code size request path
13    +16.97ms       +118us  16.85ms  200  548 /

 

Pingdom FPT Tests

Pingdom FPT uses Chrome 39 browser so HTTP/2 isn't supported so these tests would of fallen back to HTTP/1.1

Centmin Mod Nginx HTTPS over Chrome 39 which doesn't support HTTP/2


Caddy HTTPS over Chrome 39 which doesn't support HTTP/2

 

Webpagetest 3G Mobile Dulles

The test server is in Los Angeles, so using a worse case situation where test WPT location is on other side of USA in Dulles, VA with 3G Mobile Fast connection speeds and Chrome.




Nginx served page is slightly larger at 8KB vs Caddy 7KB maybe due to some additional headers in Nginx ? Both Nginx and Caddy are set to gzip level 6 compression so maybe differences in gzip implementation ?

 

KeyCDN WebSite Speed Test

Using KeyCDN WebSite Speed Test

For Centmin Mod Nginx over HTTP/2


For Caddy over HTTP/2

 

Blitz.io Benchmarks 1000 concurrent users

Blitz.io 1000 user testing results

For Centmin Mod Nginx HTTP/2 where nginx worker_processes = 2





For Caddy HTTP/2

 

Blitz.io Benchmarks 2000 concurrent users

Blitz.io 2000 user testing results

For Centmin Mod Nginx HTTP/2 where nginx worker_processes = 2





For Caddy HTTP/2

 

HTTP/2 Load Testing via h2load client

Using my Docker based nghttp2 client/library image tool box, I ran a h2load HTTP/2 client tests against Centmin Mod Nginx HTTP/2 on port 443 and Caddy 0.80 on port 445. Sites are using Letsencrypt issued SSL certificates for both Nginx and Caddy servers

h2load tests summary for single 1 h2load threads

h2load test defaults to using 1 thread for testing, so below results are for single thread based h2load load testing.

10 concurrent users, 10 requests

• Centmin Mod Nginx HTTP/2 = 126.76 req/s, 263.42KB/s
• Caddy HTTP/2 = 95.01 req/s, 181.85KB/s

100 concurrent users, 100 requests

• Centmin Mod Nginx HTTP/2 = 536.59 req/s, 1.09MB/s
• Caddy HTTP/2 = 220.95 req/s, 422.91KB/s

250 concurrent users, 1000 requests

• Centmin Mod Nginx HTTP/2 = 2439.27 req/s, 4.86MB/s
• Caddy HTTP/2 = 1049.99 req/s, 1.89MB/s

1000 concurrent users, 50000 requests

• Centmin Mod Nginx HTTP/2 = 15580.02 req/s, 30.90MB/s
• Caddy HTTP/2 = 6139.60 req/s, 10.89MB/s

Baseline ping times from h2load invoked server (Digital Ocean San Francisco KVM VPS) to le12.http2ssl.xyz OpenVZ VPS in Los Angeles

Code:

ping -c4 le12.http2ssl.xyz
PING le12.http2ssl.xyz (104.152.214.220) 56(84) bytes of data.
64 bytes from 220-214-152-104-static.reverse.queryfoundry.net (104.152.214.220): icmp_seq=1 ttl=53 time=12.2 ms
64 bytes from 220-214-152-104-static.reverse.queryfoundry.net (104.152.214.220): icmp_seq=2 ttl=53 time=12.1 ms
64 bytes from 220-214-152-104-static.reverse.queryfoundry.net (104.152.214.220): icmp_seq=3 ttl=53 time=12.1 ms
64 bytes from 220-214-152-104-static.reverse.queryfoundry.net (104.152.214.220): icmp_seq=4 ttl=53 time=12.1 ms

--- le12.http2ssl.xyz ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3004ms
rtt min/avg/max/mdev = 12.127/12.176/12.277/0.059 ms

Centmin Mod Nginx HTTP/2 Tests

h2load test against my Letsencrypt free SSL enabled Centmin Mod Nginx HTTP/2 server at le12.http2ssl.xyz:443

Code:

/usr/local/bin/h2load -c10 -m100 -n10 -v https://le12.http2ssl.xyz:443
starting benchmark...
spawning thread #0: 10 total client(s). 10 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES128-GCM-SHA256
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 78.89ms, 126.76 req/s, 263.42KB/s
requests: 10 total, 10 started, 10 done, 10 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 10 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 21280 bytes total, 2290 bytes headers (space savings 14.23%), 18320 bytes data
                     min         max         mean         sd        +/- sd
time for request:    12.18ms     15.28ms     13.97ms      1.27ms    60.00%
time for connect:    44.10ms     62.14ms     52.38ms      6.90ms    60.00%
time to 1st byte:    56.30ms     77.42ms     66.36ms      8.11ms    60.00%
req/s (client)  :      12.91       17.74       15.28        1.86    60.00%

Caddy HTTP/2 Tests

h2load test against my Letsencrypt free SSL enabled Caddy 0.80 HTTP/2 server at le12.http2ssl.xyz:445

Code:

/usr/local/bin/h2load -c10 -m100 -n10 -v https://le12.http2ssl.xyz:445
starting benchmark...
spawning thread #0: 10 total client(s). 10 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES128-GCM-SHA256
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 105.26ms, 95.01 req/s, 181.85KB/s
requests: 10 total, 10 started, 10 done, 10 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 10 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 19600 bytes total, 740 bytes headers (space savings 54.04%), 18320 bytes data
                     min         max         mean         sd        +/- sd
time for request:    12.46ms     15.64ms     13.57ms      1.07ms    70.00%
time for connect:    71.42ms     88.76ms     78.97ms      5.69ms    70.00%
time to 1st byte:    84.71ms    104.41ms     92.55ms      6.44ms    60.00%
req/s (client)  :       9.57       11.79       10.85        0.74    60.00%

note I had updated Centmin Mod Nginx to 1.9.8 too

h2load 100 user concurrency + 100 requests

Bumping up concurrency from 10 to 100 for h2load tests

Centmin Mod Nginx HTTP/2 Tests

Code:

/usr/local/bin/h2load -c100 -m100 -n100 -v https://le12.http2ssl.xyz:443
starting benchmark...
spawning thread #0: 100 total client(s). 100 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES128-GCM-SHA256
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 186.36ms, 536.59 req/s, 1.09MB/s
requests: 100 total, 100 started, 100 done, 100 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 212800 bytes total, 22900 bytes headers (space savings 14.23%), 183200 bytes data
                     min         max         mean         sd        +/- sd
time for request:    12.82ms     20.79ms     16.61ms      1.63ms    72.00%
time for connect:    44.65ms    162.87ms    105.29ms     35.55ms    54.00%
time to 1st byte:    61.19ms    178.16ms    121.91ms     35.14ms    54.00%
req/s (client)  :       5.61       16.32        9.02        2.97    66.00%

Caddy HTTP/2 Tests

Code:

/usr/local/bin/h2load -c100 -m100 -n100 -v https://le12.http2ssl.xyz:445
starting benchmark...
spawning thread #0: 100 total client(s). 100 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES128-GCM-SHA256
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 452.59ms, 220.95 req/s, 422.91KB/s
requests: 100 total, 100 started, 100 done, 100 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 196000 bytes total, 7400 bytes headers (space savings 54.04%), 183200 bytes data
                     min         max         mean         sd        +/- sd
time for request:    11.53ms    109.40ms     20.61ms     15.39ms    97.00%
time for connect:    66.70ms    437.34ms    274.33ms    100.08ms    51.00%
time to 1st byte:   165.23ms    449.83ms    294.95ms     94.69ms    47.00%
req/s (client)  :       2.22        6.05        3.79        1.27    49.00%

h2load 250 user concurrency + 1000 requests

Bumping up concurrency to 250 + 1000 requests for h2load tests

Centmin Mod Nginx HTTP/2 Tests

Code:

/usr/local/bin/h2load -c250 -m100 -n1000 -v https://le12.http2ssl.xyz:443
starting benchmark...
spawning thread #0: 250 total client(s). 1000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES128-GCM-SHA256
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 409.96ms, 2439.27 req/s, 4.86MB/s
requests: 1000 total, 1000 started, 1000 done, 1000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 1000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 2091250 bytes total, 229000 bytes headers (space savings 14.23%), 1832000 bytes data
                     min         max         mean         sd        +/- sd
time for request:    11.13ms     42.11ms     18.73ms      4.33ms    85.00%
time for connect:    46.62ms    384.38ms    213.22ms     99.91ms    57.20%
time to 1st byte:    62.25ms    399.59ms    231.86ms     99.15ms    57.00%
req/s (client)  :      10.00       64.17       22.20       12.93    82.40%

Caddy HTTP/2 Tests

Code:

/usr/local/bin/h2load -c250 -m100 -n1000 -v https://le12.http2ssl.xyz:445
starting benchmark...
spawning thread #0: 250 total client(s). 1000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES128-GCM-SHA256
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 952.39ms, 1049.99 req/s, 1.89MB/s
requests: 1000 total, 1000 started, 1000 done, 1000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 1000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 1882750 bytes total, 23750 bytes headers (space savings 85.25%), 1832000 bytes data
                     min         max         mean         sd        +/- sd
time for request:    11.63ms    242.21ms     25.68ms     18.26ms    93.80%
time for connect:   134.45ms    928.42ms    610.27ms    203.87ms    49.60%
time to 1st byte:   338.60ms    944.03ms    635.53ms    199.10ms    50.40%
req/s (client)  :       4.24       11.80        7.03        2.44    60.80%

h2load 1000 user concurrency + 50000 requests

Bumping up concurrency to 1000 + 50000 requests for h2load tests

Centmin Mod Nginx HTTP/2 Tests

Code:

/usr/local/bin/h2load -c1000 -m100 -n50000 -v https://le12.http2ssl.xyz:443 
starting benchmark...
spawning thread #0: 1000 total client(s). 50000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES128-GCM-SHA256
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 3.21s, 15580.02 req/s, 30.90MB/s
requests: 50000 total, 50000 started, 50000 done, 50000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 103999000 bytes total, 11450000 bytes headers (space savings 14.23%), 91600000 bytes data
                     min         max         mean         sd        +/- sd
time for request:    15.67ms       2.34s       1.05s    476.85ms    69.38%
time for connect:   532.55ms       2.71s       1.29s    394.21ms    59.00%
time to 1st byte:   810.47ms       2.97s       1.73s    459.23ms    64.60%
req/s (client)  :      15.82       30.53       20.55        5.34    70.80%

Caddy HTTP/2 Tests

Code:

/usr/local/bin/h2load -c1000 -m100 -n50000 -v https://le12.http2ssl.xyz:445
starting benchmark...
spawning thread #0: 1000 total client(s). 50000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES128-GCM-SHA256
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 8.14s, 6139.60 req/s, 10.89MB/s
requests: 50000 total, 50000 started, 50000 done, 50000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 92953000 bytes total, 417000 bytes headers (space savings 94.82%), 91600000 bytes data
                     min         max         mean         sd        +/- sd
time for request:    14.51ms       3.08s       1.18s    827.53ms    52.96%
time for connect:   285.69ms       7.49s       3.86s       2.28s    50.00%
time to 1st byte:   422.20ms       8.02s       4.66s       2.33s    54.70%
req/s (client)  :       6.15      103.05       15.66       19.05    90.60%

 

Caddy Integration Into Centmin Mod LEMP Stack

Just a bit of a preview of what Caddy server integration into Centmin Mod LEMP Stack might look like

Caddy serving Centmin Mod's main hostname virtualhost off of port 8888

Code (Text):

curl -I localhost:8888
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=86400
Content-Length: 3801
Content-Type: text/html; charset=utf-8
Etag: W/"579b144d-ed9"
Last-Modified: Fri, 29 Jul 2016 08:31:09 GMT
Server: Caddy
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block
Date: Sat, 06 Aug 2016 01:41:41 GMT

Caddy commands

Code (Text):

/root/tools/caddy.sh              

Command Usage:
/root/tools/caddy.sh {install|update|uninstall|addsite|start|stop|restart|status|domains|certs|version|help}

Caddy status, stop, start and restart commands

Code (Text):

/root/tools/caddy.sh status
caddy is running (4991)

/root/tools/caddy.sh stop

/root/tools/caddy.sh status
caddy is not running

/root/tools/caddy.sh start

/root/tools/caddy.sh status
caddy is running (5084)

Caddy addsite function

Code (Text):

/root/tools/caddy.sh addsite      

enter domain name wihtout www.: newdomain.com

/home/nginx/domains/newdomain.com/public directory does not exist

--------------------------------------------------------------------
caddy.sh was written for centminmod.com LEMP stack environments and
requires a Centmin Mod Nginx generated vhost directory to work as
Caddy HTTP vhost domains are setup on port 8888 and Caddy HTTPS
vhost domains are setup on port 448 leaving Centmin Mod Nginx to run
on HTTP port 80 and HTTPS port 443
--------------------------------------------------------------------

Generating Caddy site vhosts via addsite with self-signed ssl certs

Code (Text):

/root/tools/caddy.sh addsite      

enter domain name wihtout www.: domain1.com

/etc/caddy/conf.d/domain1.com.conf created on 8888
/etc/caddy/conf.d/domain1.com.ssl.conf created on 448

Code (Text):

curl -I http://domain1.com:8888
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=86400
Content-Length: 1399
Content-Type: text/html; charset=utf-8
Etag: W/"57a542e8-577"
Last-Modified: Sat, 06 Aug 2016 01:52:40 GMT
Server: Caddy
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block
Date: Sat, 06 Aug 2016 05:33:38 GMT

Code (Text):

curl -Ik https://domain1.com:448
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=86400
Content-Length: 1399
Content-Type: text/html; charset=utf-8
Etag: W/"57a542e8-577"
Last-Modified: Sat, 06 Aug 2016 01:52:40 GMT
Server: Caddy
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block
Date: Sat, 06 Aug 2016 05:33:41 GMT

 

Caddy Integration Revised - caddy.sh

Updated caddy.sh integration script so now addsite option can automatically generate the site domain vhost for both Caddy and Nginx servers simultaneously - including issuing of self-signed SSL certificates + Letsencrypt SSL certificates for both Caddy and Nginx to run HTTP/2 based HTTPS sites and generation of pure-ftpd virtual FTP user which can access the web root for site for both Caddy and Nginx servers

Web Server Binary Sizes

Code (Text):

ls -lah $(which nginx)
-rwxr-xr-x 1 root root 3.1M Aug  6 15:44 /usr/local/sbin/nginx

ls -lah $(which caddy)
-rwxrwxr-x 1 nginx nginx 29M Aug  5 04:08 /usr/local/bin/caddy

Command Usage options

Code (Text):

/root/tools/caddy/caddy.sh

Command Usage:
/root/tools/caddy/caddy.sh {install|update|uninstall|addsite|start|stop|restart|status|domains|certs|version|help}

Install run on CentOS 7.2 64bit Virtualbox Instance

Code (Text):

/root/tools/caddy/caddy.sh install
Downloading Caddy for linux/amd64...
https://caddyserver.com/download/build?os=linux&arch=amd64&arm=&features=filemanager%2Cgit%2Chugo%2Cipfilter%2Clocale%2Cmailout%2Cminify%2Cprometheus%2Cratelimit%2Crealip%2Csearch%2Ccloudflare%2Cdigitalocean%2Cgooglecloud%2Cnamecheap%2Croute53%2Cvultr
Extracting...
Putting caddy in /usr/local/bin (may require password)
Caddy 0.9.0
Successfully installed
Server types:
  http

Caddyfile loaders:
  short
  flag
  default

Other plugins:
  http.basicauth
  http.bind
  http.browse
  http.errors
  http.expvar
  http.ext
  http.fastcgi
  http.filemanager
  http.git
  http.gzip
  http.header
  http.hugo
  http.internal
  http.ipfilter
  http.locale
  http.log
  http.mailout
  http.markdown
  http.mime
  http.minify
  http.pprof
  http.prometheus
  http.proxy
  http.ratelimit
  http.realip
  http.redir
  http.rewrite
  http.root
  http.search
  http.templates
  http.websocket
  shutdown
  startup
  tls
  tls.dns.cloudflare
  tls.dns.digitalocean
  tls.dns.googlecloud
  tls.dns.namecheap
  tls.dns.route53
  tls.dns.vultr

Starting Caddy via caddy.sh and checking status

Code (Text):

/root/tools/caddy/caddy.sh status
caddy is not running

Code (Text):

/root/tools/caddy/caddy.sh start
/root/tools/caddy/caddy.sh status
caddy is running (3070)
-----------------------------------------------
Max processes     8192     8192     processes
Max open files    262144   262144   files
-----------------------------------------------

Code (Text):

ps aufxw | grep caddy | grep -v grep
root      3070  0.0  0.4  24552  7864 pts/2    Sl   19:16   0:00 /usr/local/bin/caddy -log /usr/local/nginx/logs/caddy.log -quiet=true -agree=true -conf=/etc/caddy/Caddyfile -root=/var/tmp -pidfile=/var/run/caddy/caddy.pid

caddy.sh addsite usage

Simultaneous Caddy + Nginx vhost generation with self-signed SSL certificates for both servers for domain1.com testing. This will allow me to properly do side by side comparison benchmarks of Caddy and Centmin Mod's Nginx server

Code (Text):

/root/tools/caddy/caddy.sh addsite

Add a new Caddy site domain vhost
Enter domain name wihtout www.: domain1.com

Caddy integration into Centmin Mod LEMP runs Nginx on port 80 & 443
Caddy server runs on 8888 & 448
Do you want to create the Nginx vhost for domain1.com as well ? [y/n]: y
Create self-signed SSL cert for Nginx vhost domain1.com ? recommended [y/n]: y

-------------------------------------------------------------
Generating Nginx vhost for domain1.com
-------------------------------------------------------------

-------------------------------------------------------------
/usr/bin/nv -d domain1.com -s y -u ftpusername
-------------------------------------------------------------

---------------------------------------------------------------
Nginx Vhost Setup...
---------------------------------------------------------------


FTP password auto generated: **************

Password:
Enter it again:
---------------------------------------------------------------
SSL Vhost Setup...
---------------------------------------------------------------

---------------------------------------------------------------
Generating self signed SSL certificate...
CSR file can also be used to be submitted for paid SSL certificates
If using for paid SSL certificates be sure to keep both private key and CSR safe
creating CSR File: domain1.com.csr
creating private key: domain1.com.key
creating self-signed SSL certificate: domain1.com.crt
Generating a 2048 bit RSA private key
......................................................................+++
........................+++
writing new private key to 'domain1.com.key'
-----
No value provided for Subject Attribute C, skipped
No value provided for Subject Attribute ST, skipped
No value provided for Subject Attribute L, skipped
Signature ok
subject=/O=domain1.com/OU=domain1.com/CN=domain1.com
Getting Private key

---------------------------------------------------------------
Generating backup CSR and private key for HTTP Public Key Pinning...
creating CSR File: domain1.com-backup.csr
creating private key: domain1.com-backup.key
Generating a 2048 bit RSA private key
..................................................................................................+++
..............+++
writing new private key to 'domain1.com-backup.key'
-----
No value provided for Subject Attribute C, skipped
No value provided for Subject Attribute ST, skipped
No value provided for Subject Attribute L, skipped

---------------------------------------------------------------
Extracting Base64 encoded information for primary and secondary
private key's SPKI - Subject Public Key Information
Primary private key - domain1.com.key
Backup private key - domain1.com-backup.key
For HPKP - HTTP Public Key Pinning hash generation...

extracting SPKI Base64 encoded hash for primary private key = domain1.com.key ...
writing RSA key
urcdM0QCH7avbVN0dvPZG70I7Y6nWkTpcQ0V7af0+2s=

extracting SPKI Base64 encoded hash for backup private key = domain1.com-backup.key ...
writing RSA key
fpNNHgiU21nnUXd5ZLMaew0MjEcDXedaezrQzSGHLHg=

HTTP Public Key Pinning Header for Nginx

for 7 days max-age including subdomains

add_header Public-Key-Pins 'pin-sha256="urcdM0QCH7avbVN0dvPZG70I7Y6nWkTpcQ0V7af0+2s="; pin-sha256="fpNNHgiU21nnUXd5ZLMaew0MjEcDXedaezrQzSGHLHg="; max-age=86400; includeSubDomains';

for 7 days max-age excluding subdomains

add_header Public-Key-Pins 'pin-sha256="urcdM0QCH7avbVN0dvPZG70I7Y6nWkTpcQ0V7af0+2s="; pin-sha256="fpNNHgiU21nnUXd5ZLMaew0MjEcDXedaezrQzSGHLHg="; max-age=86400';

---------------------------------------------------------------
Generating dhparam.pem file - can take a few minutes...
Generating DH parameters, 2048 bit long safe prime, generator 2
This is going to take a long time
.........+..............++*++*
dhparam file generation time: 113.662816389

-------------------------------------------------------------

-------------------------------------------------------------
/usr/local/src/centminmod/tools/autoprotect.sh
generated nginx include file: /usr/local/nginx/conf/autoprotect/domain1.com/autoprotect-domain1.com.conf

autoprotect.sh run completed...

Restarting nginx (via systemctl):  [  OK  ]
Restarting nginx (via systemctl):  [  OK  ]
systemctl restart pure-ftpd.service

-------------------------------------------------------------
FTP hostname : IPADDR
FTP port : 21
FTP mode : FTP (explicit SSL)
FTP Passive (PASV) : ensure is checked/enabled
FTP username created for domain1.com : ftpusername
FTP password created for domain1.com : **************
-------------------------------------------------------------
vhost for domain1.com created successfully

domain: http://domain1.com
vhost conf file for domain1.com created: /usr/local/nginx/conf/conf.d/domain1.com.conf

vhost ssl for domain1.com created successfully

domain: https://domain1.com
vhost ssl conf file for domain1.com created: /usr/local/nginx/conf/conf.d/domain1.com.ssl.conf
/usr/local/nginx/conf/ssl_include.conf created
Self-signed SSL Certificate: /usr/local/nginx/conf/ssl/domain1.com/domain1.com.crt
SSL Private Key: /usr/local/nginx/conf/ssl/domain1.com/domain1.com.key
SSL CSR File: /usr/local/nginx/conf/ssl/domain1.com/domain1.com.csr
Backup SSL Private Key: /usr/local/nginx/conf/ssl/domain1.com/domain1.com-backup.key
Backup SSL CSR File: /usr/local/nginx/conf/ssl/domain1.com/domain1.com-backup.csr

upload files to /home/nginx/domains/domain1.com/public
vhost log files directory is /home/nginx/domains/domain1.com/log

-------------------------------------------------------------
Current vhost listing at: /usr/local/nginx/conf/conf.d/

    
Aug 6   15:44   845    ssl.conf
Aug 6   15:44   1.1K   demodomain.com.conf
Aug 6   15:52   1.6K   virtual.conf
Aug 6   19:20   2.0K   domain1.com.conf
Aug 6   19:20   3.8K   domain1.com.ssl.conf

-------------------------------------------------------------
Current vhost ssl files listing at: /usr/local/nginx/conf/ssl/domain1.com

    
Aug 6   19:18   1.7K   domain1.com.key
Aug 6   19:18   952    domain1.com.csr
Aug 6   19:18   1.1K   domain1.com.crt
Aug 6   19:18   1.7K   domain1.com-backup.key
Aug 6   19:18   952    domain1.com-backup.csr
Aug 6   19:18   45     hpkp-info-primary-pin.txt
Aug 6   19:18   45     hpkp-info-secondary-pin.txt
Aug 6   19:20   424    dhparam.pem

-------------------------------------------------------------
Commands to remove domain1.com

pure-pw userdel ftpusername
rm -rf /usr/local/nginx/conf/conf.d/domain1.com.conf
rm -rf /usr/local/nginx/conf/conf.d/domain1.com.ssl.conf
rm -rf /usr/local/nginx/conf/ssl/domain1.com/domain1.com.crt
rm -rf /usr/local/nginx/conf/ssl/domain1.com/domain1.com.key
rm -rf /usr/local/nginx/conf/ssl/domain1.com/domain1.com.csr
rm -rf /usr/local/nginx/conf/ssl/domain1.com
rm -rf /home/nginx/domains/domain1.com
service nginx restart

-------------------------------------------------------------
vhost for domain1.com setup successfully
domain1.com setup info log saved at:
/root/centminlogs/centminmod_060816-191811_nginx_addvhost_nv.log
-------------------------------------------------------------


/etc/caddy/conf.d/domain1.com.conf created on 8888
/etc/caddy/conf.d/domain1.com.ssl.conf created on 448

caddy restarted and is running (3479)
-----------------------------------------------
Max processes     8192     8192     processes
Max open files    262144   262144   files
-----------------------------------------------

curl domain1.com headers for Caddy and Nginx HTTP and HTTPS sites

For HTTP on port 80 and port 8888 for Nginx and Caddy respectively

Code (Text):

curl -I http://domain1.com
HTTP/1.1 200 OK
Date: Sat, 06 Aug 2016 19:24:33 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 1362
Last-Modified: Sat, 06 Aug 2016 19:18:11 GMT
Connection: keep-alive
ETag: "57a637f3-552"
Server: nginx centminmod
X-Powered-By: centminmod
Expires: Sun, 07 Aug 2016 19:24:33 GMT
Cache-Control: max-age=86400
Cache-Control: public, must-revalidate, proxy-revalidate
Accept-Ranges: bytes

Code (Text):

curl -I http://domain1.com:8888
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=86400
Content-Length: 1362
Content-Type: text/html; charset=utf-8
Etag: W/"57a637f3-552"
Last-Modified: Sat, 06 Aug 2016 19:18:11 GMT
Server: Caddy
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Powered-By: Caddy via CentminMod
X-Xss-Protection: 1; mode=block
Date: Sat, 06 Aug 2016 19:24:36 GMT

For HTTPS on port 443 and port 448 for Nginx and Caddy respectively

Code (Text):

curl -Ik https://domain1.com:443
HTTP/1.1 200 OK
Date: Sat, 06 Aug 2016 19:24:46 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 1362
Last-Modified: Sat, 06 Aug 2016 19:18:11 GMT
Connection: keep-alive
ETag: "57a637f3-552"
Server: nginx centminmod
X-Powered-By: centminmod
Expires: Sun, 07 Aug 2016 19:24:46 GMT
Cache-Control: max-age=86400
Cache-Control: public, must-revalidate, proxy-revalidate
Accept-Ranges: bytes

Code (Text):

curl -Ik https://domain1.com:448
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=86400
Content-Length: 1362
Content-Type: text/html; charset=utf-8
Etag: W/"57a637f3-552"
Last-Modified: Sat, 06 Aug 2016 19:18:11 GMT
Server: Caddy
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Powered-By: Caddy via CentminMod
X-Xss-Protection: 1; mode=block
Date: Sat, 06 Aug 2016 19:24:51 GMT

HTTPS over HTTP/2 curl'd headers

Code (Text):

curl -Ik https://domain1.com:443
HTTP/2 200
date: Sat, 06 Aug 2016 19:30:23 GMT
content-type: text/html; charset=utf-8
content-length: 1362
last-modified: Sat, 06 Aug 2016 19:18:11 GMT
etag: "57a637f3-552"
server: nginx centminmod
x-powered-by: centminmod
expires: Sun, 07 Aug 2016 19:30:23 GMT
cache-control: max-age=86400
cache-control: public, must-revalidate, proxy-revalidate
accept-ranges: bytes

Code (Text):

curl -Ik https://domain1.com:448
HTTP/2 200
accept-ranges: bytes
cache-control: max-age=86400
content-type: text/html; charset=utf-8
etag: W/"57a637f3-552"
last-modified: Sat, 06 Aug 2016 19:18:11 GMT
server: Caddy
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-powered-by: Caddy via CentminMod
x-xss-protection: 1; mode=block
content-length: 1362
date: Sat, 06 Aug 2016 19:30:28 GMT

Server Version Info

Note Nginx is setup out of box to handle 2 worker_processes on the 4 cpu VirtualBox CentOS 7.2 guest server. While I believe Caddy is setup to utilise all 4 cpus.

Code (Text):

user              nginx nginx;
worker_processes 2;
worker_priority -10;

Code (Text):

caddy -version
Caddy 0.9.0

Code (Text):

caddy -plugins
Server types:
  http

Caddyfile loaders:
  short
  flag
  default

Other plugins:
  http.basicauth
  http.bind
  http.browse
  http.errors
  http.expvar
  http.ext
  http.fastcgi
  http.filemanager
  http.git
  http.gzip
  http.header
  http.hugo
  http.internal
  http.ipfilter
  http.locale
  http.log
  http.mailout
  http.markdown
  http.mime
  http.minify
  http.pprof
  http.prometheus
  http.proxy
  http.ratelimit
  http.realip
  http.redir
  http.rewrite
  http.root
  http.search
  http.templates
  http.websocket
  shutdown
  startup
  tls
  tls.dns.cloudflare
  tls.dns.digitalocean
  tls.dns.googlecloud
  tls.dns.namecheap
  tls.dns.route53
  tls.dns.vultr

Nginx & Caddy HTTPS Vhost Files

As I am more concerned with HTTP/2 HTTPS performance, l'll focus on HTTPS vhost config files for both Nginx and Caddy. Nginx one defaults to LibreSSL 2.4.2 with chacha20_poly1305 as higher preference or can be switched to OpenSSL 1.0.2h with Cloudflare patches for chacha20 which only serve chacha20 over mobile. I'll edit this to prefer AES to match Caddy.

Caddy HTTP/2 HTTPS Vhost at /etc/caddy/conf.d/domain1.com.ssl.conf auto generated via caddy.sh addsite

Code (Text):

domain1.com:448 {
    gzip {
        level 5
        min_length 1400
    }
    #browse
    header / {
        #Strict-Transport-Security "max-age=31536000"
        Cache-Control "max-age=86400"
        X-Content-Type-Options "nosniff"
        X-Frame-Options "SAMEORIGIN"
        X-XSS-Protection "1; mode=block"
        X-Powered-By "Caddy via CentminMod"
        #-Server
    }
    tls self_signed {
        protocols tls1.0 tls1.2
        ciphers ECDHE-RSA-AES128-GCM-SHA256 ECDHE-ECDSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-CBC-SHA ECDHE-RSA-AES256-CBC-SHA ECDHE-ECDSA-AES256-CBC-SHA ECDHE-ECDSA-AES128-CBC-SHA RSA-AES128-CBC-SHA RSA-AES256-CBC-SHA ECDHE-RSA-3DES-EDE-CBC-SHA RSA-3DES-EDE-CBC-SHA
    }
    root /home/nginx/domains/domain1.com/public
    fastcgi / 127.0.0.1:9000 {
        ext   .php
        split .php
        index index.php
    }
    # for wordpress
    #rewrite {
    #    if {path} not_match ^\/wp-admin
    #    to {path} {path}/ /index.php?_url={uri}
    #}
    errors {
      log /home/nginx/domains/domain1.com/log/caddy-domain1.com-errors.log {
        size 100 # Rotate after 50 MB
        age  14 # Keep rotated files for 30 days
        keep 10 # Keep at most 5 log files
      }
      #404 404.html # Not Found
      #500 500.html # Internal Server Error
    }
    log / /home/nginx/domains/domain1.com/log/caddy-domain1.com-access.ssl.log "{remote} {when} {method} {uri} {proto} {status} {size} {>User-Agent} {latency}" {
        rotate {
                size 100 # Rotate after 100 MB
                age  14  # Keep log files for 14 days
                keep 10  # Keep at most 10 log files
        }
    }
}

Nginx HTTP/2 HTTPS Vhost at /usr/local/nginx/conf/conf.d/domain1.com.ssl.conf auto generated via caddy.sh addsite. Include sites are added by default for various Nginx security and performance features including the autoprotect generated file, site maintenance status mode, nginx vhost traffic stats include and optional cloudflare realip + csf firewall whitelisting. Nginx ngx_pagespeed is disabled for Nginx vhost as well.

FYI, Centmin Mod Nginx's ssl_buffer_size is set to 1369 due to integration of Cloudflare's dynamic TLS record size patch.

Code (Text):

# Centmin Mod Getting Started Guide
# must read http://centminmod.com/getstarted.html
# For HTTP/2 SSL Setup
# read http://centminmod.com/nginx_configure_https_ssl_spdy.html

# redirect from www to non-www  forced SSL
# uncomment, save file and restart Nginx to enable
# if unsure use return 302 before using return 301
# server {
#   server_name domain1.com www.domain1.com;
#    return 302 https://$server_name$request_uri;
# }

server {
  listen 443 ssl http2;
  server_name domain1.com www.domain1.com;

  ssl_dhparam /usr/local/nginx/conf/ssl/domain1.com/dhparam.pem;
  ssl_certificate      /usr/local/nginx/conf/ssl/domain1.com/domain1.com.crt;
  ssl_certificate_key  /usr/local/nginx/conf/ssl/domain1.com/domain1.com.key;
  include /usr/local/nginx/conf/ssl_include.conf;

  http2_max_field_size 16k;
  http2_max_header_size 32k;
  # mozilla recommended
  ssl_ciphers EECDH+CHACHA20:EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA256:EECDH+ECDSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+SHA384:EECDH+AES128:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS:!RC4:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA:!CAMELLIA;
  ssl_prefer_server_ciphers   on;
  #add_header Alternate-Protocol  443:npn-spdy/3;
  # HTTP Public Key Pinning Header uncomment only one that applies include or exclude domains.
  # You'd want to include subdomains if you're using SSL wildcard certificates
  # include subdomain
  #add_header Public-Key-Pins 'pin-sha256="urcdM0QCH7avbVN0dvPZG70I7Y6nWkTpcQ0V7af0+2s="; pin-sha256="fpNNHgiU21nnUXd5ZLMaew0MjEcDXedaezrQzSGHLHg="; max-age=86400; includeSubDomains';
  # exclude subdomains
  #add_header Public-Key-Pins 'pin-sha256="urcdM0QCH7avbVN0dvPZG70I7Y6nWkTpcQ0V7af0+2s="; pin-sha256="fpNNHgiU21nnUXd5ZLMaew0MjEcDXedaezrQzSGHLHg="; max-age=86400';
  #add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";
  #add_header X-Frame-Options SAMEORIGIN;
  #add_header X-Xss-Protection "1; mode=block" always;
  #add_header X-Content-Type-Options "nosniff" always;
  #spdy_headers_comp 5;
  ssl_buffer_size 1369;
  ssl_session_tickets on;
  # enable ocsp stapling
  #resolver 8.8.8.8 8.8.4.4 valid=10m;
  #resolver_timeout 10s;
  #ssl_stapling on;
  #ssl_stapling_verify on;
  #ssl_trusted_certificate /usr/local/nginx/conf/ssl/domain1.com/domain1.com-trusted.crt;

# ngx_pagespeed & ngx_pagespeed handler
#include /usr/local/nginx/conf/pagespeed.conf;
#include /usr/local/nginx/conf/pagespeedhandler.conf;
#include /usr/local/nginx/conf/pagespeedstatslog.conf;

  # limit_conn limit_per_ip 16;
  # ssi  on;

  access_log /home/nginx/domains/domain1.com/log/access.log main_ext buffer=256k flush=60m;
  error_log /home/nginx/domains/domain1.com/log/error.log;

  include /usr/local/nginx/conf/autoprotect/domain1.com/autoprotect-domain1.com.conf;
  root /home/nginx/domains/domain1.com/public;
  # uncomment cloudflare.conf include if using cloudflare for
  # server and/or vhost site
  #include /usr/local/nginx/conf/cloudflare.conf;
  include /usr/local/nginx/conf/503include-main.conf;

  location / {
  include /usr/local/nginx/conf/503include-only.conf;

# block common exploits, sql injections etc
#include /usr/local/nginx/conf/block.conf;

  # Enables directory listings when index file not found
  #autoindex  on;

  # Shows file listing times as local time
  #autoindex_localtime on;

  # Enable for vBulletin usage WITHOUT vbSEO installed
  # More example Nginx vhost configurations at
  # http://centminmod.com/nginx_configure.html
  #try_files    $uri $uri/ /index.php;

  }

  include /usr/local/nginx/conf/staticfiles.conf;
  include /usr/local/nginx/conf/php.conf;
  include /usr/local/nginx/conf/drop.conf;
  #include /usr/local/nginx/conf/errorpage.conf;
  include /usr/local/nginx/conf/vts_server.conf;
}

SSL Ciphers

Seems Caddy defaults to ECC 256 bit ECDSA based SSL certificates for self-signed option. While Centmin Mod Nginx defaults to RSA 2048bit standard SSL certificates for self-signed option. ECDSA is more performant than RSA 2048bit so will see how things fair. Caddy also exchange defaults to AES128 vs Centmin Mod Nginx AES256.

nghttp2 tests

Using nghttp2 HTTP/2 library/client for HTTP/2 tests.

Centmin Mod Nginx HTTP/2 on port 443

Code (Text):

nghttp -nas https://domain1.com:443
***** Statistics *****

Request timing:
  responseEnd: the  time  when  last  byte of  response  was  received
               relative to connectEnd
requestStart: the time  just before  first byte  of request  was sent
               relative  to connectEnd.   If  '*' is  shown, this  was
               pushed by server.
      process: responseEnd - requestStart
         code: HTTP status code
         size: number  of  bytes  received as  response  body  without
               inflation.
          URI: request URI

see http://www.w3.org/TR/resource-timing/#processing-model

sorted by 'complete'

id  responseEnd requestStart  process code size request path
13      +521us        +91us    429us  200   1K /

Caddy HTTP/2 on port 448

Code (Text):

nghttp -nas https://domain1.com:448
***** Statistics *****

Request timing:
  responseEnd: the  time  when  last  byte of  response  was  received
               relative to connectEnd
requestStart: the time  just before  first byte  of request  was sent
               relative  to connectEnd.   If  '*' is  shown, this  was
               pushed by server.
      process: responseEnd - requestStart
         code: HTTP status code
         size: number  of  bytes  received as  response  body  without
               inflation.
          URI: request URI

see http://www.w3.org/TR/resource-timing/#processing-model

sorted by 'complete'

id  responseEnd requestStart  process code size request path
13     +5.73ms       +247us   5.48ms  200   1K /

Cipherscan Tests

Centmin Mod Nginx HTTP/2 on port 443

Code (Text):

cipherscan https://domain1.com:443 
............
Target: domain1.com:443

prio  ciphersuite                  protocols              pfs                 curves
1     ECDHE-RSA-AES256-GCM-SHA384  TLSv1.2                ECDH,P-256,256bits  prime256v1
2     ECDHE-RSA-AES128-GCM-SHA256  TLSv1.2                ECDH,P-256,256bits  prime256v1
3     ECDHE-RSA-AES128-SHA256      TLSv1.2                ECDH,P-256,256bits  prime256v1
4     ECDHE-RSA-AES256-SHA384      TLSv1.2                ECDH,P-256,256bits  prime256v1
5     ECDHE-RSA-AES128-SHA         TLSv1,TLSv1.1,TLSv1.2  ECDH,P-256,256bits  prime256v1

Certificate: untrusted, 2048 bits, sha256WithRSAEncryption signature
TLS ticket lifetime hint: 3600
OCSP stapling: not supported
Cipher ordering: server
Curves ordering: server - fallback: no
Server supports secure renegotiation
Server supported compression methods: NONE
TLS Tolerance: yes

Caddy HTTP/2 on port 448

Code (Text):

cipherscan https://domain1.com:448
........
Target: domain1.com:448

prio  ciphersuite                    protocols              pfs                 curves
1     ECDHE-ECDSA-AES128-GCM-SHA256  TLSv1.2                ECDH,P-256,256bits  prime256v1
2     ECDHE-ECDSA-AES256-SHA         TLSv1,TLSv1.1,TLSv1.2  ECDH,P-256,256bits  prime256v1
3     ECDHE-ECDSA-AES128-SHA         TLSv1,TLSv1.1,TLSv1.2  ECDH,P-256,256bits  prime256v1

Certificate: untrusted, 256 bits, ecdsa-with-SHA256 signature
TLS ticket lifetime hint: None
OCSP stapling: not supported
Cipher ordering: server
Curves ordering: server - fallback: no
Server supports secure renegotiation
Server supported compression methods: NONE
TLS Tolerance: yes

Other HTTP/2 Checks

Code (Text):

is-http2 https://domain1.com:443
✓ HTTP/2 supported by https://domain1.com:443
Supported protocols: h2 http/1.1

Code (Text):

is-http2 https://domain1.com:448
✓ HTTP/2 supported by https://domain1.com:448
Supported protocols: h2 http/1.1

Check for ALPN protocol

Code (Text):

echo | /usr/local/http2-15/bin/openssl s_client -alpn h2 -host domain1.com -port 443

New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
ALPN protocol: h2
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384

Code (Text):

echo | /usr/local/http2-15/bin/openssl s_client -alpn h2 -host domain1.com -port 448

New, TLSv1/SSLv3, Cipher is ECDHE-ECDSA-AES128-GCM-SHA256
Server public key is 256 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
ALPN protocol: h2
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-ECDSA-AES128-GCM-SHA256

h2load HTTP/2 Load Testing

Using nghttp2 bundled h2load HTTP/2 load tester h2load - HTTP/2 benchmarking tool - HOW-TO — nghttp2 1.14.0-DEV documentation

Code (Text):

h2load --version
h2load nghttp2/1.14.0-DEV

Low concurrency tests at 10 concurrent users and 100 requests

• Caddy 0.9 HTTP/2 HTTPS = finished in 55.63ms, 1797.46 req/s, 2.43MB/s
• Centmin Mod Nginx 1.11.3 HTTP/2 HTTPS = finished in 25.92ms, 3857.58 req/s, 5.99MB/s

Caddy 0.9 HTTP/2 HTTPS

Code (Text):

/usr/local/bin/h2load -c10 -m100 -n100 -v https://domain1.com:448
starting benchmark...
spawning thread #0: 10 total client(s). 100 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-ECDSA-AES128-GCM-SHA256
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 55.63ms, 1797.46 req/s, 2.43MB/s
requests: 100 total, 100 started, 100 done, 100 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 138.33KB (141650) total, 3.21KB (3290) headers (space savings 90.00%), 133.01KB (136200) data
                     min         max         mean         sd        +/- sd
time for request:     5.01ms     40.86ms     25.26ms     10.62ms    65.00%
time for connect:     6.12ms     14.98ms     11.86ms      2.31ms    80.00%
time to 1st byte:    11.44ms     41.38ms     30.79ms      8.71ms    70.00%
req/s           :     184.33      828.90      292.50      192.24    90.00%

Centmin Mod Nginx 1.11.3 HTTP/2 HTTPS

Code (Text):

/usr/local/bin/h2load -c10 -m100 -n100 -v https://domain1.com:443
starting benchmark...
spawning thread #0: 10 total client(s). 100 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 25.92ms, 3857.58 req/s, 5.99MB/s
requests: 100 total, 100 started, 100 done, 100 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 158.88KB (162690) total, 23.63KB (24200) headers (space savings 27.76%), 133.01KB (136200) data
                     min         max         mean         sd        +/- sd
time for request:      333us      7.70ms      4.79ms      1.98ms    66.00%
time for connect:     9.23ms     21.85ms     14.42ms      3.90ms    60.00%
time to 1st byte:    15.33ms     22.19ms     18.97ms      2.18ms    60.00%
req/s           :     436.26      604.38      512.96       50.95    50.00%

Higher concurrency tests at 100 concurrent users and 1000 requests

• Caddy 0.9 HTTP/2 HTTPS = finished in 324.30ms, 3083.56 req/s, 4.17MB/s
• Centmin Mod Nginx 1.11.3 HTTP/2 HTTPS = finished in 228.77ms, 4371.15 req/s, 6.78MB/s

Caddy 0.9 HTTP/2 HTTPS

Code (Text):

/usr/local/bin/h2load -c100 -m100 -n1000 -v https://domain1.com:448
starting benchmark...
spawning thread #0: 100 total client(s). 1000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-ECDSA-AES128-GCM-SHA256
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 324.30ms, 3083.56 req/s, 4.17MB/s
requests: 1000 total, 1000 started, 1000 done, 1000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 1000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 1.35MB (1417029) total, 32.65KB (33429) headers (space savings 89.84%), 1.30MB (1362000) data
                     min         max         mean         sd        +/- sd
time for request:    20.81ms    254.72ms    139.09ms     46.84ms    72.30%
time for connect:    58.87ms    216.23ms     90.95ms     20.44ms    80.00%
time to 1st byte:    93.30ms    309.86ms    192.95ms     41.66ms    74.00%
req/s           :      31.34      106.64       37.83        9.49    97.00%

Centmin Mod Nginx 1.11.3 HTTP/2 HTTPS

Code (Text):

/usr/local/bin/h2load -c100 -m100 -n1000 -v https://domain1.com:443
starting benchmark...
spawning thread #0: 100 total client(s). 1000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 228.77ms, 4371.15 req/s, 6.78MB/s
requests: 1000 total, 1000 started, 1000 done, 1000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 1000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 1.55MB (1626900) total, 236.33KB (242000) headers (space savings 27.76%), 1.30MB (1362000) data
                     min         max         mean         sd        +/- sd
time for request:      324us     61.89ms     38.38ms      8.09ms    85.60%
time for connect:    58.44ms    174.41ms    140.73ms     22.08ms    61.00%
time to 1st byte:   120.04ms    221.25ms    178.70ms     26.93ms    57.00%
req/s           :      44.99       82.82       57.00        9.04    64.00%

Caddy has a performance overhead issue due to header middleware used for serving headers. The more headers you add to your site, the lower the performance you get see Any performance overhead as you add more headers under HTTP/2? - Help - Caddy Community

So to confirm this I edited Caddy domain1.com HTTPS vhost to remove all the added headers to see re-test performance and see what the numbers give.

so Caddy vhost for domain1.com headers changes from

Code (Text):

    header / {
        #Strict-Transport-Security "max-age=31536000"
        Cache-Control "max-age=86400"
        X-Content-Type-Options "nosniff"
        X-Frame-Options "SAMEORIGIN"
        X-XSS-Protection "1; mode=block"
        X-Powered-By "Caddy via CentminMod"
        #-Server
    }

to which leaves just the default server header in play

Code (Text):

    header / {
        #Strict-Transport-Security "max-age=31536000"
        #Cache-Control "max-age=86400"
        #X-Content-Type-Options "nosniff"
        #X-Frame-Options "SAMEORIGIN"
        #X-XSS-Protection "1; mode=block"
        #X-Powered-By "Caddy via CentminMod"
        #-Server
    }

Low concurrency tests at 10 concurrent users and 100 requests

• Caddy 0.9 HTTP/2 HTTPS with headers = finished in 55.63ms, 1797.46 req/s, 2.43MB/s
• Caddy 0.9 HTTP/2 HTTPS with without headers = finished in 48.52ms, 2060.92 req/s, 2.75MB/s

Code (Text):

/usr/local/bin/h2load -c10 -m100 -n100 -v https://domain1.com:448
starting benchmark...
spawning thread #0: 10 total client(s). 100 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-ECDSA-AES128-GCM-SHA256
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 48.52ms, 2060.92 req/s, 2.75MB/s
requests: 100 total, 100 started, 100 done, 100 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 136.81KB (140090) total, 1.69KB (1730) headers (space savings 90.80%), 133.01KB (136200) data
                     min         max         mean         sd        +/- sd
time for request:     5.70ms     37.94ms     25.76ms      8.73ms    71.00%
time for connect:     4.89ms     14.10ms      9.72ms      2.39ms    80.00%
time to 1st byte:    10.64ms     41.93ms     29.57ms      8.59ms    80.00%
req/s           :     213.41      920.36      303.60      206.22    90.00%

Higher concurrency tests at 100 concurrent users and 1000 requests

• Caddy 0.9 HTTP/2 HTTPS with headers = finished in 324.30ms, 3083.56 req/s, 4.17MB/s
• Caddy 0.9 HTTP/2 HTTPS without headers = finished in 303.17ms, 3298.46 req/s, 4.41MB/s

Code (Text):

/usr/local/bin/h2load -c100 -m100 -n1000 -v https://domain1.com:448
starting benchmark...
spawning thread #0: 100 total client(s). 1000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-ECDSA-AES128-GCM-SHA256
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 303.17ms, 3298.46 req/s, 4.41MB/s
requests: 1000 total, 1000 started, 1000 done, 1000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 1000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 1.34MB (1400900) total, 16.89KB (17300) headers (space savings 90.80%), 1.30MB (1362000) data
                     min         max         mean         sd        +/- sd
time for request:    11.45ms    238.16ms    172.13ms     43.30ms    81.20%
time for connect:    55.90ms     94.26ms     81.40ms     14.52ms    71.00%
time to 1st byte:    88.59ms    285.75ms    226.33ms     47.72ms    82.00%
req/s           :      33.30      112.68       37.89       12.44    96.00%

 

Note previous post tests with Centmin Mod Nginx is setup out of box to handle 2 worker_processes on the 4 cpu VirtualBox CentOS 7.2 guest server. While I believe Caddy is setup to utilise all 4 cpus.

Code (Text):

user              nginx nginx;
worker_processes 2;
worker_priority -10;

Retesting with Centmin Mod Nginx set to 4 worker_processes now. Seeing as it's 4 cpu threads not 4 real cpu cores, the bump from 2 to 4 only meant a marginal increase.

Low concurrency tests at 10 concurrent users and 100 requests

• Caddy 0.9 HTTP/2 HTTPS with headers = finished in 55.63ms, 1797.46 req/s, 2.43MB/s
• Caddy 0.9 HTTP/2 HTTPS with without headers = finished in 48.52ms, 2060.92 req/s, 2.75MB/s
• Centmin Mod Nginx 1.11.3 HTTP/2 HTTPS (2 cpus) = finished in 25.92ms, 3857.58 req/s, 5.99MB/s
• Centmin Mod Nginx 1.11.3 HTTP/2 HTTPS (4 cpus) = finished in 22.39ms, 4465.68 req/s, 6.93MB/s

Code (Text):

/usr/local/bin/h2load -c10 -m100 -n100 -v https://domain1.com:443
starting benchmark...
spawning thread #0: 10 total client(s). 100 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 22.39ms, 4465.68 req/s, 6.93MB/s
requests: 100 total, 100 started, 100 done, 100 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 158.88KB (162690) total, 23.63KB (24200) headers (space savings 27.76%), 133.01KB (136200) data
                     min         max         mean         sd        +/- sd
time for request:     1.55ms      3.93ms      3.03ms       690us    68.00%
time for connect:    11.39ms     18.24ms     13.94ms      2.41ms    70.00%
time to 1st byte:    14.33ms     20.01ms     16.89ms      1.96ms    60.00%
req/s           :     475.41      675.08      585.28       69.79    60.00%

Higher concurrency tests at 100 concurrent users and 1000 requests

• Caddy 0.9 HTTP/2 HTTPS with headers = finished in 324.30ms, 3083.56 req/s, 4.17MB/s
• Caddy 0.9 HTTP/2 HTTPS without headers = finished in 303.17ms, 3298.46 req/s, 4.41MB/s
• Centmin Mod Nginx 1.11.3 HTTP/2 HTTPS (2 cpus) = finished in 228.77ms, 4371.15 req/s, 6.78MB/s
• Centmin Mod Nginx 1.11.3 HTTP/2 HTTPS (4 cpus) = finished in 195.44ms, 5116.69 req/s, 7.94MB/s

Code (Text):

/usr/local/bin/h2load -c100 -m100 -n1000 -v https://domain1.com:443
starting benchmark...
spawning thread #0: 100 total client(s). 1000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 195.44ms, 5116.69 req/s, 7.94MB/s
requests: 1000 total, 1000 started, 1000 done, 1000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 1000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 1.55MB (1626900) total, 236.33KB (242000) headers (space savings 27.76%), 1.30MB (1362000) data
                     min         max         mean         sd        +/- sd
time for request:    19.62ms     67.80ms     30.76ms      8.76ms    71.30%
time for connect:    66.91ms    157.08ms    124.66ms     17.20ms    78.00%
time to 1st byte:   127.22ms    184.74ms    155.12ms     17.22ms    51.00%
req/s           :      53.82       78.32       65.01        7.37    54.00%

Even Higher Concurrency Tests - 2000 Users

Even higher concurrency tests at 2000 concurrent users and 25000 requests

• Caddy 0.9 HTTP/2 HTTPS with headers = finished in 6.16s, 4058.36 req/s, 5.47MB/s
• Centmin Mod Nginx 1.11.3 HTTP/2 HTTPS (4 cpus) = finished in 3.21s, 7795.35 req/s, 12.09MB/s

Caddy

Code (Text):

/usr/local/bin/h2load -c2000 -m100 -n25000 -v https://domain1.com:448
starting benchmark...
spawning thread #0: 2000 total client(s). 25000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-ECDSA-AES128-GCM-SHA256
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 6.16s, 4058.36 req/s, 5.47MB/s
requests: 25000 total, 25000 started, 25000 done, 25000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 25000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 33.73MB (35363701) total, 773.15KB (791701) headers (space savings 90.37%), 32.47MB (34050000) data
                     min         max         mean         sd        +/- sd
time for request:   272.24ms       5.16s       2.75s       1.23s    57.35%
time for connect:   859.65ms       2.64s       1.47s    265.03ms    78.70%
time to 1st byte:      1.55s       5.00s       3.49s       1.07s    59.25%
req/s           :       1.99        8.26        2.93        1.28    87.25%

Nginx

Code (Text):

/usr/local/bin/h2load -c2000 -m100 -n25000 -v https://domain1.com:443
starting benchmark...
spawning thread #0: 2000 total client(s). 25000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 3.21s, 7795.35 req/s, 12.09MB/s
requests: 25000 total, 25000 started, 25000 done, 25000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 25000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 38.76MB (40648000) total, 5.77MB (6050000) headers (space savings 27.76%), 32.47MB (34050000) data
                     min         max         mean         sd        +/- sd
time for request:     1.05ms       1.12s    641.55ms    193.01ms    76.68%
time for connect:   690.11ms       2.26s       1.82s    351.88ms    84.00%
time to 1st byte:   913.11ms       3.07s       2.46s    497.12ms    84.90%
req/s           :       3.91       14.22        5.51        2.21    91.85%

Current system memory usage totals via smem

Code (Text):

smem -tk
  PID User     Command                         Swap      USS      PSS      RSS 
  686 avahi    avahi-daemon: chroot helpe         0    84.0K   182.0K   400.0K 
 2897 root     /sbin/agetty --noclear tty1        0   156.0K   182.0K   840.0K 
  640 root     /sbin/rngd -f                      0   180.0K   193.0K   672.0K 
  643 root     /usr/sbin/irqbalance --fore        0   288.0K   352.0K     1.1M 
 5569 root     /usr/sbin/anacron -s               0   332.0K   365.0K   788.0K 
 2594 root     /usr/lib/systemd/systemd-ma        0   556.0K   580.0K     1.3M 
  506 root     /usr/sbin/lvmetad -f               0   536.0K   599.0K     1.5M 
  644 avahi    avahi-daemon: running [cent        0   428.0K   609.0K     1.7M 
  618 root     /sbin/auditd -n                    0   552.0K   610.0K     1.7M 
  698 root     /usr/sbin/crond -n                 0   672.0K   705.0K     1.5M 
  697 root     /usr/lib/systemd/systemd-lo        0   776.0K   808.0K     1.7M 
 1659 nobody   /usr/local/bin/memcached -d        0   800.0K   814.0K     1.3M 
  516 root     /usr/lib/systemd/systemd-ud        0   792.0K   856.0K     1.8M 
  685 dbus     /bin/dbus-daemon --system -        0  1016.0K     1.0M     1.9M 
  732 root     /usr/sbin/wpa_supplicant -u        0   896.0K     1.1M     2.7M 
  692 ntp      /usr/sbin/ntpd -u ntp:ntp -        0     1.0M     1.1M     2.3M 
 2582 root     /bin/bash                          0   488.0K     1.2M     1.9M 
 3004 root     /bin/bash                          0   512.0K     1.2M     2.0M 
  956 root     /usr/sbin/sshd -D                  0   860.0K     1.3M     3.5M 
 2402 root     /usr/libexec/postfix/master        0     1.2M     1.3M     2.2M 
 1288 root     pure-ftpd (SERVER)                 0     1.4M     1.5M     2.7M 
 4823 postfix  pickup -l -t unix -u               0     1.2M     1.7M     4.0M 
 2449 postfix  qmgr -l -t unix -u                 0     1.3M     1.8M     4.2M 
 2913 root     -bash                              0     1.4M     1.8M     3.0M 
 3179 root     -bash                              0     1.4M     1.9M     3.1M 
  642 root     /usr/sbin/rsyslogd -n              0     1.5M     2.2M     4.0M 
 2911 root     sshd: root@pts/0,pts/              0     2.2M     2.7M     5.3M 
    1 root     /usr/lib/systemd/systemd --        0     2.6M     2.7M     3.7M 
  645 root     /usr/sbin/haveged -w 4067 -        0     5.8M     5.8M     6.4M 
  639 root     /usr/sbin/NetworkManager --        0     4.9M     5.9M     8.7M 
 5649 root     python /usr/bin/smem -tk           0     5.5M     6.2M     7.6M 
  492 root     /usr/lib/systemd/systemd-jo        0     7.0M     7.7M     9.2M 
 4390 root     nginx: master process /usr/        0     3.4M     8.3M    28.0M 
 4417 root     php-fpm: master process (/u        0     8.6M     8.6M     9.2M 
  733 polkitd  /usr/lib/polkit-1/polkitd -        0     8.6M     9.5M    11.9M 
  945 root     /usr/bin/python -Es /usr/sb        0    11.6M    12.7M    16.0M 
 4392 nginx    nginx: worker process              0     8.0M    13.0M    33.7M 
 2993 root     /usr/bin/docker-current exe        0     9.5M    13.1M    17.5M 
 4393 nginx    nginx: worker process              0     7.9M    13.1M    33.9M 
  743 root     /sbin/dhclient -d -q -sf /u        0    13.1M    13.4M    15.4M 
 4394 nginx    nginx: worker process              0     8.4M    13.5M    34.1M 
 4395 nginx    nginx: worker process              0     8.7M    13.7M    34.4M 
 3059 root     lfd - sleepin                      0    22.6M    22.6M    23.3M 
 1065 root     /usr/bin/docker-current dae        0    19.2M    22.8M    27.4M 
 4334 root     /usr/local/bin/caddy -log /        0    45.2M    45.2M    45.2M 
 1477 mysql    /usr/sbin/mysqld                   0    93.8M    94.1M    96.2M 
-------------------------------------------------------------------------------
   46 9                                           0   316.8M   360.4M   521.2M

 

System Resource Usage

Next up is testing system resource usage under load. For this I'll install Centmin Mod's alpha addons/netdata.sh addon for system resource monitoring

I chose not to use nginx to proxy the netdata gui as that would add to nginx's usage loads and skew nginx usage stats

Available netdata system metrics to monitor

Code (Text):

cat /etc/netdata/netdata.conf | grep "^\[" | cut -d '[' -f 2 | cut -d ']' -f 1 | grep -v ^global$ | grep -v "^plugin" | sort -u
apps.cpu
apps.cpu_system
apps.cpu_user
apps.files
apps.lreads
apps.lwrites
apps.major_faults
apps.mem
apps.minor_faults
apps.pipes
apps.preads
apps.processes
apps.pwrites
apps.sockets
apps.threads
cgroup_34c80afbfaa5.cpu
cgroup_34c80afbfaa5.cpu_per_core
cgroup_34c80afbfaa5.io
cgroup_34c80afbfaa5.mem
cgroup_34c80afbfaa5.mem_activity
cgroup_34c80afbfaa5.pgfaults
cgroup_34c80afbfaa5.queued_ops
cgroup_34c80afbfaa5.serviced_ops
cgroup_34c80afbfaa5.throttle_serviced_ops
cgroup_34c80afbfaa5.writeback
cpu.cpu0
cpu.cpu0_interrupts
cpu.cpu0_softirqs
cpu.cpu1
cpu.cpu1_interrupts
cpu.cpu1_softirqs
cpu.cpu2
cpu.cpu2_interrupts
cpu.cpu2_softirqs
cpu.cpu3
cpu.cpu3_interrupts
cpu.cpu3_softirqs
disk_avgsz.dm-1
disk_avgsz.sda
disk_await.dm-1
disk_await.sda
disk_backlog.dm-1
disk_backlog.sda
disk.dm-1
disk_inodes.dm-1
disk_inodes.sda1
disk_iotime.dm-1
disk_iotime.sda
disk_mops.sda
disk_ops.dm-1
disk_ops.sda
disk.sda
disk_space.dm-1
disk_space.sda1
disk_svctm.dm-1
disk_svctm.sda
disk_util.dm-1
disk_util.sda
groups.cpu
groups.cpu_system
groups.cpu_user
groups.files
groups.lreads
groups.lwrites
groups.major_faults
groups.mem
groups.minor_faults
groups.pipes
groups.preads
groups.processes
groups.pwrites
groups.sockets
groups.threads
ipv4.bcast
ipv4.bcastpkts
ipv4.errors
ipv4.fragsin
ipv4.fragsout
ipv4.mcast
ipv4.mcastpkts
ipv4.packets
ipv4.tcperrors
ipv4.tcphandshake
ipv4.tcppackets
ipv4.tcpsock
ipv4.udperrors
ipv4.udppackets
ipv6.ect
ipv6.icmp
ipv6.icmpmldv2
ipv6.icmpneighbor
ipv6.icmprouter
ipv6.icmptypes
ipv6.mcast
ipv6.mcastpkts
ipv6.packets
mem.committed
mem.kernel
mem.pgfaults
mem.slab
mem.writeback
netdata.apps_children_fix
netdata.apps_cpu
netdata.apps_files
netdata.apps_fix
netdata.clients
netdata.compression_ratio
netdata.net
netdata.plugin_cgroups_cpu
netdata.plugin_proc_cpu
netdata.plugin_tc_cpu
netdata.plugin_tc_time
netdata.requests
netdata.response_time
netdata.server_cpu
net.docker0
net.enp0s3
netfilter.conntrack_changes
netfilter.conntrack_errors
netfilter.conntrack_expect
netfilter.conntrack_new
netfilter.conntrack_search
netfilter.conntrack_sockets
net_packets.docker0
net_packets.enp0s3
net_packets.vethc135f09
net.vethc135f09
registry
system.active_processes
system.cpu
system.ctxt
system.entropy
system.forks
system.idlejitter
system.interrupts
system.intr
system.io
system.ipv4
system.ipv6
system.load
system.processes
system.ram
system.softirqs
system.swap
system.swapio
users.cpu
users.cpu_system
users.cpu_user
users.files
users.lreads
users.lwrites
users.major_faults
users.mem
users.minor_faults
users.pipes
users.preads
users.processes
users.pwrites
users.sockets
users.threads

 

h2load netdata charts

With addons/netdata.sh installing and configuring netdata for system monitoring, I re-tested h2load stress test with higher loads at 1,000 concurrent users and 50,0000 requests. But this time I ran each test 3 times and filtered just the requests per second and percentage of completed requests.

Commands used:

For Centmin Mod Nginx 1.11.3 HTTP/2 port 443

Code (Text):

echo "CentminMod.com Nginx 1.11.3 HTTP/2 h2load stress test"; sleep 30; for i in {1..3}; do /usr/local/bin/h2load -c1000 -m100 -n50000 -v https://domain1.com:443 > h2load.$i.log; cat h2load.$i.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;

For Caddy 0.9 HTTP/2 port 448

Code (Text):

echo "Caddy 0.9 HTTP/2 h2load stress test"; sleep 30; for i in {1..3}; do /usr/local/bin/h2load -c1000 -m100 -n50000 -v https://domain1.com:448 > h2load.$i.log; cat h2load.$i.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;

Resulting output:

For Centmin Mod Nginx 1.11.3 HTTP/2 port 443 = 11,322.50 req/s average

Code (Text):

echo "CentminMod.com Nginx 1.11.3 HTTP/2 h2load stress test"; sleep 30; for i in {1..3}; do /usr/local/bin/h2load -c1000 -m100 -n50000 -v https://domain1.com:443 > h2load.$i.log; cat h2load.$i.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;
CentminMod.com Nginx 1.11.3 HTTP/2 h2load stress test
8896.39 req/s 100% completed
17788.95 req/s 100% completed
7282.18 req/s 100% completed

For Caddy 0.9 HTTP/2 port 448 = 3,751.71 req/s average

Code (Text):

echo "Caddy 0.9 HTTP/2 h2load stress test"; sleep 30; for i in {1..3}; do /usr/local/bin/h2load -c1000 -m100 -n50000 -v https://domain1.com:448 > h2load.$i.log; cat h2load.$i.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;
Caddy 0.9 HTTP/2 h2load stress test
3908.97 req/s 100% completed
3504.25 req/s 100% completed
3841.91 req/s 100% completed

Netdata recorded the system cpu utilisation, cpu load, disk and memory usage. Below netdata charts show 2 sets of 3 grouped spikes. The set of 3 spikes on the left is Centmin Mod Nginx h2load run and the set of 3 spikes on the right is Caddy h2load run. As you can see Caddy used more memory than Nginx and spent more time in cpu. Though both didn't max cpu beyond <100% or a 4 cpu thread system so 100/400 = 0.25 cpu load so not that much. The h2load by default only tests with a single thread. So I guess next test is to use 2 or 4 h2load threads for testing.

• CentminMod.com Nginx 1.11.3 HTTP/2 h2load stress test cpu load peaked at 0.60 and 548MB total memory used
• Caddy 0.9 HTTP/2 h2load stress test cpu load peaked at 1.57 and 1289MB total memory used

Looks like Caddy's cpu load could also be partially disk related though still very light at ~1.47% disk utilisation



Netdata has Nginx support for monitoring but no support for Caddy. Nginx peaked at around 619 active connections per second and 335 requests/second

 

h2load threads

Next is to repeat the h2load tests in previous post but instead of default 1 h2load thread, set to 2 threads as per h2load(1) — nghttp2 1.14.0-DEV documentation

Commands used:

For Centmin Mod Nginx 1.11.3 HTTP/2 port 443

Code (Text):

echo "CentminMod.com Nginx 1.11.3 HTTP/2 h2load stress test"; sleep 30; for i in {1..3}; do /usr/local/bin/h2load -t2 -c1000 -m100 -n50000 -v https://domain1.com:443 > h2load.$i.log; cat h2load.$i.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;

For Caddy 0.9 HTTP/2 port 448

Code (Text):

echo "Caddy 0.9 HTTP/2 h2load stress test"; sleep 30; for i in {1..3}; do /usr/local/bin/h2load -t2 -c1000 -m100 -n50000 -v https://domain1.com:448 > h2load.$i.log; cat h2load.$i.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;

Resulting output:

For Centmin Mod Nginx 1.11.3 HTTP/2 port 443 = 10,850.84 req/s average

Code (Text):

echo "CentminMod.com Nginx 1.11.3 HTTP/2 h2load stress test"; sleep 30; for i in {1..3}; do /usr/local/bin/h2load -t2 -c1000 -m100 -n50000 -v https://domain1.com:443 > h2load.$i.log; cat h2load.$i.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;
CentminMod.com Nginx 1.11.3 HTTP/2 h2load stress test
10320.74 req/s 100% completed
10585.68 req/s 100% completed
11646.10 req/s 100% completed

For Caddy 0.9 HTTP/2 port 448 = 3,942.75 req/s average

Code (Text):

echo "Caddy 0.9 HTTP/2 h2load stress test"; sleep 30; for i in {1..3}; do /usr/local/bin/h2load -t2 -c1000 -m100 -n50000 -v https://domain1.com:448 > h2load.$i.log; cat h2load.$i.log | awk '/finished in/ {print $4 " req/s "} /requests: / {print ($8/$2*100)"% completed"}' | tr -d '\n'; echo; sleep 30; done;
4879.98 req/s 100% completed
3438.50 req/s 100% completed
3509.76 req/s 100% completed

Again 3 spikes on left is for Centmin Mod Nginx and 3 spikes on right is for Caddy.

• CentminMod.com Nginx 1.11.3 HTTP/2 h2load stress test cpu load peaked at 0.62 and 551MB total memory used
• Caddy 0.9 HTTP/2 h2load stress test cpu load peaked at 1.86 and 1393MB total memory used

h2load 2 thread test with 1000 users and 50,000 requests




Nginx peaked at 949 active connections and requests/sec



Note, Nginx was with LibreSSL 2.4.2 compiled but seems OpenSSL 1.0.2h and 1.1.0 would have even better performance SSL - OpenSSL 1.1.0-pre6 vs OpenSSL 1.0.2h vs OpenSSL 1.0.1e vs LibreSSL 2.4.2 vs LibreSSL 2.3.6 | Centmin Mod Community

 

Caddy Commercial Licensing & Changes

Caddy has announced changes to it's licensing and introduced a paid commercial licensing option. Full details at https://caddyserver.com/blog/accouncing-caddy-commercial-licenses.

But for me using HTTP header as a means for it isn't my cup of tea when there are known performance overhead hit as you add more headers to Caddy. Though you can build from source yourself to get around the HTTP sponsor header requirement https://caddy.community/t/caddy-commercial-sponsor-header-clarification/2716. But right now building from source is sort of broken too Local build instructions from README don't work · Issue #1843 · mholt/caddy · GitHub.

I think most personal usage folks using Caddy wouldn't care about that as they'd just use free version/personal with HTTP sponsor header intact anyway not knowing or caring about the performance hit as you add more HTTP headers.

For HTTP/2 HTTPS loads, Caddy is ~1/3rd the performance of Nginx. So you'd need 3-4x Caddy servers to match the performance of 1x Nginx server for HTTP/2 based HTTPS. So for Commercial licensing you'd need 5/server license at discounted US$250/month which reverts to 4x250 = US$1,000 month once introductory licensing ends. That equates to $3,000/yr discounted or $12,000/yr for 5 commercial licenses if you only need 4 ?

Compare Nginx commercial licensing for 1-4 servers Pricing - Application Delivery for the Modern Web | NGINX

• Nginx Basic $2,500/yr per server
• Nginx Pro $3,500/yr per server
• Nginx Enterprise $5,000/yr per server

If I need 4x Caddy servers to match 1x Nginx servers performance the comparative cost is:

• Caddy 5 instance license = $3,000/yr discounted or $12,000/yr normal price
• Nginx 1 server Basic license = $2,500/yr, $3,500/yr Pro or $5,000/yr Enterprise.

If I need 8x Caddy servers to match 2x Nginx servers performance the comparative cost is:

• Caddy 10 instance license = $6,000/yr discounted or $24,000/yr normal price
• Nginx 2 server Basic license = $5,000/yr, $7,000/yr Pro or $10,000/yr Enterprise.

From financial and performance perspective, doesn't make sense unfortunately.

Edit: oh mis-read that as intro price was 1/4 of the full price in above numbers. The non-intro price isn't actually mentioned yet

Update: Started building my own Caddy binaries without the HTTP Sponsor header, though Caddy has already reverted and removed the HTTP Sponsor header in upstream source. Guess it was an learning exercise for me to figure out how to build my own custom Caddy binaries and integrate Caddy plugins.

Below is custom built Caddy binary + plugins source compiled with Go 1.9

Code (Text):

go version
go version go1.9 linux/amd64

Code (Text):

caddy -version
Caddy 0.10.9 (+545fa84 Fri Sep 15 14:36:05 UTC 2017)
1 file changed, 32 insertions(+)
caddy/caddymain/run.go

Code (Text):

./caddy.sh status
caddy is running (13188)
-----------------------------------------------
Max processes     8192     8192     processes
Max open files    262144   262144   files
-----------------------------------------------

Code (Text):

curl -I http://$(hostname):8888         
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=86400
Content-Length: 3801
Content-Type: text/html; charset=utf-8
Etag: "ow5yjt2xl"
Last-Modified: Tue, 12 Sep 2017 10:59:53 GMT
Server: Caddy
X-Powered-By: Caddy via CentminMod
Date: Fri, 15 Sep 2017 14:44:28 GMT

Code (Text):

caddy -plugins
Server types:
  http

Caddyfile loaders:
  short
  flag
  default

Other plugins:
  http.authz
  http.awses
  http.awslambda
  http.basicauth
  http.bind
  http.browse
  http.cache
  http.cgi
  http.cors
  http.datadog
  http.errors
  http.expires
  http.expvar
  http.ext
  http.fastcgi
  http.filter
  http.git
  http.gopkg
  http.grpc
  http.gzip
  http.header
  http.hugo
  http.index
  http.internal
  http.ipfilter
  http.jekyll
  http.jsonp
  http.jwt
  http.limits
  http.locale
  http.log
  http.login
  http.mailout
  http.markdown
  http.mime
  http.minify
  http.multipass
  http.nobots
  http.pprof
  http.prometheus
  http.proxy
  http.proxyprotocol
  http.push
  http.ratelimit
  http.realip
  http.reauth
  http.redir
  http.request_id
  http.restic
  http.rewrite
  http.root
  http.search
  http.status
  http.templates
  http.timeouts
  http.upload
  http.webdav
  http.websocket
  shutdown
  startup
  tls
  tls.storage.file

Update: September 16th, 2017

Just sat down to read Caddy EULA which applies to their official Caddy binaries and conclusion for me is I probably need to build my own source binaries anyway caddy/EULA.txt at master · mholt/caddy · GitHub

so I shouldn't have published benchmarks for official Caddy binaries ? Never came across a EULA that prohibits benchmark info sharing ?

I'm trying to combine/integrate several web servers under the one roof/server - nginx, openlitespeed, litespeed, apache 2.4, caddy and/or h2o.

 

Caddy Official vs Caddy Source Binary vs Centmin Mod Nginx

Testing performance of official Caddy v0.10.9 binary versus custom source compiled Caddy v0.10.9 vs Centmin Mod Nginx 1.13.5. Caddy official has intact HTTP sponsor header while custom source compiled have removed HTTP Sponsor headers. However, these quick tests were done on OpenVZ VPS so couldn't install my nghttp2 h2load Docker image. So not able to see the HTTP header overhead with h2load HTTP/2 based HTTPS tests.

Below tests are just non-HTTPS quick tests.

Test System

• 2GB OpenVZ VPS
• 2 CPU Threads @E5-1650v3 @3.50Ghz
• 50GB SSD
• Washington, DC
• CentOS 7.4 64bit
• Centmin Mod 123.09beta01

Official Caddy v0.10.9 binary

Code (Text):

curl -I http://$(hostname):8888
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=86400
Caddy-Sponsors: This free web server is licensed only for non-commercial use, made possible by its sponsors: Minio, Uptime Robot, and Sourcegraph
Content-Length: 3801
Content-Type: text/html; charset=utf-8
Etag: "ow5yjt2xl"
Last-Modified: Tue, 12 Sep 2017 10:59:53 GMT
Server: Caddy
X-Powered-By: Caddy via CentminMod
Date: Fri, 15 Sep 2017 16:02:30 GMT

Custom Caddy source built v0.10.9 binary

Code (Text):

curl -I http://$(hostname):8888     
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=86400
Content-Length: 3801
Content-Type: text/html; charset=utf-8
Etag: "ow5yjt2xl"
Last-Modified: Tue, 12 Sep 2017 10:59:53 GMT
Server: Caddy
X-Powered-By: Caddy via CentminMod
Date: Fri, 15 Sep 2017 14:44:28 GMT

Centmin Mod Nginx 1.13.5

Code (Text):

curl -I http://$(hostname):80
HTTP/1.1 200 OK
Date: Fri, 15 Sep 2017 16:04:26 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 3801
Last-Modified: Tue, 12 Sep 2017 10:59:53 GMT
Connection: keep-alive
Vary: Accept-Encoding
ETag: "59b7be29-ed9"
Server: nginx centminmod
X-Powered-By: centminmod
Accept-Ranges: bytes

Siege 4.0.2 test parameters

Code (Text):

siege -v
SIEGE 4.0.2
Usage: siege [options]
      siege [options] URL
      siege -g URL
Options:
  -V, --version             VERSION, prints the version number.
  -h, --help                HELP, prints this section.
  -C, --config              CONFIGURATION, show the current config.
  -v, --verbose             VERBOSE, prints notification to screen.
  -q, --quiet               QUIET turns verbose off and suppresses output.
  -g, --get                 GET, pull down HTTP headers and display the
                           transaction. Great for application debugging.
  -c, --concurrent=NUM      CONCURRENT users, default is 10
  -r, --reps=NUM            REPS, number of times to run the test.
  -t, --time=NUMm           TIMED testing where "m" is modifier S, M, or H
                           ex: --time=1H, one hour test.
  -d, --delay=NUM           Time DELAY, random delay before each requst
  -b, --benchmark           BENCHMARK: no delays between requests.
  -i, --internet            INTERNET user simulation, hits URLs randomly.
  -f, --file=FILE           FILE, select a specific URLS FILE.
  -R, --rc=FILE             RC, specify an siegerc file
  -l, --log[=FILE]          LOG to FILE. If FILE is not specified, the
                           default is used: PREFIX/var/siege.log
  -m, --mark="text"         MARK, mark the log file with a string.
                           between .001 and NUM. (NOT COUNTED IN STATS)
  -H, --header="text"       Add a header to request (can be many)
  -A, --user-agent="text"   Sets User-Agent in request
  -T, --content-type="text" Sets Content-Type in request

Copyright (C) 2016 by Jeffrey Fulmer, et al.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE.

• siege -b -c200 -t60s http://$(hostname):8888/ -m caddy-0.10.9-official
• siege -b -c200 -t60s http://$(hostname):8888/ -m caddy-0.10.9-sourcebuilt
• siege -b -c200 -t60s http://$(hostname):80/ -m cmm-nginx-1.13.5

Results

Code (Text):

head -n1 /usr/local/var/log/siege.log; tail -6 /usr/local/var/log/siege.log
     Date & Time,  Trans,  Elap Time,  Data Trans,  Resp Time,  Trans Rate,  Throughput,  Concurrent,    OKAY,   Failed
**** caddy-0.10.9-official ****
2017-09-15 16:07:54,  68079,      59.80,         340,       0.17,     1138.44,        5.69,      198.44,   68079,       0
**** caddy-0.10.9-sourcebuilt ****
2017-09-15 16:12:40,  67078,      59.07,         335,       0.18,     1135.57,        5.67,      198.77,   67078,       0
**** cmm-nginx-1.13.5 ****
2017-09-15 16:14:26, 138406,      59.43,         840,       0.08,     2328.89,       14.13,      192.20,  138465,       0

Wrk 4.0 Test Parameters

Code (Text):

wrk -v
wrk 4.0.2-2-g91655b5 [epoll] Copyright (C) 2012 Will Glozer
Usage: wrk <options> <url>                     
  Options:                                     
   -c, --connections <N>  Connections to keep open
   -d, --duration    <T>  Duration of test     
   -t, --threads     <N>  Number of threads to use
                                               
   -s, --script      <S>  Load Lua script file 
   -H, --header      <H>  Add header to request
       --latency          Print latency statistics
       --timeout     <T>  Socket/request timeout
   -v, --version          Print version details
                                               
  Numeric arguments may include a SI unit (1k, 1M, 1G)
  Time arguments may include a time unit (2s, 2m, 2h)

• wrk -t2 -c200 -d60s --latency http://$(hostname):8888/ # caddy-0.10.9-official
• wrk -t2 -c200 -d60s --latency http://$(hostname):8888/ # caddy-0.10.9-sourcebuilt
• wrk -t2 -c200 -d60s --latency http://$(hostname):80/ # cmm-nginx-1.13.5

caddy-0.10.9-official

Code (Text):

wrk -t2 -c200 -d60s --latency http://$(hostname):8888/
Running 1m test @ http://hostname:8888/
  2 threads and 200 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
   Latency   191.73ms  284.89ms   2.00s    86.26%
   Req/Sec     1.54k   804.48     4.25k    66.08%
  Latency Distribution
    50%   71.86ms
    75%  283.67ms
    90%  577.07ms
    99%    1.28s
  183306 requests in 1.00m, 740.23MB read
  Socket errors: connect 0, read 0, write 0, timeout 72
Requests/sec:   3053.09
Transfer/sec:     12.33MB

caddy-0.10.9-sourcebuilt

Code (Text):

wrk -t2 -c200 -d60s --latency http://$(hostname):8888/
Running 1m test @ http://hostname:8888/
  2 threads and 200 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
   Latency   131.75ms  184.25ms   1.50s    85.65%
   Req/Sec     1.87k     0.92k    4.84k    69.21%
  Latency Distribution
    50%   59.60ms
    75%  202.88ms
    90%  386.41ms
    99%  799.69ms
  222556 requests in 1.00m, 867.45MB read
Requests/sec:   3706.13
Transfer/sec:     14.45MB

cmm-nginx-1.13.5

Code (Text):

wrk -t2 -c200 -d60s --latency http://$(hostname):80/
Running 1m test @ http://hostname:80/
  2 threads and 200 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
   Latency     6.91ms    4.59ms  88.18ms   84.16%
   Req/Sec     7.50k     2.56k   24.46k    75.28%
  Latency Distribution
    50%    6.36ms
    75%    8.65ms
    90%   11.01ms
    99%   23.25ms
  890755 requests in 1.00m, 3.41GB read
Requests/sec:  14833.86
Transfer/sec:     58.11MB

Summary

web server	siege req/s	siege resp time	wrk req/s	wrk avg latency
caddy 0.10.9 official binary	1138	0.17s	3053	191.73ms
caddy 0.10.9 source built binary	1136	0.18s	3706	131.75ms
cmm nginx 1.13.5	2329	0.08s	14834	6.91ms

 

Part 2 Caddy vs Nginx benchmarks this time for doing HTTP/2 based HTTPS load testing via h2load tester SSL - Caddy HTTP/2 server & benchmarks - Part 2

 

Caddy 0.11.5 with TLS 1.3 support was released so re-tested to see how performance is at SSL - Caddy 0.11.5 TLS 1.3 HTTP/2 HTTPS Benchmarks Part 1