Join the community today
Become a Member

SSL Caddy 0.11.5 TLS 1.3 HTTP/2 HTTPS Benchmarks Part 1

Discussion in 'Domains, DNS, Email & SSL Certificates' started by eva2000, Mar 5, 2019.

Thread Status:
Not open for further replies.
  1. eva2000

    eva2000 Administrator Staff Member

    42,286
    9,554
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +14,714
    Local Time:
    12:44 AM
    Nginx 1.17.x
    MariaDB 5.5/10.x
    Caddy 0.11.5 was released with TLS 1.3 support when built with Go 1.12 latest release. This thread will cover some HTTP/2 HTTPS TLS 1.3 benchmarks comparing Centmin Mod Nginx 1.15.9 with Caddy 0.11.5 custom built binary with Go 1.12.
    h2load test server:

    The h2load test server is ImpactVPS KVM VPS
    • 2 CPU Core - Intel Xeon Silver 4114 Skylake cpu
    • 4GB ram
    • 60GB NVMe SSD
    • CentOS 7.6 64bit
    • Centmin Mod 123.09beta01 LEMP Stack
    Code (Text):
    lscpu
    Architecture:          x86_64
    CPU op-mode(s):        32-bit, 64-bit
    Byte Order:            Little Endian
    CPU(s):                2
    On-line CPU(s) list:   0,1
    Thread(s) per core:    1
    Core(s) per socket:    1
    Socket(s):             2
    NUMA node(s):          1
    Vendor ID:             GenuineIntel
    CPU family:            6
    Model:                 85
    Model name:            Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz
    Stepping:              4
    CPU MHz:               2199.992
    BogoMIPS:              4399.98
    Virtualization:        VT-x
    Hypervisor vendor:     KVM
    Virtualization type:   full
    L1d cache:             32K
    L1i cache:             32K
    L2 cache:              4096K
    NUMA node0 CPU(s):     0,1
    Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl eagerfpu pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch ibrs ibpb tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f rdseed adx smap clflushopt clwb avx512cd xsaveopt xsavec xgetbv1 arat spec_ctrl
    


    Test Parameters



    I created Centmin Mod Nginx HTTP/2 HTTPS vhost site = test.com via nv command line with autogenerated self-signed RSA 2048bit SSL certificate.
    Code (Text):
    nv -d test.com -s y -u MYFTPUSERNAME

    This will create Nginx HTTP/HTTPS vhost with HTTP/2 HTTPS and self-signed RSA 2048bit SSL certificate automatcially on HTTPS port 443

    Then I created Caddy HTTP/2 HTTPS site with TLS 1.3 on HTTPS port 444 and used Caddy's own internal self-signed SSL certificate generation using the following configs:

    For Caddy generated self-signed SSL certificate generation
    Code (Text):
       tls self_signed
    

    Centmin Mod Nginx 1.15.9 curl header check on port 443 and version info
    Code (Text):
    curl -Isk https://test.com:443/
    HTTP/2 200
    date: Tue, 05 Mar 2019 10:23:16 GMT
    content-type: text/html; charset=utf-8
    content-length: 6552
    last-modified: Tue, 05 Mar 2019 03:17:21 GMT
    vary: Accept-Encoding
    etag: "5c7dea41-1998"
    server: nginx centminmod
    x-powered-by: centminmod
    x-xss-protection: 1; mode=block
    x-content-type-options: nosniff
    accept-ranges: bytes
    

    Caddy 0.11.5 curl header check on port 444 and version info
    Code (Text):
    curl -Isk https://test.com:444/
    HTTP/2 200
    accept-ranges: bytes
    content-type: text/html; charset=utf-8
    etag: "pnvigx520"
    last-modified: Tue, 05 Mar 2019 03:17:21 GMT
    server: Caddy
    x-content-type-options: nosniff
    x-powered-by: Caddy via CentminMod
    x-xss-protection: 1; mode=block
    content-length: 6552
    date: Tue, 05 Mar 2019 10:23:33 GMT
    

    Code (Text):
    caddy -version
    Caddy 0.11.5 (+fda7350 Tue Mar 05 06:04:26 UTC 2019) (unofficial)
    1 file changed, 46 insertions(+), 1 deletion(-)
    caddy/caddymain/run.go
    

    Code (Text):
    caddy -plugins
    Server types:
      http
    
    Caddyfile loaders:
      short
      flag
      default
    
    Other plugins:
      http.authz
      http.awses
      http.awslambda
      http.basicauth
      http.bind
      http.browse
      http.cache
      http.cgi
      http.cors
      http.datadog
      http.errors
      http.expires
      http.expvar
      http.ext
      http.fastcgi
      http.filter
      http.forwardproxy
      http.geoip
      http.git
      http.gopkg
      http.grpc
      http.gzip
      http.header
      http.index
      http.internal
      http.ipfilter
      http.jwt
      http.limits
      http.locale
      http.log
      http.login
      http.mailout
      http.markdown
      http.mime
      http.minify
      http.nobots
      http.pprof
      http.prometheus
      http.proxy
      http.proxyprotocol
      http.push
      http.ratelimit
      http.realip
      http.reauth
      http.redir
      http.request_id
      http.restic
      http.rewrite
      http.root
      http.s3browser
      http.secrets
      http.status
      http.templates
      http.timeouts
      http.upload
      http.webdav
      http.websocket
      on
      tls
      tls.cluster.file
      tls.dns.azure
      tls.dns.cloudflare
      tls.dns.digitalocean
      tls.dns.dnsmadeeasy
      tls.dns.dyn
      tls.dns.godaddy
      tls.dns.googlecloud
      tls.dns.linode
      tls.dns.namecheap
      tls.dns.ns1
      tls.dns.ovh
      tls.dns.powerdns
      tls.dns.rackspace
      tls.dns.route53
      tls.dns.vultr
    

    h2load HTTP/2 HTTPS Benchmarks



    h2load HTTP/2 HTTPS benchmark tool version used
    Code (Text):
    h2load --version
    h2load nghttp2/1.37.0-DEV
    

    h2load test parameters 200 users and 25,000 requests for gzip encoded HTTP/2 HTTPS request/load testing. Each server had 3x h2load test runs.
    Code (Text):
    users=200
    requests=25000
    h2load -t1 -H 'Accept-Encoding: gzip' -c${users} -n${requests} https://test.com:443
    h2load -t1 -H 'Accept-Encoding: gzip' -c${users} -n${requests} https://test.com:444
    


    Centmin Mod Nginx 1.15.9 HTTP/2 HTTPS run 1
    Code (Text):
    users=200
    requests=25000
    h2load -t1 -H 'Accept-Encoding: gzip' -c${users} -n${requests} https://test.com:443
    starting benchmark...
    spawning thread #0: 200 total client(s). 25000 total requests
    TLS Protocol: TLSv1.3
    Cipher: TLS_AES_256_GCM_SHA384
    Server Temp Key: X25519 253 bits
    Application protocol: h2
    progress: 10% done
    progress: 20% done
    progress: 30% done
    progress: 40% done
    progress: 50% done
    progress: 60% done
    progress: 70% done
    progress: 80% done
    progress: 90% done
    progress: 100% done
    
    finished in 4.72s, 5299.41 req/s, 11.73MB/s
    requests: 25000 total, 25000 started, 25000 done, 25000 succeeded, 0 failed, 0 errored, 0 timeout
    status codes: 25000 2xx, 0 3xx, 0 4xx, 0 5xx
    traffic: 55.35MB (58034800) total, 4.94MB (5175000) headers (space savings 26.86%), 49.97MB (52400000) data
                        min         max         mean         sd        +/- sd
    time for request:     2.50ms    282.07ms     35.11ms     16.66ms    92.66%
    time for connect:    42.63ms    326.84ms    158.68ms     77.33ms    58.50%
    time to 1st byte:   181.01ms    383.96ms    318.63ms     35.77ms    64.00%
    req/s           :      26.53       28.83       27.50        0.77    55.00%
    

    Centmin Mod Nginx 1.15.9 HTTP/2 HTTPS run 2
    Code (Text):
    users=200
    requests=25000
    h2load -t1 -H 'Accept-Encoding: gzip' -c${users} -n${requests} https://test.com:443
    starting benchmark...
    spawning thread #0: 200 total client(s). 25000 total requests
    TLS Protocol: TLSv1.3
    Cipher: TLS_AES_256_GCM_SHA384
    Server Temp Key: X25519 253 bits
    Application protocol: h2
    progress: 10% done
    progress: 20% done
    progress: 30% done
    progress: 40% done
    progress: 50% done
    progress: 60% done
    progress: 70% done
    progress: 80% done
    progress: 90% done
    progress: 100% done
    
    finished in 4.66s, 5362.61 req/s, 11.87MB/s
    requests: 25000 total, 25000 started, 25000 done, 25000 succeeded, 0 failed, 0 errored, 0 timeout
    status codes: 25000 2xx, 0 3xx, 0 4xx, 0 5xx
    traffic: 55.35MB (58034800) total, 4.94MB (5175000) headers (space savings 26.86%), 49.97MB (52400000) data
                        min         max         mean         sd        +/- sd
    time for request:     3.68ms    282.83ms     34.26ms     17.93ms    95.15%
    time for connect:    47.09ms    332.00ms    160.37ms     80.32ms    60.00%
    time to 1st byte:   101.30ms    429.20ms    349.01ms     43.27ms    68.00%
    req/s           :      26.87       30.41       28.18        1.19    58.50%
    

    Centmin Mod Nginx 1.15.9 HTTP/2 HTTPS run 3
    Code (Text):
    users=200
    requests=25000
    h2load -t1 -H 'Accept-Encoding: gzip' -c${users} -n${requests} https://test.com:443
    starting benchmark...
    spawning thread #0: 200 total client(s). 25000 total requests
    TLS Protocol: TLSv1.3
    Cipher: TLS_AES_256_GCM_SHA384
    Server Temp Key: X25519 253 bits
    Application protocol: h2
    progress: 10% done
    progress: 20% done
    progress: 30% done
    progress: 40% done
    progress: 50% done
    progress: 60% done
    progress: 70% done
    progress: 80% done
    progress: 90% done
    progress: 100% done
    
    finished in 4.79s, 5223.73 req/s, 11.56MB/s
    requests: 25000 total, 25000 started, 25000 done, 25000 succeeded, 0 failed, 0 errored, 0 timeout
    status codes: 25000 2xx, 0 3xx, 0 4xx, 0 5xx
    traffic: 55.35MB (58034800) total, 4.94MB (5175000) headers (space savings 26.86%), 49.97MB (52400000) data
                         min         max         mean         sd        +/- sd
    time for request:     1.67ms    302.63ms     34.59ms     14.58ms    89.50%
    time for connect:    52.32ms    357.80ms    173.24ms     82.30ms    59.00%
    time to 1st byte:   218.43ms    401.12ms    310.55ms     61.72ms    55.00%
    req/s           :      26.17       29.99       27.88        1.59    56.50%
    


    Caddy 0.11.5 HTTP/2 HTTPS run 1
    Code (Text):
    users=200
    requests=25000
    h2load -t1 -H 'Accept-Encoding: gzip' -c${users} -n${requests} https://test.com:444
    starting benchmark...
    spawning thread #0: 200 total client(s). 25000 total requests
    TLS Protocol: TLSv1.3
    Cipher: TLS_AES_128_GCM_SHA256
    Server Temp Key: X25519 253 bits
    Application protocol: h2
    progress: 10% done
    progress: 20% done
    progress: 30% done
    progress: 40% done
    progress: 50% done
    progress: 60% done
    progress: 70% done
    progress: 80% done
    progress: 90% done
    progress: 100% done
    
    finished in 13.42s, 1862.89 req/s, 3.79MB/s
    requests: 25000 total, 25000 started, 25000 done, 25000 succeeded, 0 failed, 0 errored, 0 timeout
    status codes: 25000 2xx, 0 3xx, 0 4xx, 0 5xx
    traffic: 50.81MB (53279316) total, 408.51KB (418316) headers (space savings 94.67%), 49.97MB (52400000) data
                        min         max         mean         sd        +/- sd
    time for request:      951us    460.43ms    100.43ms     47.72ms    70.94%
    time for connect:    43.01ms       1.29s    506.25ms    414.93ms    65.50%
    time to 1st byte:   273.81ms       1.54s    674.25ms    408.92ms    82.00%
    req/s           :       9.32       10.48        9.58        0.23    76.50%
    

    Caddy 0.11.5 HTTP/2 HTTPS run 2
    Code (Text):
    users=200
    requests=25000
    h2load -t1 -H 'Accept-Encoding: gzip' -c${users} -n${requests} https://test.com:444
    starting benchmark...
    spawning thread #0: 200 total client(s). 25000 total requests
    TLS Protocol: TLSv1.3
    Cipher: TLS_AES_128_GCM_SHA256
    Server Temp Key: X25519 253 bits
    Application protocol: h2
    progress: 10% done
    progress: 20% done
    progress: 30% done
    progress: 40% done
    progress: 50% done
    progress: 60% done
    progress: 70% done
    progress: 80% done
    progress: 90% done
    progress: 100% done
    
    finished in 14.53s, 1721.05 req/s, 3.50MB/s
    requests: 25000 total, 25000 started, 25000 done, 25000 succeeded, 0 failed, 0 errored, 0 timeout
    status codes: 25000 2xx, 0 3xx, 0 4xx, 0 5xx
    traffic: 50.82MB (53283985) total, 413.07KB (422985) headers (space savings 94.61%), 49.97MB (52400000) data
                        min         max         mean         sd        +/- sd
    time for request:     1.09ms    556.96ms    110.05ms     57.95ms    78.02%
    time for connect:    40.10ms       1.04s    446.79ms    325.63ms    62.00%
    time to 1st byte:   343.31ms       1.29s    656.70ms    275.09ms    74.50%
    req/s           :       8.61        9.56        8.80        0.18    77.00%
    

    Caddy 0.11.5 HTTP/2 HTTPS run 3
    Code (Text):
    users=200
    requests=25000
    h2load -t1 -H 'Accept-Encoding: gzip' -c${users} -n${requests} https://test.com:444
    starting benchmark...
    spawning thread #0: 200 total client(s). 25000 total requests
    TLS Protocol: TLSv1.3
    Cipher: TLS_AES_128_GCM_SHA256
    Server Temp Key: X25519 253 bits
    Application protocol: h2
    progress: 10% done
    progress: 20% done
    progress: 30% done
    progress: 40% done
    progress: 50% done
    progress: 60% done
    progress: 70% done
    progress: 80% done
    progress: 90% done
    progress: 100% done
    
    finished in 12.83s, 1949.13 req/s, 3.96MB/s
    requests: 25000 total, 25000 started, 25000 done, 25000 succeeded, 0 failed, 0 errored, 0 timeout
    status codes: 25000 2xx, 0 3xx, 0 4xx, 0 5xx
    traffic: 50.81MB (53276786) total, 406.04KB (415786) headers (space savings 94.70%), 49.97MB (52400000) data
                         min         max         mean         sd        +/- sd
    time for request:      995us    593.22ms     97.70ms     47.46ms    75.41%
    time for connect:    47.90ms    652.23ms    322.09ms    164.51ms    60.00%
    time to 1st byte:   401.57ms    810.19ms    659.48ms     84.67ms    64.00%
    req/s           :       9.75       10.85        9.98        0.21    79.50%
    


    3x h2load HTTP/2 HTTPS TLSv1.3 load test runs' individual and averages.

    centminmod-nginx-vs-caddy-h2load-http2-https-tls13-02.png
     
    Last edited: Mar 5, 2019
    • Winner Winner x 2
Thread Status:
Not open for further replies.