Nginx Official Nginx HTTP/3 QUIC Tech Preview announced

Looks like Nginx folks have released a separate Nginx development branch for official Nginx HTTP/3 QUIC tech preview with Nginx 1.19+ mainline versions built against BoringSSL QUIC with current HTTP/3 h3-27 draft at Introducing a Technology Preview of NGINX Support for QUIC and HTTP/3 - NGINX. Official Nginx HTTP/3 QUIC isn't the same as Cloudflare's released Nginx HTTP/3 QUIC patch which is built with Cloudflare Quiche/BoringSSL for current HTTP/3 h3-29 draft against Nginx 1.16.1 stable only. However, I will be setting up a separate Centmin Mod 123.09beta01 preview branch for adding official Nginx HTTP/3 QUIC support for testing along side Cloudflare's Nginx HTTP/3 QUIC patch

Actual nginx-quic preview branch readme notes at https://hg.nginx.org/nginx-quic/file/tip/README which highlights not all HTTP/3 QUIC features have been fully implemented yet.

---

Code (Text):

1. Introduction

    This is an experimental QUIC [1] / HTTP/3 [2] support for nginx.

    The code is developed in a separate "quic" branch available
    at https://hg.nginx.org/nginx-quic.  Currently it is based
    on nginx mainline 1.19.x. We are planning to merge new nginx
    releases into this branch regularly.

    The project code base is under the same BSD license as nginx.

    The code is at an early alpha level of quality and should not
    be used in production.

    We are working on improving HTTP/3 support with the goal of
    integrating it to the main NGINX codebase.  Expect frequent
    updates of this code and don't rely on it for whatever purpose.

    We'll be grateful for any feedback and code submissions however
    we don't bear any responsibilities for any issues with this code.

    You can always contact us via nginx-devel mailing list [3].

    What works now:

    Currently we support IETF-QUIC draft 27
    Earlier drafts are NOT supported as they have incompatible wire format;

    Newer drafts development (draft-28 at the time of writing) is in progress.
    You may look at src/event/ngx_event_quic.h for alternative values of the
    NGX_QUIC_DRAFT_VERSION macro used to select IETF draft version number.

    nginx should be able to respond to simple HTTP/3 requests over QUIC and
    it should be possible to upload and download big files without errors.

    + The handshake completes successfully
    + One endpoint can update keys and its peer responds correctly
    + 0-RTT data is being received and acted on
    + Connection is established using TLS Resume Ticket
    + A handshake that includes a Retry packet completes successfully
    + Stream data is being exchanged and ACK'ed
    + An H3 transaction succeeded
    + One or both endpoints insert entries into dynamic table and
      subsequently reference them from header blocks

     Not (yet) supported features:

    - Version negotiation
    - ECN, Congestion control and friends as specified in quic-recovery [5]
    - A connection with the spin bit succeeds and the bit is spinning
    - Structured Logging
    - QUIC recovery (proper congestion and flow control)
    - NAT Rebinding
    - Address Mobility
    - Server push
    - HTTP/3 trailers

    Since the code is experimental and still under development,
    a lot of things may not work as expected, for example:

    - ACK handling is basic: every received ack-eliciting packet
      is acknowledged, no ack ranges are used

    - Flow control mechanism is basic and intended to avoid CPU hog and make
      simple interactions possible

    - Not all draft requirements are strictly followed; some of checks are
      omitted for the sake of simplicity of initial implementation

 

Yeah what @buik said Nginx HTTP/3 from Nginx or even Cloudflare is still a work in progress. I personally am now less motivated to want this on origin, given Cloudflare already supports HTTP/3 and only communicates to origin Nginx servers over HTTP/1.1 anyway.

FYI, HTTP/3 via QUIC UDP will also use more CPU and memory resources than HTTP/2 for that performance benefit depending on the HTTP/3 implementation it can be quite high in resource usage.

 

Indeed though my previous sentiments still hold But we will see as Cloudflare also have plans to support HTTP proxying of UDP connections as well - which in theory would allow Cloudflare edge servers to communicate with HTTP/3 QUIC over UDP origin based servers instead of just HTTP/1.1 and HTTP/2 at origin side

 

Centmin Mod already optionally supports BoringSSL and LibreSSL (no active development for support as it's slower than BoringSSL and OpenSSL) and QuicTLS should be easy to add as it's a fork of OpenSSL 1.1.1/3.0 with QUIC supported added. There's also BoringSSL + Cloudflare Quiche for HTTP/3 QUIC support for Nginx. Looking at Centmin Mod's code commit history, BoringSSL support started taking shape back in June 2016. Though it's been ages since I tested Centmin Mod's BoringSSL and LibreSSL support the limitations of BoringSSL aren't ideal so OpenSSL is preferred. i.e. BoringSSL does not support dual RSA + ECDSA SSL certificates and doesn't support OCSP stapling

I developed Centmin Mod's Nginx routines to be flexible in supporting various TLS libraries which makes it one step easier to eventual Centmin Mod Nginx HTTP/3 QUIC support

 

Just checked looks like BoringSSL needed some bug fixes which I'll add for 130.00beta01 soon. Otherwise LibreSSL 3.5.3, OpenSSL 1.1.1/3.0 are working it seems.

Centmin Mod 130.00beta01's Nginx default OpenSSL 1.1.1

Code (Text):

nginx -V
nginx version: nginx/1.23.2 (301022-065714-centos7-ed582fb-br-6e975bc)
built by gcc 11.2.1 20220127 (Red Hat 11.2.1-9) (GCC)
built with OpenSSL 1.1.1q  5 Jul 2022
TLS SNI support enabled

With OpenSSL 3.0.5

Code (Text):

nginx -V
nginx version: nginx/1.23.2 (301022-055900-centos7-ed582fb-br-6e975bc)
built by gcc 11.2.1 20220127 (Red Hat 11.2.1-9) (GCC)
built with OpenSSL 3.0.5 5 Jul 2022
TLS SNI support enabled

and BoringSSL with bug fixes

Code (Text):

nginx -V
nginx version: nginx/1.23.2 (301022-065301-centos7-ed582fb-br-6e975bc)
built by gcc 11.2.1 20220127 (Red Hat 11.2.1-9) (GCC)
built with OpenSSL 1.1.1 (compatible; BoringSSL) (running with BoringSSL)
TLS SNI support enabled

LibreSSL 3.5.3

Code (Text):

nginx -V
nginx version: nginx/1.23.2 (301022-061258-centos7-ed582fb-br-6e975bc)
built by gcc 11.2.1 20220127 (Red Hat 11.2.1-9) (GCC)
built with LibreSSL 3.5.3
TLS SNI support enabled

edit: Seems for LibreSSL only 3.6.0 and higher has experimental BoringSSL QUIC API support added

• https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.6.0-relnotes.txt
• Experimental support for the BoringSSL QUIC API

Code (Text):

nginx -V
nginx version: nginx/1.23.2 (301022-081947-centos7-ed582fb-br-6e975bc)
built by gcc 11.2.1 20220127 (Red Hat 11.2.1-9) (GCC) 
built with LibreSSL 3.6.0
TLS SNI support enabled

 

Still buggy for me but did a quick test of Centmin Mod Nginx with Nginx QUIC HTTP/3 support via quictls openssl 1.1.1q fork TLS library and using my custom built curl with QUIC HTTP/3 support.

Custom curl version with QUIC/HTTP3 support

Code (Text):

/opt/el-compat-quic/bin/curl -V
curl 7.85.0 (x86_64-unknown-linux-gnu) libcurl/7.85.0 OpenSSL/1.1.1q zlib/1.2.7 brotli/1.0.9 zstd/1.5.2 libpsl/0.21.1 (+libicu/71.1) libssh2/1.10.0 nghttp2/1.33.0 ngtcp2/0.8.1 nghttp3/0.8.0-DEV
Release-Date: 2022-08-31
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

Test Centmin Mod Nginx vhost domain1.com with HTTP/3 QUIC enabled which shows

• Connect socket 5 over QUIC
• Using HTTP/3 Stream ID
• HTTP/3 200

Code (Text):

/opt/el-compat-quic/bin/curl -Ikv --http3 https://domain1.com/
*   Trying 111.222.333.444:443...
* Connect socket 5 over QUIC to 111.222.333.444:443
* Skipped certificate verification
* Connected to domain1.com (111.222.333.444) port 443 (#0)
* h2h3 [:method: HEAD]
* h2h3 [:path: /]
* h2h3 [:scheme: https]
* h2h3 [:authority: domain1.com]
* h2h3 [user-agent: curl/7.85.0]
* h2h3 [accept: */*]
* Using HTTP/3 Stream ID: 0 (easy handle 0xa80c30)
> HEAD / HTTP/3
> Host: domain1.com
> user-agent: curl/7.85.0
> accept: */*
>
* ngh3_stream_recv returns 0 bytes and EAGAIN
< HTTP/3 200
HTTP/3 200
< date: Tue, 01 Nov 2022 11:27:01 GMT
date: Tue, 01 Nov 2022 11:27:01 GMT
< content-type: text/html; charset=utf-8
content-type: text/html; charset=utf-8
< content-length: 2
content-length: 2
< last-modified: Sun, 16 Oct 2022 02:09:43 GMT
last-modified: Sun, 16 Oct 2022 02:09:43 GMT
< vary: Accept-Encoding
vary: Accept-Encoding
< etag: "634b67e7-2"
etag: "634b67e7-2"
< server: nginx centminmod
server: nginx centminmod
< x-powered-by: centminmod
x-powered-by: centminmod
< x-cache-status: MISS
x-cache-status: MISS
< accept-ranges: bytes
accept-ranges: bytes

<
* Connection #0 to host domain1.com left intact

 

That is for HTTP/2 only. While Dynamic TLS not sure bug Centmin Mod Nginx will disable both with HTTP/3

 

Not publicly available for testing as Nginx HTTP/3 is still pretty much experimental. If you use Cloudflare in front of your sites it will have HTTP/3 already and don't require HTTP/3 on origin Nginx servers.

 

Yeah Cloudflare's Nginx quiche/HTTP/3 implementation I don't think is getting much updates now with Nginx 1.16 seeing as Cloudflare is moving away from Nginx as a proxy to their own inhouse Pingora How we built Pingora, the proxy that connects Cloudflare to the Internet

So only implementation will be official Nginx QUIC version using quickTLS OpenSSL fork with QUIC port.