QUIC: using QUIC API introduced in OpenSSL 3.5. #646

pluknet · 2025-04-22T01:48:10Z

QUIC: using QUIC API introduced in OpenSSL 3.5.

Similarly to QUIC API originated in BoringSSL, this API allows to
register custom TLS callbacks for a QUIC implementation external
to OpenSSL. See the SSL_set_quic_tls_cbs manual page for details.

Notably, due to a different approach in managing data flows,
ngx_quic_handle_crypto_frame() was streamlined to always write
an incoming buffer to the crypto context. For libraries using
SSL_provide_quic_data(), this results in a transient allocation
of chain links and respecting buffers for CRYPTO frames received
in order.

Also, in 0-RTT enabled configurations, SSL_do_handshake() appears
to signal about a successfully completed TLS handshake before the
client's Finished message is received (and verified). To prevent
sending Application Data to a not yet authenticated peer, special
handling is added, which goes in line with RFC 8446, section 4.4.4
and RFC 9001, section 4.1.1.

Closes #537.

pluknet · 2025-04-22T12:26:49Z

Added a distinct macro for OpenSSL QUIC API to improve readability, per @bavshin-f5 suggestion.

pluknet · 2025-04-22T13:15:38Z

NGX_QUIC_OPENSSL_API autotest is now run first.
This matches OpenSSL as the most commonly used library, and allows to reduce indentation in autotests.

pluknet · 2025-04-22T13:20:46Z

Removed extraneous debug crept in during development.

auto/lib/openssl/conf

pluknet · 2025-04-24T17:01:31Z

Addressed @bavshin-f5 comments, applied some style.

pluknet · 2025-04-30T15:31:45Z

Some ministat on instructions retired.

no debug, -O2
master_process off;
1000 handshakes

Should be no difference:

OpenSSL 3.4, QUIC compat layer, master branch,
OpenSSL 3.4, QUIC compat later, this branch,

x ossl34-master
+ ossl34-patched
+------------------------------------------------------------------------------+
|x                  x       +     +               x  +  x   +                 +|
|          |___________________|___A______________MA_M_____|___________|       |
+------------------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5 1.2463821e+10 1.2470292e+10 1.2469577e+10 1.2467874e+10     2811484.5
+   5 1.2467053e+10 1.2472955e+10 1.2470048e+10 1.2469718e+10     2395157.9
No difference proven at 95.0% confidence

Expected difference:

OpenSSL 3.5, QUIC compat layer, master branch
OpenSSL 3.5, QUIC native callbacks, this branch

x ossl35-master
+ ossl35-patched
+------------------------------------------------------------------------------+
| +                                                                            |
| +                                                                          x |
| +                                                                          xx|
|++                                                                          xx|
||A                                                                          A||
+------------------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5 1.1762029e+10 1.1767168e+10 1.1764533e+10 1.1764679e+10     2147454.8
+   5 1.1470993e+10 1.1473826e+10 1.1473589e+10 1.1473049e+10     1171440.3
Difference at 95.0% confidence
	-2.9163e+08 +/- 2.52269e+06
	-2.47886% +/- 0.0214429%
	(Student's t, pooled s = 1.72972e+06)

Conclusions:

no performance regression if keep using compat layer
2..3% performance win after switch to native callbacks

arut

Looks like in OpenSSL codepath there's no check SSL_AD_NO_APPLICATION_PROTOCOL. In BoringSSL codepath it's done in ngx_quic_add_handshake_data(). I personally would love this code to be moved elsewhere, like after the handshake, in which case it would serve both cases.

src/event/quic/ngx_event_quic_ssl.c

pluknet · 2025-05-06T13:35:36Z

Looks like in OpenSSL codepath there's no check SSL_AD_NO_APPLICATION_PROTOCOL. In BoringSSL codepath it's done in ngx_quic_add_handshake_data(). I personally would love this code to be moved elsewhere, like after the handshake, in which case it would serve both cases.

Application level already provides such handling, the missing part is when no ALPN extension is sent.
While I concur with you to make this check in a library-agnostic way such as in the SSL_do_handshake() handling,
this may be too late and may result in unwanted progressing of the handshake. I addressed this in a different way (the updated change to come).

Similarly to QUIC API originated in BoringSSL, it allows to register custom TLS callbacks for external QUIC implementation. For details, see the SSL_set_quic_tls_cbs manual page. Due to a different approach used in OpenSSL 3.5, handling of CRYPTO frames was streamlined to always write an incoming CRYPTO buffer to the crypto context. Using SSL_provide_quic_data(), this results in transient allocation of chain links and buffers for CRYPTO frames received in order. Testing didn't reveal performance degradation of QUIC handshakes, nginx#646 provides specific results. Unlike for the BoringSSL API, the OSSL_FUNC_SSL_QUIC_TLS_CRYPTO_SEND callback is made to return success for missing ALPN or QUIC transport parameters extensions but stop progressing the QUIC handshake further. This is used to avoid reporting such errors inappropriately as "record layer failure" at the "crit" log level due to the failed callback. SSL_do_handshake() is tested for SSL_ERROR_WANT_WRITE to handle this. In 0-RTT enabled configurations, SSL_do_handshake() appears to signal a successfully completed TLS handshake before the Finished message is received (and verified). Special handling is added to prevent sending Application Data prior to authentication of the handshake. This goes in line with RFC 8446, section 4.4.4, and RFC 9001, section 4.1.1.

pluknet · 2025-05-06T16:52:59Z

some preparatory cleanup changes
addressed @arut comments
got rid of ssl_encryption_level_t
last minute breakages & fixes

src/event/quic/ngx_event_quic_openssl_compat.c

src/event/quic/ngx_event_quic_ssl.c

arut · 2025-05-08T12:42:55Z

src/event/quic/ngx_event_quic_ssl.c

+            if (qc->error
+                != NGX_QUIC_ERR_CRYPTO(SSL_AD_NO_APPLICATION_PROTOCOL))


Could you elaborate on when does this play a role? Normally this would result in an error and we should not return here afterwards. I expect TLS stack to trigger an error and abort the handshake.

Also, qc->error may be overwritten, so this is not a fully reliable condition. Probably just checking qc->error == (ngx_uint_t) -1 could be enough. If another error has already happened, nothing else matter.

Could you elaborate on when does this play a role?

This is needed for OpenSSL compatibility layer, when the ALPN extension is missing.
The reason is that it doesn't propagate the add_handshake_data error up to TLS stack, due to how this is implemented from within the void OpenSSL message callback:

if (com->method->add_handshake_data(ssl, level, buf, len) != 1) { goto failed; } ... failed: ngx_post_event(&qc->close, &ngx_posted_events);

So the callback may be called multiple times before closing QUIC connection:

2025/05/13 18:05:17 [debug] 25357#1227751: *1 quic ngx_quic_add_handshake_data 2025/05/13 18:05:17 [info] 25357#1227751: *1 quic unsupported protocol in ALPN extension while handling frames, client: 127.0.0.1, server: 127.0.0.1:8980 ... 2025/05/13 18:05:17 [debug] 25357#1227751: *1 quic compat add transport params 2025/05/13 18:05:17 [debug] 25357#1227751: *1 quic compat tx hs len:126 2025/05/13 18:05:17 [debug] 25357#1227751: *1 quic ngx_quic_add_handshake_data 2025/05/13 18:05:17 [info] 25357#1227751: *1 quic unsupported protocol in ALPN extension while handling frames, client: 127.0.0.1, server: 127.0.0.1:8980 2025/05/13 18:05:17 [debug] 25357#1227751: *1 update posted event 000061E000005B58 2025/05/13 18:05:17 [debug] 25357#1227751: *1 quic compat tx hs len:744 2025/05/13 18:05:17 [debug] 25357#1227751: *1 quic ngx_quic_add_handshake_data 2025/05/13 18:05:17 [info] 25357#1227751: *1 quic unsupported protocol in ALPN extension while handling frames, client: 127.0.0.1, server: 127.0.0.1:8980 2025/05/13 18:05:17 [debug] 25357#1227751: *1 update posted event 000061E000005B58 2025/05/13 18:05:17 [debug] 25357#1227751: *1 quic compat tx hs len:264 2025/05/13 18:05:17 [debug] 25357#1227751: *1 quic ngx_quic_add_handshake_data 2025/05/13 18:05:17 [info] 25357#1227751: *1 quic unsupported protocol in ALPN extension while handling frames, client: 127.0.0.1, server: 127.0.0.1:8980 2025/05/13 18:05:17 [debug] 25357#1227751: *1 update posted event 000061E000005B58 2025/05/13 18:05:17 [debug] 25357#1227751: *1 quic compat tx hs len:36 2025/05/13 18:05:17 [debug] 25357#1227751: *1 quic ngx_quic_add_handshake_data 2025/05/13 18:05:17 [info] 25357#1227751: *1 quic unsupported protocol in ALPN extension while handling frames, client: 127.0.0.1, server: 127.0.0.1:8980

In opposite to e.g. using BoringSSL:

2025/05/13 17:52:46 [info] 20535#1211146: *1 SSL_do_handshake() failed (SSL: error:10000133:SSL routines:OPENSSL_internal:NO_APPLICATION_PROTOCOL) while handling frames, client: 127.0.0.1, server: 127.0.0.1:8980

For the record, while QuicTLS and LibreSSL adopted the BoringSSL API model, they both fail to properly return an appropriate handshake error, which ends up in using the critical log level in nginx:

2025/05/13 18:30:51 [info] 39976#1271039: *1 quic unsupported protocol in ALPN extension while handling frames, client: 127.0.0.1, server: 127.0.0.1:8980 2025/05/13 18:30:51 [crit] 39976#1271039: *1 SSL_do_handshake() failed (SSL: error:1402B29B:SSL routines:ACCEPT_SW_SRVR_HELLO:QUIC: internal error) while handling frames, client: 127.0.0.1, server: 127.0.0.1:8980

For comparison, this can be handled properly with OpenSSL CBS (as currently done, by triggering WANT_WRITE):

2025/05/13 18:38:08 [info] 49160#1292115: *1 quic unsupported protocol in ALPN extension while handling frames, client: 127.0.0.1, server: 127.0.0.1:8980 2025/05/13 18:38:08 [debug] 49160#1292115: *1 SSL_do_handshake: -1 2025/05/13 18:38:08 [debug] 49160#1292115: *1 SSL_get_error: 3 2025/05/13 18:38:08 [debug] 49160#1292115: *1 quic packet done rc:-1 level:init decr:1 pn:0 perr:0 2025/05/13 18:38:08 [debug] 49160#1292115: *1 quic close initiated rc:-1 2025/05/13 18:38:08 [debug] 49160#1292115: *1 quic close immediate term:1 drain:0 error:376 "unsupported protocol in ALPN extension" ...```

Also, qc->error may be overwritten, so this is not a fully reliable condition. Probably just checking qc->error == (ngx_uint_t) -1 could be enough. If another error has already happened, nothing else matter.

As discussed in private, I rewrote this part to make qc->error initialization back to zero, again.
After compat layer simplification to no longer post close event, it seems that we no longer
have ngx_quic_close_connection() consumers that need such adjustment to -1, so simplified too.
Also this eliminates a suspicious place when we calloc(qc), then reset qc->error to -1 later in another place.

Subsequently, this check is replaced with qc->error == 0.

src/event/quic/ngx_event_quic_ssl.c

arut · 2025-05-10T14:41:54Z

src/event/quic/ngx_event_quic_ssl.c

+    if (!ngx_quic_keys_available(qc->keys, NGX_QUIC_ENCRYPTION_APPLICATION, 0))
+    {
+
+        /*
+         * OpenSSL fails to release read keys in the full SSL handshake by the
+         * time it was completed in configurations with enabled early data
+         */
+
+        return NGX_OK;
+    }
+


This looks like a workaround. OpenSSL suspends the handshake for us to read early data. Obviously we don't need this and want to continue. We can use SSL_is_init_finished() to test if the handshake is complete. For early data the state will be TLS_ST_EARLY_DATA and this function will return 0. In fact, we can substitute SSL_in_init() above with !SSL_is_init_finished() to cover the early data case.

In OpenSSL these functions are implemented as follows:

int SSL_in_init(const SSL *s) { ... return sc->statem.in_init; } int SSL_is_init_finished(const SSL *s) { ... return !(sc->statem.in_init) && (sc->statem.hand_state == TLS_ST_OK); }

In BoringSSL it's even simpler and obviously won't hurt:

int SSL_is_init_finished(const SSL *ssl) { return !SSL_in_init(ssl); }

LibreSSL is still using the old OpenSSL code here, which seems to be OK as well:

#define SSL_is_init_finished(a) (SSL_state((a)) == SSL_ST_OK) #define SSL_in_init(a) (SSL_state((a))&SSL_ST_INIT)

Indeed, replacing SSL_in_init() with SSL_is_init_finished() is a more proper and generic fix.
This also allows to remove testing SSL_do_handshake() in the condition, because !SSL_is_init_finished() is a superset of SSL_do_handshake() <= 0.
Thanks for the suggestion.

For the record, below is the result of testing SSL_do_handshake(), SSL_in_init(), and SSL_is_init_finished() in:

initial handshake

resumed handshake with accepted early data

combinations after ClientHello / ClientHello+ClientFinished

As seen from the table, BoringSSL and OpenSSL 3.5 (CBS) are special cases:
SSL_do_handshake() returns 1 for the incomplete handshake, addressed by testing SSL_is_init_finished().

** OpenSSL 3.4 uses compat layer, no 0-RTT support
*** LibreSSL doesn't support TLSv1.3 session resumption

Library / case SSL_do_handshake SSL_in_init SSL_is_init_finished

OpenSSL 3.4 / initial / CH -1 1 0

OpenSSL 3.4 / initial / CF 1 0 1

LibreSSL / initial / CH -1 8192 0

LibreSSL / initial / CF 1 0 1

QuicTLS / initial / CH 0 0 0

QuicTLS / initial / CF 1 0 1

QuicTLS / 0-RTT / CH -1 1 0

QuicTLS / 0-RTT / CF 1 0 1

BoringSSL / initial / CH -1 1 0

BoringSSL / initial / CF 1 0 1

BoringSSL / 0-RTT / CH 1 1 0

BoringSSL / 0-RTT / CF 1 0 1

OpenSSL 3.5 / initial / CH 1 0 0

OpenSSL 3.5 / initial / CF 1 0 1

OpenSSL 3.5 / 0-RTT / CH -1 1 0

OpenSSL 3.5 / 0-RTT / CF 1 0 1

Hi! OpenSSL 3.5 / initial / CH and OpenSSL 3.5 / 0-RTT / CH look unusual in your results. Could you verify? It might be an oversight. Thanks!

hi @baoyi84930
that's what I described above as:

SSL_do_handshake() returns 1 for the incomplete handshake, addressed by testing SSL_is_init_finished().

@pluknet Yes, I agree. But this data may better match your description. 😄

Library / case SSL_do_handshake SSL_in_init SSL_is_init_finished

OpenSSL 3.5 / initial / CH -1 1 0

OpenSSL 3.5 / 0-RTT / CH 1 0 0

You can verify this yourself by running h3_ssl_early_data.t from nginx/nginx-tests repo with simple patch for extended logging:

diff --git a/src/event/quic/ngx_event_quic_ssl.c b/src/event/quic/ngx_event_quic_ssl.c index 376c219be..f3e73fe18 100644 --- a/src/event/quic/ngx_event_quic_ssl.c +++ b/src/event/quic/ngx_event_quic_ssl.c @@ -691,7 +691,9 @@ ngx_quic_handshake(ngx_connection_t *c, ngx_uint_t level) n = SSL_do_handshake(ssl_conn); - ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, "SSL_do_handshake: %d", n); + ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0, + "SSL_do_handshake:%d SSL_in_init:%d SSL_is_init_finished:%d", + n, SSL_in_init(ssl_conn), SSL_is_init_finished(ssl_conn)); if (qc->error) { return NGX_ERROR;

$ ./objs/nginx -V nginx version: nginx/1.29.0 built by clang 16.0.0 (clang-1600.0.26.6) built with OpenSSL 3.5.1-dev ...

$ TEST_NGINX_CATLOG=1 perl h3_ssl_early_data.t | grep SSL_do_handshake

@pluknet you are right, I misunderstood. This behavior is inherited from SSL_read_early_data. When 0-RTT is enabled, SSL_do_handshake returns 1, and then SSL_read_early_data handles it. However, in the quic_tls scenario, returning 1 is indeed confusing. I tried to modify it, you can try this patch，perhaps this would be better.

diff --git a/ssl/statem/statem_srvr.c b/ssl/statem/statem_srvr.c index b93a97999d..95b675ba5f 100644 --- a/ssl/statem/statem_srvr.c +++ b/ssl/statem/statem_srvr.c @@ -840,17 +840,15 @@ WORK_STATE ossl_statem_server_pre_work(SSL_CONNECTION *s, WORK_STATE wst) && (s->s3.flags & TLS1_FLAGS_STATELESS) == 0) return WORK_FINISHED_CONTINUE; - /* - * In QUIC with 0-RTT we just carry on when otherwise we would stop - * to allow the server to read early data - */ - if (SSL_NO_EOED(s) && s->ext.early_data == SSL_EARLY_DATA_ACCEPTED - && s->early_data_state != SSL_EARLY_DATA_FINISHED_READING) { - s->early_data_state = SSL_EARLY_DATA_FINISHED_READING; - if (!ssl->method->ssl3_enc->change_cipher_state(s, SSL3_CC_HANDSHAKE + if (SSL_NO_EOED(s)) { + if (s->ext.early_data == SSL_EARLY_DATA_ACCEPTED + && s->early_data_state != SSL_EARLY_DATA_FINISHED_READING) { + s->early_data_state = SSL_EARLY_DATA_FINISHED_READING; + if (!ssl->method->ssl3_enc->change_cipher_state(s, SSL3_CC_HANDSHAKE | SSL3_CHANGE_CIPHER_SERVER_READ)) { - SSLfatal(s, SSL_AD_INTERNAL_ERROR, ERR_R_INTERNAL_ERROR); - return WORK_ERROR; + SSLfatal(s, SSL_AD_INTERNAL_ERROR, ERR_R_INTERNAL_ERROR); + return WORK_ERROR; + } } return WORK_FINISHED_SWAP; }

It was made unused in d15f8f2 after introducing reusable crypto contexts.

Similarly to QUIC API originated in BoringSSL, it allows to register custom TLS callbacks for external QUIC implementation. For details, see the SSL_set_quic_tls_cbs manual page. Due to a different approach used in OpenSSL 3.5, handling of CRYPTO frames was streamlined to always write an incoming CRYPTO buffer to the crypto context. Using SSL_provide_quic_data(), this results in transient allocation of chain links and buffers for CRYPTO frames received in order. Testing didn't reveal performance degradation of QUIC handshakes, nginx#646 provides specific results.

pluknet · 2025-05-16T10:38:27Z

changes:

added ref to d15f8f2
removed "quic" prefixing from error messages, conformant error text, log level fixes, dead code removal
reset qc->error to 0 again (partially reverts d3fb12d)
new NGX_QUIC_QUICTLS_API, all libSSL macros moved to ngx_event_quic_connection.h
internal encryption level macros, related reshuffling
callbacks dont' fail
fixes in cbs error path
ngx_quic_crypto_provide() again
SSL_is_init_finished, ossl3.5 hack removal

ALPN support is present in all libraries that have QUIC support, it is safe to compile it unconditionally.

Various errors reported by SSL_do_handshake() are now logged at the "info" or "crit" level, akin to handshakes on regular TCP connections.

Similarly to QUIC API originated in BoringSSL, it allows to register custom TLS callbacks for external QUIC implementation. For details, see the SSL_set_quic_tls_cbs manual page. Due to a different approach used in OpenSSL 3.5, handling of CRYPTO frames was streamlined to always write an incoming CRYPTO buffer to the crypto context. Using SSL_provide_quic_data(), this results in transient allocation of chain links and buffers for CRYPTO frames received in order. Testing didn't reveal performance degradation of QUIC handshakes, nginx#646 provides specific results.

pluknet · 2025-05-21T14:39:29Z

addressed comments
SSL API macros commit rebased as a preparatory change
refined commit messages

arut · 2025-05-21T15:45:06Z

I suggest moving ngx_quic_crypto_provide() out of ngx_quic_handshake() for simplicity and similarity with ngx_ssl_handshake(). Plus a couple of commit log fixes, discussed privately. Otherwise looks ok to me.

Similarly to the QUIC API originated in BoringSSL, this API allows to register custom TLS callbacks for an external QUIC implementation. See the SSL_set_quic_tls_cbs manual page for details. Due to a different approach used in OpenSSL 3.5, handling of CRYPTO frames was streamlined to always write an incoming CRYPTO buffer to the crypto context. Using SSL_provide_quic_data(), this results in transient allocation of chain links and buffers for CRYPTO frames received in order. Testing didn't reveal performance degradation of QUIC handshakes, nginx#646 provides specific results.

pluknet · 2025-05-21T17:15:22Z

Addressed comments, also:

ngx_quic_handshake() is rebased as a preparatory change
ngx_quic_crypto_provide() call is moved
Ultimately this results in a tiny change.

Also, as discussed in private, removed the "quic" prefix renaming patch from the series, to facilitate with review.

Similarly to the QUIC API originated in BoringSSL, this API allows to register custom TLS callbacks for an external QUIC implementation. See the SSL_set_quic_tls_cbs manual page for details. Due to a different approach used in OpenSSL 3.5, handling of CRYPTO frames was streamlined to always write an incoming CRYPTO buffer to the crypto context. Using SSL_provide_quic_data(), this results in transient allocation of chain links and buffers for CRYPTO frames received in order. Testing didn't reveal performance degradation of QUIC handshakes, nginx#646 provides specific results.

pluknet · 2025-05-21T17:40:27Z

Afterparty cosmetic fixes, mostly fallout from the "quic" prefix patch removal.

src/event/quic/ngx_event_quic_ssl.c

Logging level for such errors, which should not normally happen, is changed to NGX_LOG_ALERT, and ngx_log_error() is replaced with ngx_ssl_error() for consistency with the rest of the code.

Changed handshake callbacks to always return success. This allows to avoid logging SSL_do_handshake() errors with empty or cryptic "internal error" OpenSSL error messages at the inappropriate "crit" log level. Further, connections with failed callbacks are closed now right away when using OpenSSL compat layer. This change supersedes and reverts c37fdcd, with the conditions to check callbacks invocation kept to slightly improve code readability of control flow; they are optimized out in the resulting assembly code.

Following the previous change that removed posting a close event in OpenSSL compat layer, now ngx_quic_close_connection() is always called on error path with either NGX_ERROR or qc->error set. This allows to remove a special value -1 served as a missing error, which simplifies the code. Partially reverts d3fb12d. Also, this improves handling of the draining connection state, which consists of posting a close event with NGX_OK and no qc->error set, where it was previously converted to NGX_QUIC_ERR_INTERNAL_ERROR. Notably, this is rather a cosmetic fix, because drained connections do not send any packets including CONNECTION_CLOSE, and qc->error is not otherwise used.

Previously, they might be logged on every add_handshake_data callback invocation when using OpenSSL compat layer and processing coalesced handshake messages. Further, the ALPN error message is adjusted to signal the missing extension. Possible reasons were previously narrowed down with ebb6f7d changes in the ALPN callback that is invoked earlier in the handshake.

All definitions now set in ngx_event_quic.h, this includes moving NGX_QUIC_OPENSSL_COMPAT from autotests to compile time. Further, to improve code readability, a new NGX_QUIC_QUICTLS_API macro is used for QuicTLS that provides old BoringSSL QUIC API.

It is now called from ngx_quic_handle_crypto_frame(), prior to proceeding with the handshake. With this logic removed, the handshake function is renamed to ngx_quic_handshake() to better match ngx_ssl_handshake().

Encryption level values are decoupled from ssl_encryption_level_t, which is now limited to BoringSSL QUIC callbacks, with mappings provided. Although the values match, this provides a technically safe approach, in particular, to access protection level sized arrays. In preparation for using OpenSSL 3.5 TLS callbacks.

Using SSL_in_init() to inspect a handshake state was replaced with SSL_is_init_finished(). This represents a more complete fix to the BoringSSL issue addressed in 22671b3. This provides awareness of the early data handshake state when using OpenSSL 3.5 TLS callbacks in 0-RTT enabled configurations, which, in particular, is used to avoid premature completion of the initial TLS handshake, before required client handshake messages are received. This is a non-functional change when using BoringSSL. It supersedes testing non-positive SSL_do_handshake() results in all supported SSL libraries, hence simplified. In preparation for using OpenSSL 3.5 TLS callbacks.

Similarly to the QUIC API originated in BoringSSL, this API allows to register custom TLS callbacks for an external QUIC implementation. See the SSL_set_quic_tls_cbs manual page for details. Due to a different approach used in OpenSSL 3.5, handling of CRYPTO frames was streamlined to always write an incoming CRYPTO buffer to the crypto context. Using SSL_provide_quic_data(), this results in transient allocation of chain links and buffers for CRYPTO frames received in order. Testing didn't reveal performance degradation of QUIC handshakes, nginx#646 provides specific results.

arut

Looks good to me.

Similarly to the QUIC API originated in BoringSSL, this API allows to register custom TLS callbacks for an external QUIC implementation. See the SSL_set_quic_tls_cbs manual page for details. Due to a different approach used in OpenSSL 3.5, handling of CRYPTO frames was streamlined to always write an incoming CRYPTO buffer to the crypto context. Using SSL_provide_quic_data(), this results in transient allocation of chain links and buffers for CRYPTO frames received in order. Testing didn't reveal performance degradation of QUIC handshakes, nginx#646 provides specific results.

pluknet requested a review from arut April 22, 2025 01:48

pluknet force-pushed the quic-ossl35 branch from e2ee019 to a4c966a Compare April 22, 2025 12:25

pluknet requested a review from bavshin-f5 April 22, 2025 12:27

pluknet force-pushed the quic-ossl35 branch 2 times, most recently from bde5f2d to 2c4e5dc Compare April 22, 2025 13:11

pluknet force-pushed the quic-ossl35 branch from 2c4e5dc to 83c6922 Compare April 22, 2025 13:20

bavshin-f5 reviewed Apr 22, 2025

View reviewed changes

auto/lib/openssl/conf Outdated Show resolved Hide resolved

pluknet force-pushed the quic-ossl35 branch from 83c6922 to ffbdec9 Compare April 24, 2025 15:32

pluknet self-assigned this Apr 28, 2025

arut reviewed May 5, 2025

View reviewed changes

src/event/quic/ngx_event_quic_ssl.c Outdated Show resolved Hide resolved

src/event/quic/ngx_event_quic_ssl.c Show resolved Hide resolved

src/event/quic/ngx_event_quic_ssl.c Outdated Show resolved Hide resolved

pluknet force-pushed the quic-ossl35 branch from ffbdec9 to fac7c96 Compare May 6, 2025 16:11

pluknet force-pushed the quic-ossl35 branch from fac7c96 to f66ebdb Compare May 6, 2025 16:42

pluknet force-pushed the quic-ossl35 branch from f66ebdb to b6b1de0 Compare May 6, 2025 16:48

arut reviewed May 10, 2025

View reviewed changes

QUIC: removed level field from ngx_quic_compat_record_t.

1be3fa6

It was made unused in d15f8f2 after introducing reusable crypto contexts.

pluknet force-pushed the quic-ossl35 branch from 68f2e0a to 465b657 Compare May 16, 2025 10:25

baoyi84930 mentioned this pull request May 17, 2025

SSL_do_handshake return value is confusing when 0-RTT is enabled via QUIC TLS API openssl/openssl#27650

Open

Maryna-f5 added this to the nginx-1.29.0 milestone May 19, 2025

pluknet added 2 commits May 21, 2025 16:38

QUIC: removed ALPN feature test.

5cbd006

ALPN support is present in all libraries that have QUIC support, it is safe to compile it unconditionally.

QUIC: logging level of handshake errors.

a98741d

Various errors reported by SSL_do_handshake() are now logged at the "info" or "crit" level, akin to handshakes on regular TCP connections.

pluknet force-pushed the quic-ossl35 branch from 465b657 to 8db0af8 Compare May 21, 2025 14:38

pluknet force-pushed the quic-ossl35 branch from 8db0af8 to ab95a43 Compare May 21, 2025 17:12

pluknet force-pushed the quic-ossl35 branch from ab95a43 to bb81e9e Compare May 21, 2025 17:38

arut reviewed May 22, 2025

View reviewed changes

src/event/quic/ngx_event_quic_ssl.c Outdated Show resolved Hide resolved

pluknet added 9 commits May 22, 2025 19:25

QUIC: logging of SSL library errors.

40842f7

Logging level for such errors, which should not normally happen, is changed to NGX_LOG_ALERT, and ngx_log_error() is replaced with ngx_ssl_error() for consistency with the rest of the code.

QUIC: factored out SSL_provide_quic_data() to the helper function.

3733446

It is now called from ngx_quic_handle_crypto_frame(), prior to proceeding with the handshake. With this logic removed, the handshake function is renamed to ngx_quic_handshake() to better match ngx_ssl_handshake().

pluknet force-pushed the quic-ossl35 branch from bb81e9e to 6af1dc0 Compare May 22, 2025 15:55

arut approved these changes May 23, 2025

View reviewed changes

arut merged commit 6a134df into nginx:master May 23, 2025
1 check passed

pluknet deleted the quic-ossl35 branch May 23, 2025 15:56

Maryna-f5 modified the milestones: nginx-1.29.0, nginx-1.29.1 Jun 26, 2025

Maryna-f5 added the feature label Jul 24, 2025

		if (qc->error
		!= NGX_QUIC_ERR_CRYPTO(SSL_AD_NO_APPLICATION_PROTOCOL))

Library / case	SSL_do_handshake	SSL_in_init	SSL_is_init_finished
OpenSSL 3.4 / initial / CH	-1	1	0
OpenSSL 3.4 / initial / CF	1	0	1
LibreSSL / initial / CH	-1	8192	0
LibreSSL / initial / CF	1	0	1
QuicTLS / initial / CH	0	0	0
QuicTLS / initial / CF	1	0	1
QuicTLS / 0-RTT / CH	-1	1	0
QuicTLS / 0-RTT / CF	1	0	1
BoringSSL / initial / CH	-1	1	0
BoringSSL / initial / CF	1	0	1
BoringSSL / 0-RTT / CH	1	1	0
BoringSSL / 0-RTT / CF	1	0	1
OpenSSL 3.5 / initial / CH	1	0	0
OpenSSL 3.5 / initial / CF	1	0	1
OpenSSL 3.5 / 0-RTT / CH	-1	1	0
OpenSSL 3.5 / 0-RTT / CF	1	0	1

QUIC: using QUIC API introduced in OpenSSL 3.5. #646

QUIC: using QUIC API introduced in OpenSSL 3.5. #646

Uh oh!

Conversation

pluknet commented Apr 22, 2025

Uh oh!

pluknet commented Apr 22, 2025

Uh oh!

pluknet commented Apr 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pluknet commented Apr 22, 2025

Uh oh!

Uh oh!

pluknet commented Apr 24, 2025

Uh oh!

pluknet commented Apr 30, 2025

Uh oh!

arut left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pluknet commented May 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pluknet commented May 6, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

baoyi84930 May 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pluknet commented May 16, 2025

Uh oh!

pluknet commented May 21, 2025

Uh oh!

arut commented May 21, 2025

Uh oh!

pluknet commented May 21, 2025

Uh oh!

pluknet commented May 21, 2025

Uh oh!

Uh oh!

arut left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

pluknet commented Apr 22, 2025 •

edited

Loading

pluknet commented May 6, 2025 •

edited

Loading

baoyi84930 May 16, 2025 •

edited

Loading