Files
Mounir IDRASSI 3e4f7e7720 Reject unknown TLS 1.3 ServerHello extensions
TLS 1.3 does not permit clients to ignore unknown extensions in ServerHello
or HelloRetryRequest. Add a validation pass for these messages so fully
unknown extension types fail early with unsupported_extension.

Keep the existing ignore behavior for other extension contexts and leave
TLS 1.2 ServerHello processing unchanged. Add TLSProxy coverage for the
TLS 1.3 ServerHello case.

Correct the unknown-extension path in tls_collect_extensions(). Collection
still runs before the new TLS 1.3 validation, and verify_extension()
returns success with thisex == NULL for fully unknown extension types.
The previous idx calculation used invalid pointer arithmetic on NULL in
that path. Map unknown extensions explicitly outside the built-in extension
range instead, preserving the existing ignore semantics while avoiding
undefined behavior.

Update the truncated HRR ECH corruption vector to keep testing malformed
ECH length after the new unknown-extension validation. The old 0xdddd bytes
were only padding to preserve the replacement length, but they now trigger
the unknown-extension check before the ECH parser. Use a known HRR cookie
extension as padding so the test still reaches the intended
SSL_R_LENGTH_MISMATCH path.

Reviewed-by: Andrew Dinh <andrewd@openssl.org>
Reviewed-by: Daniel Kubec <kubec@openssl.foundation>
Reviewed-by: Tomas Mraz <tomas@openssl.foundation>
MergeDate: Thu Jul 23 15:55:09 2026
(Merged from https://github.com/openssl/openssl/pull/31086)
2026-07-23 17:55:07 +02:00
..
2021-09-27 09:39:17 +10:00
2026-06-30 12:47:03 +02:00

State Machine Design

This file provides some guidance on the thinking behind the design of the state machine code to aid future maintenance.

The state machine code replaces an older state machine present in OpenSSL versions 1.0.2 and below. The new state machine has the following objectives:

  • Remove duplication of state code between client and server
  • Remove duplication of state code between TLS and DTLS
  • Simplify transitions and bring the logic together in a single location so that it is easier to validate
  • Remove duplication of code between each of the message handling functions
  • Receive a message first and then work out whether that is a valid transition - not the other way around (the other way causes lots of issues where we are expecting one type of message next but actually get something else)
  • Separate message flow state from handshake state (in order to better understand each)
    • message flow state = when to flush buffers; handling restarts in the event of NBIO events; handling the common flow of steps for reading a message and the common flow of steps for writing a message etc
    • handshake state = what handshake message are we working on now
  • Control complexity: only the state machine can change state: keep all the state changes local to the state machine component

The message flow state machine is divided into a reading sub-state machine and a writing sub-state machine. See the source comments in statem.c for a more detailed description of the various states and transitions possible.

Conceptually the state machine component is designed as follows:

                      libssl
                         |
-------------------------|-----statem.h------------------------------------
                         |
                  _______V____________________
                 |                            |
                 |    statem.c                |
                 |                            |
                 |    Core state machine code |
                 |____________________________|
      statem_local.h     ^          ^
               _________|          |_______
              |                            |
 _____________|____________   _____________|____________
|                          | |                          |
| statem_clnt.c            | | statem_srvr.c            |
|                          | |                          |
| TLS/DTLS client specific | | TLS/DTLS server specific |
| state machine code       | | state machine code       |
|__________________________| |__________________________|
             |        |_______________|__       |
             |        ________________|  |      |
             |       |                   |      |
 ____________V_______V________   ________V______V_______________
|                             | |                               |
| statem_lib.c                | | statem_dtls.c                 |
|                             | |                               |
| Non core functions common   | | Non core functions common to  |
| to both servers and clients | | both DTLS servers and clients |
|_____________________________| |_______________________________|