Unauthenticated TLS records can exhaust memory or CPU during a handshake

Reseau 1.1.0 through 1.3.1 contain two resource-exhaustion conditions in the native TLS 1.2 and TLS 1.3 record layers. An unauthenticated network peer can send records that consume memory without a bound or keep a handshake task processing non-progressing input.

Impact

A peer can retain approximately 16 KiB of plaintext per record until the process exhausts memory, or stream empty handshake records to keep a connection task consuming CPU. Multiple connections can amplify either condition and impair service availability.

Technical details

This advisory consolidates two handshake-time input-validation defects:

1. After obtaining handshake traffic keys but before completing the handshake, a peer could send application-data records. The record layers appended the decrypted bytes to an unbounded plaintext buffer instead of rejecting the record. Because each record was nonempty, it also reset the useless-record counter.

2. Zero-length handshake records produced no handshake bytes but were not rejected or charged against the useless-record limit. The handshake reader could therefore loop forever while continuing to receive non-progressing records.

Both defects were introduced with Reseau's native TLS stack in version 1.1.0. Releases 1.0.x used the previous OpenSSL TLS engine and are not affected.

Patches

Upgrade to Reseau 1.3.2 or later. The fixed record layers reject both application data received before handshake completion and empty handshake records with a fatal unexpected_message alert.

Workarounds

There is no complete in-process workaround. Set a finite handshake_timeout_ns and enforce connection rate and concurrency limits to bound each handshake's duration and reduce repeated attacks; an attacker can still consume substantial resources within that window. Terminating TLS in a patched proxy before forwarding traffic to a vulnerable Reseau service avoids the vulnerable record parser.

Reported to the JuliaLang security team through Anthropic's Coordinated Vulnerability Disclosure program.