Safeguard
Cryptography

How ML-KEM (Kyber) works and implementation pitfalls

FIPS 203's ML-KEM Kyber is landing in TLS, SSH, and VPNs everywhere — here's how the lattice math works and the timing bugs, like KyberSlash, already found in real implementations.

James
Principal Security Architect
8 min read

Kyber is not a niche algorithm anymore — as ML-KEM, it is the default post-quantum key exchange baked into TLS 1.3 in Chrome, Firefox, OpenSSH, and AWS's s2n-tls. NIST finalized it as FIPS 203 in August 2024, three years after selecting the underlying CRYSTALS-Kyber design in July 2022. For engineering teams shipping software today, ML-KEM Kyber is quickly becoming a mandatory checkbox for any product that touches TLS, VPNs, or firmware updates and needs to survive a future quantum-capable adversary "harvest now, decrypt later" attack. But the same properties that make lattice-based cryptography resistant to quantum computers — large keys, modular arithmetic, polynomial sampling — also create a wider attack surface for implementers. This post explains how Kyber key encapsulation actually works mathematically, then walks through the real bugs that have already shown up in production ML-KEM code.

What Is ML-KEM Kyber and Why Did NIST Standardize It?

ML-KEM Kyber is NIST's standardized module-lattice-based key encapsulation mechanism, published as FIPS 203 on August 13, 2024, and it exists because Shor's algorithm breaks RSA and elliptic-curve Diffie-Hellman on a sufficiently large quantum computer. NIST opened its post-quantum cryptography competition in December 2016, received 82 initial submissions, and after three elimination rounds announced in July 2022 that CRYSTALS-Kyber would become the primary standard for general-purpose key encapsulation, alongside CRYSTALS-Dilithium (now ML-DSA) for signatures. The rename to ML-KEM ("Module-Lattice-based Key-Encapsulation Mechanism") happened during standardization to reflect NIST's naming convention, but it is the same algorithm with minor encoding tweaks — notably a change to how the shared secret is derived, which means Kyber round-3 and final ML-KEM are not wire-compatible. FIPS 203 defines three parameter sets: ML-KEM-512, ML-KEM-768, and ML-KEM-1024, mapped to NIST security categories 1, 3, and 5 respectively, with ML-KEM-768 (roughly equivalent to AES-192 strength) the recommended default for most TLS deployments, including Chrome's X25519Kyber768 hybrid rollout that began in 2023.

How Does Kyber Key Encapsulation Actually Work?

Kyber key encapsulation works by having one party generate a lattice-based public key, the other party use it to encrypt a random shared secret, and both sides derive the same session key without ever transmitting it directly. Concretely, key generation samples a public matrix A of dimension k×k over the ring Z_q[x]/(x^256+1), where q = 3329 and k is 2, 3, or 4 for ML-KEM-512, 768, and 1024. The private key is a set of small polynomial vectors sampled from a centered binomial distribution, and the public key is A·s + e, where e is another small error vector — this is the module learning-with-errors (M-LWE) problem, and its hardness is what makes recovering the private key computationally infeasible. To encapsulate, the sender samples fresh randomness, computes a ciphertext as a noisy inner product against the public key, compresses it to reduce bandwidth, and derives the actual 32-byte shared secret through a hash function rather than using the raw lattice output directly — this last step is the Fujisaki-Okamoto transform, and it's what upgrades Kyber from a passively secure scheme (CPA) to one resistant to chosen-ciphertext attacks (CCA2), which is required for any real-world protocol. For ML-KEM-768, the public key is 1,184 bytes, the ciphertext is 1,088 bytes, and the resulting shared secret is always exactly 32 bytes regardless of parameter set.

What Makes Lattice-Based Cryptography Resistant to Quantum Attacks?

Lattice-based cryptography resists quantum attacks because the best known quantum algorithms don't get a meaningful speedup against the underlying lattice problems the way Shor's algorithm demolishes factoring and discrete log. Learning-with-errors and its module variant reduce, in the worst case, to approximating the shortest vector in a high-dimensional lattice — a problem for which even Grover's algorithm only offers a quadratic speedup, not the exponential break Shor's algorithm provides against RSA-2048 or P-256. Kyber operates in a 256-dimensional polynomial ring per module component, and ML-KEM-768 uses k=3, giving an effective lattice dimension of 768 — large enough that both classical lattice-reduction techniques (BKZ) and quantum-accelerated variants remain computationally infeasible at current and foreseeable qubit counts. This is also why Kyber ciphertexts and keys are noticeably larger than elliptic-curve equivalents: a Kyber-768 public key at 1,184 bytes is roughly 37 times larger than a 32-byte X25519 public key, and that size difference is a direct consequence of needing enough lattice dimension and modulus to make the M-LWE problem hard.

Why Do Post-Quantum Key Exchange Implementations Leak Secrets Through Timing?

Post-quantum key exchange implementations leak secrets through timing because compression, decompression, and modular reduction steps in Kyber are easy to implement with data-dependent branches or variable-time division, and both attackers and auditors have already found this in shipped code. The clearest example is KyberSlash, disclosed in December 2023 and January 2024 by researcher Antoon Purnal and collaborators: several widely used implementations, including versions of liboqs, PQClean-derived code, and Kyber code embedded in libraries like Botan and parts of the Linux kernel's WireGuard-adjacent PQC work, performed a division by the modulus q using the CPU's native division instruction instead of constant-time arithmetic. Because integer division time on most CPUs varies with operand value, an attacker who can measure decapsulation latency — even at network distance, given enough samples — could recover bits of the secret key. The fix, adopted across affected projects within weeks, replaces division with constant-time approximations using fixed-point multiplication and shifts. This wasn't a flaw in the ML-KEM specification; FIPS 203 doesn't mandate an implementation strategy, so the vulnerability lived entirely in how individual codebases translated the math into machine instructions, which is exactly the category of bug that specification conformance testing alone will never catch.

What Are the Most Common ML-KEM Implementation Pitfalls?

The most common ML-KEM implementation pitfalls cluster around randomness, side channels, and version confusion rather than the core lattice math itself. First, weak or reused randomness in key generation and encapsulation is catastrophic — because Kyber's security depends on fresh, unpredictable noise vectors, a broken or predictable RNG (the kind seen in embedded devices without a good entropy source) can make the M-LWE instance trivially solvable, similar to historical ECDSA nonce-reuse disasters. Second, non-constant-time NTT (Number Theoretic Transform) operations, polynomial comparison during the Fujisaki-Omoto re-encryption check, or the KyberSlash-style division bug all create timing or power side channels that have been demonstrated in academic and real-world attacks against smart cards and microcontrollers running Kyber. Third, decryption failures: Kyber has a small but non-zero probability that a valid ciphertext fails to decrypt correctly — about 2^-164 for ML-KEM-768 — and implementations that don't handle this per spec, or that leak whether a failure occurred, open a decryption-failure oracle that can leak key material over repeated queries. Fourth, and increasingly relevant in 2025 and 2026 as migrations accelerate, is protocol-level version confusion: round-3 Kyber, the NIST draft, and final FIPS 203 ML-KEM produce different shared secrets from the same keys due to encoding changes, and mixing libraries or hybrid-mode implementations that assume the wrong version silently produces interoperability failures or, worse, downgrade paths an attacker can exploit. Finally, because ML-KEM is almost always deployed in hybrid mode alongside a classical algorithm like X25519, bugs in how the two shared secrets are concatenated and fed into the KDF — including a mismatched or missing domain separator — can quietly weaken the hybrid construction back down to the classical algorithm's security level.

How Safeguard Helps

Safeguard treats post-quantum readiness as a supply chain problem, not just a cryptography checklist. Our software composition analysis maps every dependency in your build — down to vendored Kyber implementations pulled in transitively through TLS libraries — and flags versions known to carry timing issues like KyberSlash or that implement pre-final Kyber round-3 encoding instead of FIPS 203 ML-KEM, so you're not shipping a shared secret your peer can't reproduce. For teams building or embedding their own PQC code, Safeguard's SAST rules detect the specific anti-patterns behind these pitfalls: native division and modulo operations on secret-dependent values, missing constant-time comparison in decapsulation paths, and RNG calls that bypass a vetted CSPRNG. Because ML-KEM is landing across dozens of dependencies simultaneously — TLS stacks, SSH clients, VPN daemons, HSM firmware — Safeguard's SBOM-driven monitoring gives security teams a single view of exactly where lattice-based cryptography is deployed across their software supply chain, what version and parameter set is in use, and whether a newly disclosed advisory like KyberSlash actually reaches a production artifact. As post-quantum migration mandates from NIST, CISA, and enterprise customers move from "should" to "must" over the next few years, that visibility is what turns a cryptographic transition from a scramble into a tracked, auditable rollout.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.