PCIe 6.0 Explained: FLIT Mode, PAM4, FEC and IDE Link Encryption

PCIe 6.0 doubles the per lane rate to 64 GT/s by switching from two level NRZ signalling to four level PAM4, which carries two bits per symbol but makes raw bit errors far more common. To absorb those errors it moves all traffic into fixed 256 byte FLITs protected by a lightweight forward error correction code and a strong 8 byte CRC, with replay as the backstop, at a design cost of a few nanoseconds. Separately, Integrity and Data Encryption (IDE) adds AES-GCM encryption and integrity to the packets on the link, with keys set up over a DMTF SPDM session, and TDISP lets a confidential VM accept a device into its trust boundary.

This post goes one level below our bus evolution history, which covers ISA to PCIe 7.0 and CXL in one sweep. Here we stay on what PCIe 6.0 changed and how to check a real link.

Key takeaways

• PAM4 doubles the bits, not the frequency. PCIe 6.0 sends 2 bits per unit interval on four voltage levels, so 64 GT/s keeps the same Nyquist frequency as 32 GT/s NRZ. The price is three smaller eyes and a first bit error rate target of 1E-6, far more raw errors than earlier generations were built around.
• A FLIT is 256 bytes: 236 bytes of transaction layer packets, 6 bytes of data link payload, 8 bytes of CRC and 6 bytes of FEC, according to a PCI-SIG board director's presentation. FEC needs a fixed block, so 128b/130b framing went away.
• Light FEC plus strong CRC plus replay. The FEC corrects a burst within 3 bytes, the CRC detects what FEC misses, and the link replays the bad FLIT. Correction and checking take under 2 ns, against roughly 100 ns for networking style FEC.
• L0p lets a link drop and restore lanes without stalling traffic.
• IDE encrypts and authenticates TLPs with AES-GCM (256 bit key, 96 bit MAC); keys are programmed with IDE_KM inside an SPDM session.
• TDISP moves a device interface through CONFIG_UNLOCKED, CONFIG_LOCKED and RUN so a confidential VM can verify it first.
• PCIe 7.0 (released to members in June 2025) runs 128 GT/s with PAM4 and FLITs; PCIe 8.0 targets 256 GT/s, with release to members planned by 2028.
• IDE does not replace the IOMMU. It stops interposers on the wire. A device that is malicious or compromised can still issue DMA, so IOMMU policy, Kernel DMA Protection and device attestation still matter.

A dated timeline of PCIe 6.0 and its security features

A note on sourcing: PCI-SIG's website refused our automated requests and the base specification is members only. PCI-SIG facts here come from its press releases as reprinted by trade publications and from a PCI-SIG board director's presentations; IDE registers come from the Linux kernel's headers and IDE_KM and TDISP messages from DMTF's reference implementation.

PCIe 5.0, 6.0, 7.0 and 8.0 compared

PCI-SIG quotes bandwidth with both directions added together. For a storage read or a GPU upload, one direction is what you feel, so halve the headline number before comparing it with a drive or a network port.

Why did PCIe 6.0 move from NRZ to PAM4, and what does that do to bit error rates?

The frequency wall

Through PCIe 5.0, every symbol on a lane was one bit, high or low, called non return to zero (NRZ). Doubling the data rate meant doubling the frequency the channel had to carry, and at 32 GT/s boards already needed retimers and heavy equalisation.

PCIe 6.0 did not double the frequency again. It changed what a symbol means. Pulse amplitude modulation with four levels (PAM4) puts two bits in each unit interval. The PCI-SIG board director's slides list the four transmit levels as roughly minus Vtx, minus a third of Vtx, plus a third of Vtx and plus Vtx, mapped to the bit pairs 00, 01, 11 and 10. Because each symbol carries two bits, 64 GT/s runs at the same symbol rate, and so the same Nyquist frequency, as 32 GT/s NRZ. The slides state it plainly: PAM4 "helps channel loss (same Nyquist as 32.0 GT/s)".

What PAM4 costs

Fitting four levels into the same voltage swing means the receiver now has to tell apart three stacked "eyes" instead of one. Each eye is about a third of the height of an NRZ eye in the ideal case. As a worked number, a third of the amplitude is 20 times log10 of 3, or about 9.5 dB less vertical margin before you count any other effect. The eyes are also narrower in time. Noise that an NRZ receiver would have shrugged off now flips symbols.

The presentations describe three ways the errors behave:

• First bit error rate (FBER): the probability that a bit is received in error in the first place. PCIe 6.0 is designed around an FBER of 1E-6, or one error per million bits.
• Bursts within a lane: a decision feedback equaliser (DFE) uses past decisions to cancel interference, so one wrong decision can drag the next few bits with it.
• Correlation across lanes: a common cause such as power supply noise can hit several lanes of a x16 link at once.

One error per million bits sounds rare until you do the arithmetic. A x16 link at 64 GT/s moves 1,024 gigabits per second in each direction. At an FBER of 1E-6 that is around a million first errors per second per direction, before any burst effects. No transaction protocol can expose that to software, which is why PCIe 6.0 needed new error handling.

Gray coding and precoding

Gray coding orders the bit pairs so adjacent levels differ by one bit (00, 01, 11, 10), so the most likely mistake, landing in the neighbouring eye, flips only one bit. Precoding transforms the bit stream so that a DFE error burst produces fewer wrong bits than it otherwise would. Both appear on the slides as mitigations alongside "FEC + Replay on CRC error".

Why not just use Ethernet style FEC?

High speed Ethernet also uses PAM4 and pays with strong FEC that adds on the order of 100 ns. PCIe carries CPU load and store traffic, and the requirements slide says it bluntly: "We can not afford the 100ns FEC latency as networking does with PAM4". The target was under 10 ns added for transmitter plus receiver over 32 GT/s, including FEC, under 2 percent bandwidth inefficiency compared with PCIe 5.0, and a failure in time (FIT, failures per billion hours) well below 1 for a x16 link. That is only reachable if the FEC is weak and fast and something else, CRC plus replay over fixed size blocks, catches what it misses.

What is FLIT mode, how big is a FLIT, and why did 128b/130b encoding go away?

The FLIT layout

A FLIT (flow control unit) is a fixed size block. In PCIe 6.0 it is 256 bytes, laid out like this according to the OpenFabrics 2022 presentation:

What disappeared

In the 128b/130b world of PCIe 3.0 to 5.0, every 128 bit block on a lane carried a 2 bit sync header, each TLP had a framing token at the front, a 4 byte link CRC (LCRC) at the back and a sequence number, and acknowledgements and credits travelled in separate data link layer packets (DLLPs). The slides list what FLIT mode removes: "No Sync hdr, no Framing Token (TLP reformat), no TLP/DLLP CRC". The TLP header format was also updated for FLIT mode; PCI-SIG's release describes an "updated packet layout used in Flit Mode to provide additional functionality and simplify processing".

The reason is structural. Forward error correction protects a known number of bytes. With variable length TLPs and per packet CRCs, there is no fixed block to compute parity over. The slides put it as "FEC needs fixed set of bytes". So the protocol moved to fixed blocks, and once you have fixed blocks, per packet framing and per packet CRCs are redundant overhead. The 2 bit sync header that 128b/130b used to mark block boundaries is no longer needed either.

Once in FLIT mode, always in FLIT mode

FLIT mode is negotiated during link training, and the slides note that "once in Flit mode, we are always in Flit mode at all speeds (2.5 GT/s through 64.0 GT/s)". If a PCIe 6.0 device trains down to 16 GT/s because of a marginal slot, it still uses FLITs. A PCIe 6.0 device in a PCIe 5.0 slot falls back to 128b/130b, which keeps backward compatibility.

The Linux kernel's register header shows the plumbing: the PCI Express capabilities register has a "Flit Mode Supported" bit, and Link Status 2 has a "Flit Mode Status" bit (0x0400 at offset 0x32 of the PCIe capability). We use that in the walkthrough below.

Worked example: bandwidth

Take a x16 link at 64 GT/s.

1. Raw bits per direction: 64 times 10 to the 9th transfers per second times 16 lanes, which is 1,024 Gb/s, or 128 GB/s.
2. Each FLIT is 256 bytes, so the link moves 128 GB/s divided by 256, or 500 million FLITs per second. That matches the "Flits per sec 500000000" row in the board director's retry table.
3. TLP bytes per FLIT are 236, so the ceiling for transaction layer bytes is 500 million times 236, or 118 GB/s per direction. Counting the 6 DLP bytes as useful link traffic gives 242 of 256 bytes, or about 121 GB/s.

On paper, 236 of 256 bytes (92.2 percent) looks worse than 128b/130b's 98.5 percent, but that figure ignores the framing, sequence number, LCRC and DLLPs every small packet paid for. The slides claim "better B/W efficiency for small packets (important for networking) in Flit Mode due to CRC amortization and removal of PHY overheads", and that PCIe 6.0 achieves more than twice PCIe 5.0's bandwidth "in most cases", beating the under 2 percent inefficiency target.

Worked example: how long a FLIT takes

A FLIT is 2,048 bits. On a x16 link, each lane carries 128 of them. At 64 GT/s that is 128 divided by 64 billion, or 2 ns per FLIT. On a x1 link the same FLIT takes 32 ns. The slides list exactly that series: "2ns x16, 4ns x8, 8ns x4, 16ns x2, 32ns x1". The receiver must collect the whole FLIT before it can run FEC and check the CRC.

How do the lightweight FEC and CRC work together, and what is the latency cost?

Three layers of defence

The PCIe 6.0 approach, in the board director's words, is a "light-weight FEC with strong CRC and low-latency replay at link level". Each layer has a job.

Interleaving is the trick: bytes are spread across three independent codes, so a 3 byte burst lands one byte in each and each code fixes a single symbol, which covers typical DFE bursts cheaply.

Worked example: how often does the link replay?

The IEEE presentation includes a table of retry probability against FBER for a x16 link with lane correlation. At the design point of FBER 1E-6:

• The probability that a FLIT needs a replay is about 5 times 10 to the minus 6.
• With 500 million FLITs per second on x16, that is about 500 million times 5E-6, or roughly 2,500 replays per second per direction.
• If each replay costs on the order of 100 ns of link time, the total is about 250 microseconds per second, or 0.025 percent. The slides, using a more conservative go back N model, give a bandwidth loss of 0.05 percent.
• The FIT for undetected corruption at that error rate is about 4.6 times 10 to the minus 10, which the slides call "almost 0".

The point of the table is the trade the designers made: "We are better off retrying a packet with 10-6 (or 10-5) probability with a retry latency of 100ns" than paying 100 ns of FEC latency on every packet.

Worked example: latency per packet

The OpenFabrics slides compare the time to accumulate a TLP in 128b/130b at 32 GT/s with FLIT mode at 64 GT/s.

Small packets on narrow links pay a little more latency in FLIT mode, because the receiver waits for a whole FLIT. Larger packets get faster, because the rate doubled and per packet overhead went away. The presentation's evaluation slide marks the under 10 ns latency goal as exceeded.

What is L0p, and how does it save power?

The problem with earlier power states

Before PCIe 6.0, a link that wanted to save power had four tools, and each had a drawback the slides describe:

• L0s turned off one direction's transmitter for short idle periods. Its savings were small, and it is not supported once FLIT mode is negotiated.
• L1 saves a lot of power but has long entry and exit times, so it suits idle devices, not busy ones with quiet moments.
• Dynamic link width could change the number of active lanes, but a width increase sent the entire link through recovery and configuration. Traffic stalled for tens of microseconds while the link retrained.
• Speed change saves less power than width change and takes milliseconds.

How L0p works

L0p is a new state in which the link keeps passing traffic on a reduced number of lanes while the others are powered down. It is negotiated together with FLIT mode during configuration. Either side can request a width change. According to the slides:

• The link partner acknowledges or rejects the request within 1 microsecond. With no answer in 2 microseconds, the requester abandons or retries.
• If both sides ask at once, the request for the higher width wins unless a priority flag says the lower width should.
• To shrink, the ports send a control skip ordered set on all active lanes, put the lanes being turned off into electrical idle, and carry on with data on the remaining lanes.
• To grow, the lanes being woken run their training sequences while data keeps flowing on the active lanes, then the link switches to the new width.

Power can follow bandwidth without stalling traffic: a mostly idle x16 accelerator link can sit at a narrow width between bursts. The board director's evaluation slide called L0p's power results design dependent, so savings depend on the silicon.

What is Integrity and Data Encryption (IDE), and which traffic does it protect?

The threat IDE addresses

Without IDE, every transaction on a PCIe link is plaintext. Anyone who can put hardware on the link, such as an analyser, an interposer card or a malicious switch, can read memory traffic and in principle change it. That matters in shared data centres and in confidential computing, where the cloud operator sits outside the trust boundary.

IDE adds authenticated encryption to transaction layer packets. The Linux kernel's register definitions for the IDE extended capability (capability ID 0x30) list the only algorithm currently defined: "AES-GCM 256 key size, 96b MAC". In other words, a 256 bit AES key in Galois/Counter Mode, with a 96 bit authentication tag. Engineering explainers describe the payload as encrypted and the whole TLP as integrity protected; we could not check that wording against the members only specification.

Link IDE versus Selective IDE

The same capability register advertises two kinds of IDE stream:

Other capability bits show the options a device can support: flow through IDE (a switch port passing streams it does not terminate), partial header encryption (hiding some address bits and byte enables), aggregation (one MAC over several TLPs to save bandwidth), PCRC (a plaintext CRC check over the payload), Selective IDE for configuration requests, and TEE limited streams that only carry traffic for a trusted execution environment. Stream state is either Insecure or Secure, and a stream can report that an IDE failure message was received.

IDE failures surface through ordinary error reporting. The kernel header defines an "IDE Check Failed" bit in the AER uncorrectable error status register, so a MAC mismatch shows up the same way a malformed TLP would.

What IDE does not protect

IDE protects TLPs on the wire only. It does not make a device trustworthy, protect data after decryption at either end, replace the IOMMU, or say which firmware the device runs; that last job belongs to SPDM.

How are IDE keys set up with SPDM and IDE_KM?

SPDM: identity, measurements and a secure channel

The DMTF Security Protocol and Data Model (SPDM, DSP0274) is a request and response protocol between a host and a device. DMTF's reference implementation, libspdm, lists the messages each version added:

libspdm tracks DSP0274 versions 1.0.3, 1.1.5, 1.2.4, 1.3.3 and 1.4.1. The session messages themselves are wrapped as described in the companion specification, Secured Messages using SPDM (DSP0277), which libspdm tracks at 1.0.1 through 1.3.0.

On PCIe, SPDM messages travel through a Data Object Exchange (DOE) mailbox, an extended capability (ID 0x2E) that lets software send structured messages through configuration space. The PCIe binding of SPDM is called Component Measurement and Authentication (CMA). A Linux Plumbers 2024 slide summarises the stack in one line each: DOE is a "PCIe config space blob", SPDM gives "secure config space access", CMA means "authenticated device", and IDE means "encrypted PCIe link".

IDE_KM: programming the keys

Once the host and device share an SPDM secured session, the host uses the IDE Key Management protocol (IDE_KM) inside that session to program stream keys. The message set in libspdm's header for the PCIe definitions is short:

Each key is programmed per stream, per key set (K0 or K1), per direction (receive or transmit) and per sub stream: posted requests (memory writes), non posted requests (reads and other requests that expect an answer) and completions. That is up to 12 keys per stream. Two key sets let the host program fresh keys into the idle set and switch with K_SET_GO without tearing down the stream, which is how keys are rotated in service.

The full sequence

Put together, bringing up a protected link looks like this:

1. The host finds the DOE capability and starts SPDM: GET_VERSION, GET_CAPABILITIES, NEGOTIATE_ALGORITHMS.
2. It reads the device certificate chain with GET_DIGESTS and GET_CERTIFICATE and validates it against a trusted root from the device maker.
3. It asks for GET_MEASUREMENTS and compares the firmware and configuration measurements against expected values.
4. It runs KEY_EXCHANGE and FINISH to create a secured session.
5. Inside the session it sends IDE_KM QUERY, then KEY_PROG for each sub stream and direction, on both the device and its partner port (the root port, which the host programs directly or through platform firmware).
6. It sends K_SET_GO, enables the stream in the IDE control registers, and checks that the stream state reads Secure.

The Linux PCI TSM documentation shows the kernel side of steps 5 and 6: allocate Selective IDE register blocks on endpoint and root port, fill the Requester ID and address association registers, record the stream in sysfs and program both ports.

What is TDISP, and how does it let a confidential VM trust a device?

Why confidential VMs need it

Confidential computing encrypts a VM's memory so even the hypervisor cannot read it, which breaks ordinary passthrough: an assigned device cannot DMA into private memory, so guests fall back to bounce buffers in shared memory.

The TEE Device Interface Security Protocol (TDISP) lets the VM bring a specific device function into its trust boundary after checking it. The unit of assignment is a TEE Device Interface (TDI), often an SR-IOV virtual function. The device side is managed by a Device Security Manager (DSM) inside the device; the host side by a TEE Security Manager (TSM), typically platform firmware or a security processor.

The TDISP state machine

DMTF's libspdm header for TDISP lists four interface states and the requests that move between them.

In between locking and running, the VM requests GET_DEVICE_INTERFACE_REPORT. The report lists the interface's MMIO ranges with attributes (which range holds the MSI-X table, which is non TEE memory) and flags such as whether it does DMA with or without PASID, and whether it supports address translation services. Combined with SPDM measurements and an IDE stream from the device to the root port, the VM can decide whether this is the device it expected, running firmware it accepts, over a protected link.

The Linux PCI TSM API mirrors this with a TDI context bound to a KVM guest. A Linux Plumbers 2024 session described the host verbs as DEV_CONNECT (SPDM, IDE and measurements for the device) and TDI_BIND (lock and run for the assigned function), with the guest validating before secure DMA is enabled.

Where it runs

TDISP needs every part of the chain to cooperate: a CPU with confidential VM support and a security manager, a root port with IDE, a device with DOE, SPDM, IDE and TDISP support, firmware on both sides, a host kernel and hypervisor that drive the flow, and a guest kernel that knows how to validate. Expect it on recent servers, accelerators and NICs first.

What is new in PCIe 7.0 and the 8.0 work announced by PCI-SIG?

PCIe 7.0

PCI-SIG released PCIe 7.0 to its members in June 2025. Its release lists 128.0 GT/s per lane, up to 512 GB/s bidirectional on x16, PAM4 and FLIT encoding, improved power efficiency (no figure given) and backward compatibility, aimed at AI, 800G Ethernet, cloud and quantum computing. In practice, 7.0 is a doubling of the 6.0 architecture rather than a new one: the FLIT, FEC, CRC and replay model, L0p and the IDE and TDISP security features carry over. The new difficulty is electrical: 128 GT/s PAM4 means 64 billion symbols per second per lane across boards and connectors.

PCIe 8.0

On 5 August 2025, PCI-SIG announced that PCIe 8.0 will double the rate again to 256.0 GT/s, with release to members planned by 2028. Its stated objectives are 256.0 GT/s raw and up to 1 TB/s bidirectional on x16, reviewing new connector technology, confirming latency and FEC targets, meeting reliability targets, keeping backward compatibility, protocol enhancements to improve bandwidth, and power reduction.

"Reviewing new connector technology" suggests today's card edge connector may not carry 256 GT/s, and the FEC wording suggests error correction may need to change. The release does not name the modulation, so treat claims about 8.0 signalling as speculation.

A cadence, not a product date

PCI-SIG describes its rhythm as doubling bandwidth every three years. Specification dates are not shipping dates; 6.0 hardware followed its 2022 specification by roughly three years, servers first.

How do you check link speed, width and AER errors on Linux and Windows?

These steps only read state. Run them on machines you administer. setpci can also write to configuration space, which can hang or damage a running system: use only the read form shown here, with no equals sign.

1. Find the device address (Linux)

lspci -tv            # tree of root ports, switches and endpoints
lspci -nnk           # [vendor:device] IDs and the kernel driver in use

The lspci man page explains that -s takes a domain, bus, device and function, -nn shows numeric IDs with names, and -vv and -vvv add progressively more decoded detail. Note the address of the device you care about, for example 0000:41:00.0.

2. Compare capability with what the link negotiated

DEV=0000:41:00.0
sudo lspci -vvv -s "$DEV" | grep -E "LnkCap:|LnkSta:|LnkCap2:|LnkSta2:"
for a in max_link_speed max_link_width current_link_speed current_link_width; do
  printf "%-20s %s\n" "$a" "$(cat /sys/bus/pci/devices/$DEV/$a)"
done

LnkCap is what the port can do; LnkSta is what it trained to. The sysfs attributes come from the kernel's pci-sysfs.c and are readable without root. A 64 GT/s capable device reporting 32 GT/s, or x8 where you expected x16, points at the slot, a riser, a retimer, firmware settings or a marginal channel.

3. Check whether the link is in FLIT mode

Newer pciutils releases decode FLIT mode in the lspci -vvv output. If yours does not, read Link Status 2 directly and test bit 10 (mask 0x0400), which the kernel header names "Flit Mode Status":

sudo setpci -s 41:00.0 CAP_EXP+0x32.w     # read only: prints Link Status 2 in hex
# 0x0400 set in the result means the link is operating in Flit Mode

A PCIe 6.0 device in a PCIe 5.0 slot will show the bit clear: it has fallen back to 128b/130b.

4. Look for IDE and DOE capabilities

sudo lspci -vvv -s 41:00.0 | grep -iE "Integrity|Data Object Exchange|Extended Capability"

Depending on your pciutils version, IDE (extended capability ID 0x30) and DOE (ID 0x2E) are either decoded by name or listed by number. Presence means the device can do IDE and SPDM over DOE; it does not mean a stream is active. Where the kernel has set up a Selective IDE stream through the PCI TSM subsystem, it is recorded in sysfs under the device.

5. Read AER error counters

cd /sys/bus/pci/devices/$DEV
cat aer_dev_correctable aer_dev_nonfatal aer_dev_fatal 2>/dev/null
# On a root port, the totals reported to it:
cat aer_rootport_total_err_cor aer_rootport_total_err_nonfatal aer_rootport_total_err_fatal 2>/dev/null
sudo journalctl -k | grep -i "PCIe Bus Error"

The kernel ABI documentation describes these counters, added in kernel 4.19 (July 2018), and warns that errors caused by an endpoint may be counted at its link partner, so check the root port too. Rising correctable "Receiver Error" or "Bad TLP" counts suggest a marginal channel that FEC and replay are hiding. The AER guide also notes that Linux only handles AER when firmware grants it control through ACPI _OSC; on some servers, the firmware owns AER and logs to the baseboard management controller instead. Messages are rate limited by default, 10 events per 5 seconds.

6. Confirm IOMMU state (Linux)

ls /sys/kernel/iommu_groups/ | wc -l          # 0 means no active IOMMU
cat /proc/cmdline                              # look for iommu.passthrough=1 or iommu.strict=0
sudo dmesg | grep -iE "DMAR|AMD-Vi|IOMMU" | head

The kernel parameter docs explain the trade offs: iommu.passthrough=1 bypasses the IOMMU for DMA, and iommu.strict=0 defers IOTLB invalidation "for increased throughput at the cost of reduced device isolation".

7. Windows: link state, DMA protection and hardware errors

# Find the device and list its properties; look for link speed and width entries
Get-PnpDevice -Class Net, Display, SCSIAdapter | Format-Table FriendlyName, InstanceId
Get-PnpDeviceProperty -InstanceId '<InstanceId>' | Where-Object KeyName -like '*Link*'

# Corrected and uncorrected hardware errors, including PCIe, are logged by WHEA
Get-WinEvent -FilterHashtable @{LogName='System'; ProviderName='Microsoft-Windows-WHEA-Logger'} -MaxEvents 50

Microsoft documents Get-PnpDeviceProperty as returning key value properties with friendly DEVPKEY_ names; which PCI link properties appear depends on your build and driver. The Windows Hardware Error Architecture treats a PCI Express root port error interrupt as a hardware error source and separates corrected from uncorrected errors. For DMA protection, Microsoft's Kernel DMA Protection page says to open Windows Security, Device security, Core isolation details, Memory access protection, or check the Kernel DMA Protection line in msinfo32.

What does IDE mean for DMA attacks and IOMMU policy?

Separate the threats

"DMA attack" covers several different attackers, and IDE only addresses one of them.

Policy implications

Keep the IOMMU on, and strict where you can. IDE makes the wire trustworthy; it says nothing about what the device at the end is allowed to do. On Linux, avoid iommu.passthrough=1 on hosts that take untrusted devices, and weigh iommu.strict=0 carefully: lazy invalidation leaves a small window in which a device can still use a mapping the driver has freed. Microsoft's guidance is that Windows uses the IOMMU to block external peripherals from DMA unless their drivers support DMA remapping, and blocks incompatible drivers until a user signs in.

Pair IDE with attestation. SPDM certificate and measurement checks are what tie IDE keys to a known device running known firmware.

Lock firmware during trusted use. TDISP's NO_FW_UPDATE flag exists because firmware is part of the trust decision; update in the unlocked state, then re-attest.

Plan for stream limits. Selective IDE streams are finite per host bridge (256 by default in Linux) and per device.

Watch the error counters. IDE check failures arrive as AER uncorrectable errors; a sudden spike deserves a look as possible tampering.

Map it to firmware resiliency. NIST SP 800-193 frames firmware security as protection, detection and recovery. SPDM measurements support detection; your recovery plan should cover re-attesting and re-keying devices after a firmware change.

Where Safeguard fits, honestly

Most of this post is about silicon, link training and device firmware, and Safeguard does not work at that level. It does not configure IDE, verify SPDM certificates, inspect FLIT traffic, scan device firmware, generate a hardware bill of materials or manage endpoints. What it does is analyse software. The systems that bring up IDE and TDISP run a great deal of open source software: host kernels and hypervisors, user space management agents, attestation services, SPDM libraries, and the tools your team builds to collect link and AER data. Safeguard scans repositories, container images and SBOM files for known vulnerable components (safeguard scan --dir ., --image <name>, --sbom <file>), shows which projects contain a given component and version, and applies policies and gates in the platform. If a device vendor supplies an SBOM for its driver or management stack, you can scan that file too. Firmware fixes still come from the device maker.

Common mistakes

• Reading GT/s as GB/s. 64 GT/s is per lane and raw. A x16 PCIe 6.0 link carries 128 GB/s raw each way; PCI-SIG's 256 GB/s adds both directions.
• Assuming a PCIe 6.0 device runs at 6.0 speed. It runs at the highest rate both ends and the channel support. Check LnkSta on both ends, not the product sheet.
• Treating correctable errors as harmless. FEC and replay hide them, but a rising count is lost margin, and one day it becomes uncorrectable or a link that trains down.
• Thinking IDE stops DMA attacks. It protects the wire. A hostile or compromised device still needs an IOMMU and attestation to contain it.
• Encrypting before authenticating. Programming IDE keys into a device whose certificate and measurements you have not checked gives you an encrypted link to an unknown party.
• Turning the IOMMU off for performance. iommu.passthrough=1 and lazy invalidation trade isolation for throughput. Measure first, and keep isolation on hosts that take untrusted devices.
• Equating capability with activation. Only an IDE stream in the Secure state means the link is protected.
• Forgetting firmware in inventory. TDISP trust decisions depend on device firmware. Record device firmware versions in your asset inventory.

FAQ

What is the difference between PCIe 5.0 and PCIe 6.0?

PCIe 6.0 doubles the raw rate from 32 to 64 GT/s per lane by using PAM4 instead of NRZ signalling. It adds FLIT mode with 256 byte FLITs, a lightweight FEC and 8 byte CRC, and the L0p power state. It stays backward compatible with earlier generations.

How big is a FLIT in PCIe 6.0?

256 bytes: 236 bytes of TLP data, 6 bytes of data link payload, 8 bytes of CRC and 6 bytes of FEC, according to a PCI-SIG board director's presentation.

Does PCIe 6.0 add latency because of FEC?

Very little. The FEC correction and CRC check take under 2 ns, and the design target was under 10 ns added across transmitter and receiver. Small packets on narrow links can wait slightly longer to fill a FLIT; large packets get faster because the rate doubled.

Why does PCIe 6.0 use PAM4?

Doubling the NRZ symbol rate again would have doubled the channel frequency and the losses. PAM4 sends two bits per symbol, so 64 GT/s keeps the same Nyquist frequency as 32 GT/s NRZ, at the cost of more raw bit errors that FEC and replay handle.

What is PCIe IDE encryption?

Integrity and Data Encryption is an optional PCIe feature that encrypts and authenticates transaction layer packets with AES-GCM using a 256 bit key and a 96 bit MAC, either on a single link or end to end through switches.

How are PCIe IDE keys exchanged?

The host first authenticates the device and opens a secured session with DMTF SPDM over the DOE mailbox. Inside that session it uses IDE_KM messages (QUERY, KEY_PROG, K_SET_GO, K_SET_STOP) to load keys for each stream, direction and sub stream into both ports.

What is TDISP in PCIe?

The TEE Device Interface Security Protocol. It lets a confidential VM lock a device interface's configuration, read a report about it, verify it and move it to the RUN state, so that the device can do trusted DMA into the VM's private memory.

When is PCIe 8.0 coming out?

PCI-SIG announced on 5 August 2025 that PCIe 8.0 targets 256 GT/s and is planned for release to members by 2028. Products would come later.

Fix it without waiting for upstream

PCIe 6.0 security lives in silicon and firmware, but the hosts that bring up IDE, attest devices and assign them to confidential VMs run kernels, hypervisors, SPDM libraries, management agents and monitoring tools built from open source components. When those components carry known vulnerabilities, the same inventory and patch discipline applies as for any other software, and that is the part your team can act on today.

Whatever the vulnerability, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.

1. Autonomous remediation: pull requests, not tickets

When a fixed version exists, Safeguard's Griffin models open a tested, policy gated pull request that moves the affected dependency to the safe version. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation covers eight categories, including dependencies, licenses, security posture and findings. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Autonomous remediation ships switched off, with every category set to Manual, so your team decides how much runs on its own and what still needs a human approval.

2. Gold Open Source: the safe version in seconds, free

gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds. The safeguard-sh/gold-scan-action GitHub Action can fail a build on a known exploited vulnerability, on any known vulnerability, or never, so a version you have just removed cannot quietly come back in next week's merge.

3. Gold Registry: start from components that are already fixed

registry.safeguard.sh provides hardened, drop in replacements for upstream packages and container images. Every Gold artifact is held continuously to zero exploitable critical or high CVEs, zero malware and a completed license check, with SLSA L3 or higher provenance, a Sigstore signature and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.

When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened, compatibility tested build of a specific package. This fork and patch flow is in early access today, so ask which packages it covers before you plan around it.

One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.

See how Safeguard would help

The local runner demo shows how to set up Safeguard's runner on your own machine, approve it with its device code and then scan from the terminal. On a lab or confidential computing test host, that is how you would scan the repositories and images behind your attestation service or link monitoring scripts locally. It does not touch the PCIe hardware itself.

Click through it at your own pace, or open it full screen.

The CLI demo is a real session: the Safeguard CLI reads a lodash finding and its advisory, asks before each command, edits package.json to lodash 4.17.21, runs npm install and npm test with approval and pushes nothing. The same approve each step flow applies when a vulnerable library turns up in a tool your team wrote to collect AER counters or drive SPDM checks across a fleet.

Click through it at your own pace, or open it full screen.

The Claude demo shows how to connect Claude to the Safeguard MCP server with OAuth and choose which tools Claude can use. Once connected, you can ask which of your projects contain a given version of an SPDM library, a kernel package or an agent, and get the answer from your own Safeguard data.

Click through it at your own pace, or open it full screen.

To try it on your own machines, download Safeguard and follow the CLI documentation.

Quick check

Sources

• PCI-SIG: PCIe 6.0 specification release (reprinted by Signal Integrity Journal)
• PCI-SIG: PCIe 7.0 specification release (reprinted by Signal Integrity Journal)
• PCI-SIG: PCIe 8.0 specification announcement (Business Wire release, reprinted by Silicon UK)
• OpenFabrics Alliance 2022: PCIe 6.0 specification, a high performance I/O interconnect (PCI-SIG board director)
• IEEE San Diego, 27 August 2021: PCIe 6.0 error model, FEC, CRC and retry (PCI-SIG board director)
• DMTF libspdm: README with SPDM and Secured Messages versions and message sets
• DMTF libspdm: PCIe IDE_KM definitions (pci_idekm.h)
• DMTF libspdm: PCIe TDISP definitions (pci_tdisp.h)
• Linux kernel: PCI register definitions, including FLIT and IDE (pci_regs.h)
• Linux kernel: PCI Trusted Execution Environment Security Manager (TSM)
• Linux kernel: PCI Express Advanced Error Reporting driver guide
• Linux kernel: sysfs ABI for PCIe AER statistics
• Linux kernel: pci-sysfs.c link speed and width attributes
• Linux kernel: kernel parameters (iommu.strict, iommu.passthrough)
• Linux kernel: x86 IOMMU support
• lspci(8) manual page
• Linux Plumbers Conference 2024: configuring IDE and TDISP for confidential VMs (slides)
• NIST SP 800-193: Platform Firmware Resiliency Guidelines
• Microsoft Learn: Kernel DMA Protection
• Microsoft Learn: Hardware errors and error sources (WHEA)
• Microsoft Learn: Get-PnpDeviceProperty

Last reviewed: 23 October 2026.