Scale up vs scale out: NVLink, UALink, Ethernet and InfiniBand

Large AI clusters use two different networks. A scale up network ties a few dozen accelerators together inside one rack so they behave like one big chip. A scale out network links many racks together. NVLink dominates scale up today, while InfiniBand and Ethernet compete for scale out, according to analysts as of October 2026.

This post explains both layers in plain terms, lines up the main technologies, names who supplies them, and flags where the sources disagree. It is about networking inside AI data centers, not the internet connection a customer uses to reach them.

Key numbers at a glance

• Dell'Oro Group (analyst forecast, 28 July 2026): cumulative spending on AI back end data center switches is forecast to approach 1 trillion dollars over 2026 to 2030.
• Dell'Oro (same release): scale up is expected to exceed half of the AI back end switch market by 2030.
• Dell'Oro (same release): scale up will more than double the addressable market for AI back end switches, and the market will stay supply constrained rather than demand constrained through 2030.
• Dell'Oro (same release): the 1.6 Tbps port speed ramp is expected to be at least five times faster than the 800 Gbps ramp.
• NVIDIA NVLink page (accessed 19 October 2026): fifth generation, 1,800 GB/s per GPU; sixth generation, 3,000 GB/s per GPU, under a note reading "Preliminary specifications; may be subject to change."
• NVIDIA NVLink page: 72 GPU rack domain, 130 TB/s total with NVLink 5 and 216 TB/s with NVLink 6 (preliminary).
• UALink Consortium (accessed 19 October 2026): 200G 1.0 specification released April 2025; four new specifications published April 2026.
• Ultra Ethernet Consortium (press release, 11 June 2025): specification 1.0 released, which the consortium says scales to millions of endpoints. Its site now offers a 1.1 specification, with no release date stated on the page.
• NVIDIA Spectrum-X page: Spectrum-6 switch chip at 102.4 Tb/s, a company claim of up to 1.6 times the performance of off the shelf Ethernet.
• NVIDIA Quantum-X Photonics page: 144 ports of 800 Gb/s InfiniBand, over 10,000 GPUs in a two level fat tree.

Two networks, two jobs

Picture a cluster as a city. Inside each building (a rack), people on different floors need to talk constantly and instantly. Between buildings, traffic is heavier in total but can tolerate slightly longer trips.

The rack level network is called scale up. Its job is to make a group of accelerators (GPUs or similar chips) share memory and exchange data at very high speed, as if they were one device. NVIDIA's NVL72 design, for example, joins 72 GPUs in one NVLink domain.

The cluster level network is called scale out. It connects racks to one another, and to thousands of other accelerators, so a single training job can span the whole building. NVIDIA's own Vera Rubin NVL72 page says the system "scales out" with either Quantum-X800 InfiniBand or Spectrum-X Ethernet.

You will also see "back end" and "front end." The back end is the accelerator to accelerator fabric just described, both scale up and scale out. The front end is the ordinary data center network that connects servers to storage, users and the internet. Dell'Oro's July 2026 forecast treats the back end as its own switch market, approaching 1 trillion dollars over five years, which shows how far it has grown beyond being a corner of the ordinary data center network.

Scale up: where NVLink leads

NVLink is NVIDIA's own link between accelerators. NVIDIA's page lists three recent generations:

• Fourth generation (Hopper): 900 GB/s per GPU over 18 links, with 8 GPU domains.
• Fifth generation (Blackwell): 1,800 GB/s per GPU, with 8 or 72 GPU domains. In a 72 GPU rack, NVIDIA says total bandwidth is 130 TB/s.
• Sixth generation (Vera Rubin): 3,000 GB/s per GPU over 36 links, which NVIDIA says is 1.7 times the previous generation and 12 times PCIe Gen6. In a 72 GPU rack it states 216 TB/s. The specification table carries the note "Preliminary specifications; may be subject to change."

The glue is the NVLink Switch. NVIDIA says its switch chips connect many NVLinks to give all to all GPU communication at full NVLink speed across the entire rack, and that each switch carries SHARP engines for in network reductions and multicast. In plain terms, a reduction is the step where many chips add their partial results together; doing it inside the switch saves round trips. NVIDIA also says NVLink connections can be extended across nodes, which blurs the line between scale up and scale out at the top end. When comparing NVLink figures from different sources, check whether each counts one direction or both.

Dell'Oro says NVLink "currently dominates" scale up deployments, while Ethernet and UALink are gaining momentum.

NVIDIA also sells a way for other chip designers to use NVLink. NVLink Fusion, according to NVIDIA's page, lets hyperscalers (very large cloud companies) build what it calls ASIC hybrid AI infrastructure, pairing NVLink and rack scale designs with their own semi custom chips or CPUs. The page does not name partners. The offer matters because it lets a company that designs its own accelerator keep NVIDIA's scale up fabric rather than adopt an open alternative.

Scale up: the open alternatives

Two open efforts aim at the same job.

UALink

UALink stands for Ultra Accelerator Link. Its promoter group formed in May 2024 with AMD, Broadcom, Cisco, Google, Hewlett Packard Enterprise, Intel, Meta and Microsoft, according to the consortium's own page. The consortium incorporated in October 2024, and Alibaba, Apple and Synopsys joined its board in January 2025. The 200G 1.0 specification came out in April 2025. In April 2026 the consortium published four further specifications, covering in network compute, chiplets, manageability and 200G performance, per the page's link names.

The consortium's home page does not state technical limits such as the maximum number of accelerators in a pod or the per lane rate beyond the "200G" in the specification's name. Readers who need those details should go to the specification itself rather than rely on secondary summaries. What the membership list does show is who wants an alternative to a single vendor's scale up link: chip designers, switch makers and the largest buyers of accelerators.

Ethernet for scale up

The other path is Ethernet itself, adapted for the rack. The Open Compute Project (a hardware standards community) lists an ESUN workstream, titled "Ethernet for Scale up Networks," and has posted an "OCP ESUN Network Operator Requirements Base Specification V1.0.0" under its networking project. The project page does not set out ESUN's membership or goals beyond that title.

Dell'Oro expects Ethernet to become "the leading scale up technology by the end of the decade." That is a forecast, not a measured result.

Scale out: InfiniBand versus Ethernet

Scale out is where the long running argument sits.

InfiniBand

InfiniBand is a networking standard built for high performance computing. It uses a lossless, credit based design with built in remote direct memory access, or RDMA, which lets one machine write straight into another machine's memory without bothering its processor. Lossless means the switch does not drop packets when busy; a sender only transmits when the receiver has signalled room for the data. NVIDIA sells InfiniBand switches and adapters, and its current flagship is the Quantum-X800 generation named on its Vera Rubin page.

NVIDIA's page highlights SHARP, which moves some collective math, such as adding up results from many chips, into the switches themselves. NVIDIA also claims self healing recovery "5,000X faster than any other software based solution." Those are vendor claims without supporting data on the page.

Ethernet for AI

Ethernet is the networking standard almost every data center already uses, which brings a large supplier base and familiar tools. The challenge is that classic Ethernet drops packets under heavy load, while AI training traffic is bursty and punishes loss. RDMA over Converged Ethernet, called RoCE, adds RDMA to Ethernet. RoCE works best when the Ethernet network is tuned to avoid drops, using settings such as priority flow control, which pauses a sender when a switch buffer fills. Getting those settings right across a large fabric takes care, which is part of why InfiniBand kept its place in large training clusters for so long.

To fix the weak points, vendors and cloud companies formed the Ultra Ethernet Consortium. Its steering members include AMD, Arista, Broadcom, Bull, Cisco, HPE, Intel, Meta, Microsoft and Oracle, and its general members include NVIDIA among others. It released specification 1.0 on 11 June 2025. The announcement describes a stack spanning network cards, switches, optics and cables, built on standard Ethernet, covering transport, congestion control, direct memory access, Ethernet link and physical layer technology, and network security, and says it scales to millions of endpoints. Its site now offers a 1.1 document. A consortium update from August 2024, written before the 1.0 release, described the planned contents: multipath packet spraying (spreading one flow across many paths), a sender based congestion control scheme, optional packet trimming in switches, link layer retry, in network collectives, and security features. Treat that as the plan as described then, not a verified list of what ships.

NVIDIA's own Ethernet

NVIDIA sells Ethernet too. Its Spectrum-X platform pairs Spectrum switches with SuperNIC network cards (BlueField 3, ConnectX 8 and ConnectX 9 across generations). The page claims up to 1.6 times the performance of off the shelf Ethernet, and the newest Spectrum-6 chip is listed at 102.4 Tb/s, with the SN6600 offering 128 ports at 800G. NVIDIA names xAI as a customer whose 100,000 GPU Hopper system runs on Spectrum-X. The 1.6 times figure is a vendor claim and its baseline is not defined on the page.

The takeaway is that "Ethernet versus InfiniBand" is no longer a clean contest. One company sells both, and the open Ethernet camp is trying to make Ethernet behave more like InfiniBand.

How one training job uses both networks

A simple way to see the split is to follow one job. When a very large model is trained, its parameters and the data it works on are divided across many accelerators. Some of that division needs constant, fine grained exchange: pieces of a single calculation that must be combined many times per second. That traffic stays inside the rack, on the scale up network, where bandwidth is highest and delay is lowest.

Other parts of the division can tolerate more delay. Groups of racks, each holding a full or partial copy of the work, periodically share results with each other. That traffic crosses the scale out network. Engineers choose how to split the job so the busiest conversations stay on the fastest links. This is why a bigger scale up domain can matter: if 72 accelerators share one fast domain instead of 8, more of the chatter stays on the fast side.

It also explains why the two layers use different technology. Scale up links are short, so they can use copper and chip level signalling, and they need to look like memory access. Scale out links are longer, run over switches and optics, and need congestion control so that thousands of senders do not flood the same path. The congestion features in InfiniBand and in Ultra Ethernet exist for the second case.

How to read a networking claim

Networking announcements are full of large numbers, and a few habits help. First, check the unit: GB/s means gigabytes per second, Gb/s means gigabits, and one byte is eight bits, so the same link can look eight times faster if the unit changes. Second, check whether a figure is per direction or both directions, which is a common reason published bandwidth figures for the same link differ. Third, check the baseline of any "times faster" claim. The 1.6 times figure on NVIDIA's Spectrum-X page compares with "off the shelf" Ethernet, which the page does not define. Fourth, separate per port, per switch and per rack totals, because a rack total sums many links. Finally, check the date and whether a spec is final or labelled preliminary, as NVIDIA's own NVLink 6 figures are.

Who supplies what

Based on company and consortium pages:

• Accelerators with proprietary scale up links: NVIDIA (NVLink).
• Open scale up standards: members of the UALink Consortium, with AMD, Broadcom, Cisco, Google, HPE, Intel, Meta and Microsoft as the original promoters.
• Ethernet switch chips and switches: NVIDIA (Spectrum-6 at 102.4 Tb/s), and Ultra Ethernet steering members including Broadcom, Cisco and Arista.
• InfiniBand switches and adapters: NVIDIA.
• Custom accelerators: cloud companies designing their own chips must choose a scale up fabric, whether NVLink Fusion, UALink or Ethernet. The chips themselves are covered in the look at AI chip market share.

Market share by networking supplier is not public in primary sources. Dell'Oro's research page says it tracks Ethernet and InfiniBand shares in scale out and Ethernet, UALink and NVLink shares in scale up, but the figures sit in paid reports.

Why it matters to buyers

Network cost is now a large budget line. Dell'Oro's forecast of nearly 1 trillion dollars in switch spending over five years sits on top of the compute and memory costs discussed in the HBM explainer. Dell'Oro also says the market will stay "supply constrained rather than demand constrained" through its forecast period. For anyone reading commitment announcements such as those in Anthropic's compute commitments, switches, cables and optics are part of what is being bought, not just chips.

What is firm and what is uncertain

Firm, from primary sources:

• NVIDIA's published NVLink bandwidth per GPU for Hopper and Blackwell, and its preliminary figure for Vera Rubin.
• UEC specification 1.0 on 11 June 2025, and UALink 200G 1.0 in April 2025.
• Spectrum-X and Quantum-X product descriptions as NVIDIA states them.

Uncertain:

• Final NVLink 6 numbers, which NVIDIA marks as preliminary.
• Market share between InfiniBand and Ethernet in scale out, which sits in paid analyst reports.
• Real world gains from Ultra Ethernet features and Spectrum-X claims, which come from vendors and standards bodies, not published independent tests.
• Dell'Oro's forecasts, which are projections that the firm itself raised substantially from its January 2026 outlook.

None of this is a recommendation to buy or avoid any product.

What to watch next

• Whether UALink and Ethernet scale up silicon ships in real racks and at what port speeds.
• Contents and dates of the Ultra Ethernet 1.1 specification, which the site lists without a date.
• OCP ESUN outputs after the October 2026 summit.
• Dell'Oro's full 2026 share data for NVLink, Ethernet and UALink.
• The shift to 1.6 Tbps and then 3.2 Tbps ports, and how the optics keep pace, covered in the post on co-packaged optics.

Clusters like these run a lot of open source software, and Safeguard offers SBOM and dependency scanning for it, with a free scan.

Quick check

Sources

• NVIDIA NVLink page
• NVIDIA Spectrum-X page
• NVIDIA InfiniBand page
• NVIDIA silicon photonics networking page
• NVIDIA Rubin platform page
• UALink Consortium
• Ultra Ethernet Consortium
• Ultra Ethernet Consortium, specification 1.0 announcement
• Ultra Ethernet Consortium, specification update, August 2024
• Open Compute Project networking project
• Dell'Oro Group, AI back end switch sales to approach 1 trillion dollars, 28 July 2026
• Dell'Oro Group, AI back end networks research page

Last reviewed: 19 October 2026.