Bare metal vs virtualized GPU cloud: what the evidence shows

For a single GPU, the public evidence shows little difference between bare metal and a well configured virtual machine. The gap, if any, shows up in multi-GPU and multi-node work, where networking and scheduling matter more. Bare metal removes a layer of software and gives you the whole host. Virtual machines give you flexibility and faster provisioning.

This article explains the options in plain words, sets out the public evidence and what large operators say about their own designs, notes what that evidence does not cover, and gives a method for testing your own workload. No independent multi-node benchmark with published overhead figures was available at the time of writing, and the gaps are stated below rather than filled with guesses.

Key numbers at a glance

• dstack single-GPU benchmark, published 22 July 2025: GPU utilization about 95% to 100% on both a virtual machine and bare metal, one AMD MI300X, with near-identical results at some load levels and small differences either way at others.
• That benchmark's inference test: Llama 3.3 70B Instruct, 1,024 input tokens and 1,024 output tokens, concurrency swept from 4 to 256.
• That benchmark's training test: fine-tuning a 0.5 billion parameter model with LoRA, one epoch, batch sizes swept; the VM was slightly faster at batch size 8 and above, bare metal slightly faster at small batches.
• CoreWeave's 2025 annual report (Form 10-K), on its managed Kubernetes service: it "runs on CoreWeave Cloud's bare-metal GPU infrastructure, eliminating hypervisor overhead to maximize performance." This is a company statement, not an independent measurement.
• CoreWeave's 10-K: committed contracts were over 98% of 2025 revenue, with a weighted-average duration of approximately five years at 31 December 2025.
• StorageSwiss overview, 30 March 2026: three common ways to give a virtual machine a GPU are pass-through, vGPU and MIG. It gives no overhead percentages.
• NVIDIA MIG user guide: one GPU can be split in hardware into up to seven GPU instances, starting with the Ampere generation. NVIDIA's supported GPU list shows seven for A100, H100, H200 and B200.
• AWS documentation on Nitro instances: the Nitro hypervisor "delivers performance that is indistinguishable from bare metal for most workloads." This is a company statement with no benchmark attached.
• The same AWS page lists its H100 and Blackwell training instances (P5, P5e, P5en, P6) as virtualized; the only GPU bare metal families it lists are G4dn and G5g. A bare metal instance can take up to 20 minutes or more after it enters the running state before it is reachable on the network.
• IBM Research, 7 February 2023: virtualization overhead on its Vela A100 cluster is "less than 5%". Company statement, not independently tested.
• Provider quotes for an 8-GPU H100 server span $3.99 to $12.29 per GPU-hour on list (IntuitionLabs survey, first-party prices checked 10 October 2026). The survey does not separate bare metal offerings from VM offerings, so price alone does not tell you which you are buying.

The terms, in plain language

A server is a computer. A hypervisor is software that lets one physical computer run several pretend computers, called virtual machines or VMs. Each VM thinks it has its own processor, memory and disks. Cloud providers use hypervisors to carve up big machines and sell the slices.

Bare metal means you rent the physical server itself, with no hypervisor between your software and the hardware. You get every processor core, all the memory, all the GPUs on that machine, and you install or boot your own operating system or a provider image. Nobody else runs on the box while you hold it.

A virtualized GPU instance means you rent a VM that has been given access to one or more GPUs. How the GPU is handed to the VM is the key technical detail. The StorageSwiss overview describes three methods:

• Pass-through. One physical GPU is given to one VM, and the VM talks to it directly. The article says this offers the highest performance of the three and removes sharing, but utilization drops when the workload is idle and it generally blocks live migration, which is moving a running VM to another host.
• vGPU. NVIDIA's software splits one GPU among several VMs, each running a full driver. The article says performance varies under contention, because workloads compete for GPU cycles, and that it needs recurring per-instance licensing on top of the hardware cost.
• MIG, short for Multi-Instance GPU. The GPU is partitioned in hardware, and each slice gets its own compute, memory and memory bandwidth. The article says it gives guaranteed performance per slice and strong fault isolation, with fixed slice sizes and support limited to A100 and H100 class cards or newer. NVIDIA's own MIG guide says a GPU can be split into up to seven instances, that L2 cache banks, memory controllers and DRAM address buses are each assigned to a single instance, and that one client "cannot impact the work or scheduling of other clients."

When people say "virtualized GPU cloud" they usually mean pass-through VMs for big jobs and shared slices for small ones. The two are very different products, and the evidence for one does not carry over to the other.

The case for bare metal

A provider selling bare metal makes four arguments.

First, no hypervisor overhead. Every layer of software costs a little time, and bare metal removes the layer. CoreWeave says exactly this about its own managed Kubernetes service in its annual report: it runs on bare-metal GPU infrastructure, eliminating hypervisor overhead to maximize performance. The same filing says its Kubernetes clusters use bare metal nodes without a hypervisor. Treat that as the provider's design claim and a reason it was chosen, not as a measured number.

Second, predictability. Without neighbors on the same host, there is no "noisy neighbor" effect, where another tenant's workload slows yours down through shared processor cache, memory bandwidth or network.

Third, full access. You can tune the operating system, drivers and kernel settings, and use hardware features directly. Some of that is harder or impossible inside a managed VM.

Fourth, simpler performance debugging. If a training run is slow, there are fewer places to look.

The case for virtualization

The arguments for VMs are practical.

Provisioning is faster. A VM can be created in seconds or minutes, and torn down just as quickly. A bare metal server needs to be cleaned and reimaged between tenants, which takes longer, though providers have worked hard on this. AWS documentation gives one concrete figure: when a bare metal instance launches, the server runs its full boot and hardware checks, and it can take up to 20 minutes or more from the running state until the instance is reachable on the network.

Granularity is finer. You can rent one GPU, or a slice of one, instead of an entire eight GPU server. For small experiments and small inference services, that is much cheaper.

Management is easier. Providers can snapshot, migrate (with some methods), monitor and resize VMs. The StorageSwiss article lists live migration as supported with vGPU and MIG, and generally blocked with pass-through.

Isolation between tenants is stronger by default in some designs, because the hypervisor draws a hard boundary around each customer.

What the public benchmark shows

The clearest public hands-on comparison is the dstack blog's, dated 22 July 2025, supported by Hot Aisle, which provides AMD GPU bare metal and VM infrastructure. The author says that sponsorship in the post. The test used a single AMD MI300X GPU on each side.

For hardware, the VM had an Intel Xeon Platinum 8470 limited to 13 cores at 2 GHz, 224 GiB of RAM and 13 TB of NVMe storage. The bare metal host had the same processor, restricted to 13 cores and 224 GB of RAM inside Docker so the two sides were comparable.

The inference test served Llama 3.3 70B Instruct with vLLM, using prompts of 1,024 tokens and 1,024 output tokens, with concurrency raised from 4 to 256. Results, as the author describes them: near-identical performance at concurrency 16 to 64, slightly higher bare metal throughput at 4 to 16, slightly better bare metal latency and throughput at 64 to 128, and both sides saturating at 256, which the author attributes to pressure on the GPU's memory for the KV cache. (The KV cache is the memory the model uses to remember the conversation so far. See LLM throughput metrics explained.)

The training test fine-tuned a small model with LoRA, one epoch, with batch sizes swept. Bare metal was slightly faster at small batches, and the VM was slightly faster at batch size 8 and above. Loss curves were nearly identical, GPU utilization was about 95% to 100% on both sides, and memory use was similar. The author suggests the VM's small edge could come from background services on the bare metal host, which has eight GPUs but only one was used.

The honest summary is that on one GPU, with these workloads, the two setups were close and the differences went both ways.

What that benchmark does not tell you

The author states the limits clearly, and they are the limits that matter most for large buyers:

• Only one GPU was tested. Multi-GPU inference with tensor parallelism and distributed training were not covered.
• Bare metal may gain an edge in multi-GPU inference from direct hardware access. That is the author's expectation, not a result.
• For distributed training, the author expects the gap to be smaller, but says the way the network is virtualized, for example with a technology called SR-IOV, would strongly affect multi-node results. This was not tested.
• Expert hypervisor tuning, such as pinning processor cores and aligning memory to the right socket, could narrow the VM gap.
• It is one GPU model from one vendor, with one sponsor.

Large AI training runs depend on how fast the GPUs talk to each other, within a server and between servers. If a virtualization setup adds delay or cuts bandwidth on that network, the cost multiplies across thousands of GPUs. The public evidence cited here does not measure this, so it neither confirms nor rules out that risk.

What large operators say about their own designs

Beyond the single benchmark, the most useful public material comes from operators describing why they built their GPU fleets the way they did. These are company statements about their own systems, so read them as design claims, not neutral measurements. They are still informative, because they show that large clouds have made opposite choices for the same kind of hardware.

AWS sells its large NVIDIA training instances as virtual machines. Its documentation on Nitro instances lists the P5, P5e, P5en, P6-B200 and P6-B300 families under virtualized instances, and lists only two GPU families, G4dn and G5g, under bare metal. The same page describes the Nitro hypervisor as a lightweight hypervisor that "delivers performance that is indistinguishable from bare metal for most workloads." AWS's Nitro overview adds that the system "delivers practically all of the compute and memory resources of the host hardware to your instances," because networking, storage and management run on separate dedicated cards rather than on the host's own processors. Neither page publishes a benchmark for GPU work.

The design idea is worth understanding even without numbers. Much of the classic cost of virtualization came from the host processor doing extra work for every disk read and network packet. Moving that work onto separate hardware, and handing the GPU and network card to the virtual machine directly, removes most of that cost. What remains is the hypervisor's handling of memory and processor scheduling.

IBM Research took the virtual machine route on purpose for its Vela cluster. A post dated 7 February 2023 says "it's generally accepted that bare metal is the path to maximizing AI performance, but VMs provide more flexibility," and that the team accepted a possible small performance cost for productivity. It describes nodes with eight 80 GB A100 GPUs each, linked by NVLink and NVSwitch inside the node and by multiple 100G network interfaces between nodes. The post says the team configured hosts with SR-IOV, huge pages and processor virtualization extensions, and exposed the real device layout to the virtual machine, including which network cards connect to which processors and GPUs. Its stated result: virtualization overhead "less than 5%". IBM describes that as the lowest it knows of; that is its own characterisation.

An IBM workshop paper accepted at SC25, titled "To Virtualize or Not to Virtualize," covers two generations of this design for LLM training, using KVM with PCIe pass-through and SR-IOV for GPU Direct RDMA, a way for GPUs on different servers to exchange data without routing it through the host processor. Its abstract claims "near-bare-metal performance while maintaining virtualization benefits" for both RoCE (Ethernet with RDMA) and InfiniBand style networks. The abstract gives no figures.

CoreWeave, by contrast, says its managed Kubernetes runs on bare metal nodes with no hypervisor, as covered below. So the honest reading is that both designs are in production at large scale, each operator says its choice performs well, and none of them has published an independent, like-for-like test against the other.

Why some providers lean to bare metal for large clusters

CoreWeave sells mostly long take-or-pay contracts. The annual report says committed contracts were over 98% of 2025 revenue and averaged approximately five years weighted duration. When a customer pays for a cluster for years, it expects stable, predictable performance and full control, and the provider benefits from not having to resell slices of a GPU to many tenants. In that setting, bare metal is a natural fit: the customer is effectively renting a dedicated cluster. For a walk-up customer renting one GPU for a few hours, a VM is a natural fit instead.

That is a statement about commercial logic, not a proof of performance. Providers choose designs for many reasons, including cost, operations, security and customer demand. See GPU as a service economics for how those contracts work.

A decision guide

Use this as a rule of thumb, then test.

• You need one GPU for hours or days, for experiments, fine-tuning or a small service: a pass-through VM is usually fine, and the single GPU evidence shows little penalty.
• You run a steady inference service on a few GPUs: either works. Compare price, startup time and how the provider handles a failed GPU.
• You run large distributed training across many nodes: ask hard questions about network design, topology and performance isolation, and test your real job. This is where the choice may matter.
• You share a GPU among many small jobs: MIG slices or vGPU can raise utilization, which is often worth more than a few percent of raw speed. The StorageSwiss article notes pass-through utilization drops when a workload is idle.
• You need tenant isolation for sensitive data: ask exactly how isolation is enforced under each model. Do not assume either is safer without asking.

How to test a provider yourself

Run the same job on both and compare on your own metrics. A simple plan:

1. Pick a real job, not a synthetic one: a short training run or a replay of real inference traffic.
2. Use the same GPU model, driver version, container image and data on both setups.
3. Record tokens per second, time to first token, step time for training, and GPU memory use. The throughput metrics explainer defines these.
4. Repeat the run at different times of day and several times, since shared hosts vary.
5. For multi-node jobs, run a network bandwidth test between nodes and note the result.
6. Compare cost per finished result, not just hourly rate.

Keep the results with the exact versions and settings, so they can be repeated.

Security and operations: the other half of the choice

Performance is only one reason to pick a model. Security and day to day operations often decide it.

Isolation. In a VM, the hypervisor is the boundary between you and other tenants. In bare metal, the boundary is the physical server: you hold the whole machine, and the provider must make sure nothing from the previous tenant remains when you get it. Both models can be run well and both can be run badly. Ask the provider how it wipes disks, resets firmware and GPU memory between tenants, and how it handles the network between tenants. No public, independent comparison of the two on this point was available at the time of writing, so this article makes no claim about which is safer.

Control. On bare metal you control the operating system and drivers, which is useful for tuning, and also means you own patching. On a VM, the provider usually maintains the host and you maintain the guest. Who patches what is a question worth settling in writing.

Failure handling. When a GPU fails, how fast does the provider swap it, and do you lose the whole node? The Llama 3 paper, for scale, reports that GPU problems were 58.7% of the unexpected interruptions in a 54 day snapshot on a 16,000 GPU cluster. Large clusters see failures often, so the provider's replacement process matters more than a small speed difference.

Software supply chain. Whichever model you choose, you will pull drivers, container images and open source packages onto these machines. A rented GPU host is still a place where a poisoned package can run with access to your data and credentials. Treat the images and dependencies you load as something to inventory and scan, whatever the underlying hardware.

Exit and portability. Containers and standard orchestration let you move between VM and bare metal offerings with less rework. Ask whether your setup depends on something specific to one provider's virtualization.

What is firm and what is uncertain

Firm: the definitions of pass-through, vGPU and MIG as described by StorageSwiss; the dstack test design and results as reported by its author; CoreWeave's own statements about its bare metal design and contract mix from its annual report.

Also firm, as company statements rather than measurements: AWS's description of the Nitro hypervisor and its list of virtualized and bare metal GPU instance families; IBM's statement that Vela's virtualization overhead is under 5%; NVIDIA's description of how MIG partitions a GPU.

Uncertain: any independently measured overhead for multi-GPU or multi-node work; whether the single GPU result holds on NVIDIA hardware or other models; how much hypervisor tuning narrows the gap; and the performance of any specific provider's virtualization. The IBM SC25 paper on two generations of virtualized LLM training infrastructure reports "near-bare-metal performance" in its abstract, but its measured numbers are not in the public abstract, so this article does not quote any.

What to watch next

• Independent multi-GPU and multi-node benchmarks of VM against bare metal, including network virtualization.
• Provider disclosures of how GPU access is configured, and performance guarantees in contracts.
• Whether more hosts offer fast bare metal provisioning, which removes VM's speed advantage.
• Follow up tests from dstack, which says multi-GPU and multi-node comparisons are planned as future work.
• The full proceedings version of IBM's SC25 workshop paper, which should show measured multi-node results for virtualized RoCE and InfiniBand setups.
• Whether AWS or other large clouds add bare metal options for their newest GPU families, which today are sold as virtualized instances.

For related reading, see GPU rental prices in 2026, LLM throughput metrics explained and GPU as a service economics.

If you scan the container images you run on rented GPUs, Safeguard offers SBOM and dependency scanning and a free scan at gold.safeguard.sh/scan.

Quick check

Sources

• Benchmarking AMD GPUs: bare-metal and VMs, dstack, 22 July 2025
• GPU virtualization: choosing between pass-through, vGPU and MIG, StorageSwiss, 30 March 2026
• CoreWeave Form 10-K for fiscal year 2025, SEC EDGAR
• The Llama 3 Herd of Models, arXiv 2407.21783 (HTML version)
• H100 rental prices compared, IntuitionLabs, first-party prices checked 10 October 2026
• NVIDIA Multi-Instance GPU user guide: introduction, NVIDIA documentation
• NVIDIA Multi-Instance GPU user guide: supported GPUs, NVIDIA documentation
• Instances built on the AWS Nitro System, Amazon EC2 documentation
• AWS Nitro System overview, Amazon Web Services
• Why we built an AI supercomputer in the cloud, IBM Research blog, 7 February 2023
• To Virtualize or Not to Virtualize: Experiences from Building Two Generations of Virtualized Infrastructure for LLM Training, CANOPIE-HPC workshop, SC25 (abstract)

Benchmark results are as described by their authors, and operator statements are their own claims. The dstack benchmark was supported by a provider of AMD GPU infrastructure, as its author discloses.

Last reviewed: 19 October 2026.