Multiply the parameter count in billions by the bytes per weight to get the weight memory in gigabytes. A 16-bit model needs about 2 GB per billion parameters, a 4.5 bit quantization about 0.56 GB. Then add headroom for the KV cache, which grows with context length and the number of parallel requests.
The arithmetic
Hugging Face's optimization guide gives the rule for full precision: weights for a model with X billion parameters need roughly 2 times X GB in bfloat16 or float16, and 4 times X GB in float32. The same guide says this is best read as an "at most" number, because of the gap between 1000 and 1024.
For quantized files the same idea works with bits instead of bytes. Divide the bits per weight by 8, then multiply by the billions of parameters.
The Hugging Face GGUF page lists effective bits per weight for the K-quant family: Q6_K at 6.5625, Q5_K at 5.5, Q4_K at 4.5, Q3_K at 3.4375 and Q2_K at 2.625.
Worked numbers for weights only
Weights only, in GB, rounded:
- 7B: 14 at 16 bit, 5.7 at Q6_K, 4.8 at Q5_K, 3.9 at Q4_K, 3.0 at Q3_K
- 13B: 26 at 16 bit, 10.7 at Q6_K, 8.9 at Q5_K, 7.3 at Q4_K, 5.6 at Q3_K
- 70B: 140 at 16 bit, 57.4 at Q6_K, 48.1 at Q5_K, 39.4 at Q4_K, 30.1 at Q3_K
These are computed from the published bits per weight, so treat them as estimates. Real files differ a little because some tensors are stored at other precisions, and the embedding and output layers vary by model.
The guide's own measurement shows the same pattern in practice. For one 15.5 billion parameter coding model, peak memory was about 29 GB in bfloat16, about 15 GB with 8 bit quantization and about 9.5 GB with 4 bit.
What the weights number leaves out
Weights are only the fixed part. During generation the runtime also keeps a key value cache for every token in the context. The guide says that for short inputs, under about 1,024 tokens, weights dominate. For long inputs or multi-turn chat the cache can become a large share. Its own example puts the cache at roughly half the size of the weights for a 16,000 token context on that model.
Add the compute buffers and the operating system on top. A practical habit is to leave 10 to 20 percent of memory free beyond weights plus cache, then test with your real context length. That percentage is a working habit, not a published figure.
For the cache itself, see how the KV cache eats memory in long contexts.
What to do when it does not fit
You have four levers, in rough order of cost to quality:
- Choose a smaller context length.
- Quantize the KV cache if your runtime supports it.
- Pick a lower bit quantization of the same model.
- Pick a smaller model, which is often better than a heavily squeezed large one. Test it on your task, because the answer depends on the work.
llama.cpp can also split a model between GPU and CPU memory when it is larger than VRAM, at a speed cost. See what hybrid offload really costs.
One check before you download
Read the file size on the model page. The size of a GGUF file is nearly the memory you need for weights, which beats any formula. Then add cache and buffers.
If you build tools around local models, Safeguard's free scan at https://gold.safeguard.sh/scan can check the dependencies of those tools for known vulnerabilities.
Quick check
Does a 4 bit model use a quarter of the memory of a 16 bit one?
Roughly, yes for the weights. K-quants carry scale data, so Q4_K is 4.5 bits per weight, about 28 percent of 16 bits. The KV cache is separate and does not shrink unless you quantize it.
Is my VRAM estimate the same on Apple silicon?
The weight arithmetic is the same. What differs is that memory is shared between CPU and GPU, so the limit is total memory minus what the system needs.
Sources
Last reviewed: 19 October 2026.
