Pick by the number of users and how much control you need. Ollama is a simple runner for one developer. llama.cpp is the inference engine and server you tune directly, with wide hardware support. vLLM is a library and server aimed at serving many requests efficiently. All three are open projects, and their docs describe different goals.
What each project says about itself
llama.cpp. Its README says the main goal is to enable LLM inference with minimal setup and state of the art performance. It lists Metal, CUDA, Vulkan and CPU backends, supports 1.5 bit through 8 bit integer quantization, and offers CPU plus GPU hybrid inference. It includes an OpenAI compatible server.
vLLM. Its documentation calls it a fast and easy to use library for LLM inference and serving. It lists PagedAttention, continuous batching, many quantization formats including GGUF, speculative decoding, and an OpenAI compatible server. Hardware includes NVIDIA and AMD GPUs and several CPU types, with more through plugins.
Ollama. Its FAQ describes a local server on 127.0.0.1 port 11434 with models stored in a per user folder, settings through environment variables for context length, keep alive, parallel requests and loaded model count. I did not find a statement of its design goals in the pages I fetched, so judge it by the features it documents.
A decision table
- One developer, laptop, wants it working in minutes: Ollama.
- You want direct control of quantization, offload and flags: llama.cpp.
- A small team shares a GPU box and requests overlap: llama.cpp server or vLLM, test both.
- Many concurrent users, throughput matters: vLLM.
- Apple silicon: llama.cpp lists a Metal backend, and runners built on it follow.
- You need an OpenAI compatible endpoint: all three offer one, per their docs.
This is a starting point, not a benchmark. Run your own model and prompts, because results depend on hardware, quantization and traffic.
What changes with concurrent users
Single user runners optimize for one conversation at a time. Servers that batch requests together can serve more users on the same GPU. See continuous batching and PagedAttention.
Security differences to remember
Whichever you pick, the same rules apply:
- Keep the server on loopback unless a proxy with authentication fronts it. See binding and exposure.
- Pin runtime versions and watch advisories. This site has write ups on issues in several local inference engines.
- Verify model files and prefer safe formats.
- Leave tool and MCP features off on any server that others can reach.
Click through it at your own pace, or open it full screen.
Inference engines are large dependency trees. Safeguard's SBOM and dependency scanning can inventory the Python or native packages your serving stack pulls in.
Quick check
Can I switch runtimes later without losing my models?
Often partly. llama.cpp and Ollama both work with GGUF files, and vLLM lists GGUF among its supported formats. Check support for your specific model before you plan a move.
Which one is fastest?
There is no honest single answer. It depends on the model, quantization, hardware and concurrency. Benchmark on your own workload.
Sources
Last reviewed: 19 October 2026.
