Two Ollama settings decide how many requests run at once and how many models stay loaded. OLLAMA_NUM_PARALLEL defaults to 1, and OLLAMA_MAX_LOADED_MODELS defaults to 3 per GPU or 3 for CPU inference. Memory use grows with parallel requests times context length, so raise them with a budget in hand.
What the FAQ documents
According to the Ollama FAQ:
- OLLAMA_NUM_PARALLEL sets the maximum number of parallel requests each model processes at the same time. The default is 1. Required RAM scales with this value times the context length.
- OLLAMA_MAX_LOADED_MODELS sets the maximum number of models loaded concurrently, provided they fit in memory. The default is 3 times the number of GPUs, or 3 for CPU inference.
- Keep alive is 5 minutes by default, see keep_alive explained.
A sizing method
- Pick the context length your team needs. Use the smallest that works. See context window settings.
- Estimate the KV cache for one request at that length. See KV cache sizing.
- Multiply by the number of parallel requests you want.
- Add the weight memory of each model you want loaded.
- Compare with your memory, leaving margin.
If the total is too large, lower parallel requests first, then context, then consider a smaller or more heavily quantized model.
Parallel requests versus queueing
You do not need high parallelism for light team use. Requests that exceed the limit wait their turn, so a team of four with occasional prompts may be served acceptably with a low value. Test with a script that sends overlapping requests, and look at time to first token as well as total time.
For heavier use, the batching approach of server oriented engines is better suited. See continuous batching explained and choosing a runtime.
Several models at once
Loading a chat model, an embedding model and a small code model together is common. Each loaded model holds its weights and cache. The FAQ says the loaded model limit applies when they fit in memory, so the practical limit is your hardware, not the number three. If two big models cannot fit together, expect swapping between them, with load delays on each switch.
Shared server safety
A shared Ollama server needs the controls in binding and exposure: loopback behind an authenticating proxy, a network allow list and request logs. Raising concurrency also raises the cost of a misuse, because one client can occupy the GPU for everyone.
Measuring
Record these under load, for one setting at a time: memory use, time to first token, tokens per second per request, and any failures. Change one variable per test. Keep the results in a short note so the next person does not repeat the work.
Quick check
Why did memory jump after I raised OLLAMA_NUM_PARALLEL?
The FAQ says required RAM scales with parallel requests times context length. Each parallel slot needs its own cache.
Is OLLAMA_MAX_LOADED_MODELS a memory guarantee?
No. It is a ceiling on count, and the FAQ says models load as long as they fit in memory.
Sources
Last reviewed: 19 October 2026.
