Ollama answers cut off or forget things? Check the context window

If a local model seems to forget the start of a long prompt, the likely cause is the context window. Ollama's FAQ says it uses 4096 tokens by default. Anything beyond that is not seen by the model. You can raise the limit with an environment variable, a session command or an API option.

What happens when you go over

A context window is the number of tokens the model can look at in one request, counting both your prompt and its reply. When a conversation or a pasted file exceeds it, the runtime has to drop something. In practice that tends to look like a model that ignores the top of your document or contradicts something you said earlier.

This is not a sign of a weak model. It is often a settings problem, and it is easy to confirm. Paste a short instruction at the very top of a long input, ask the model to repeat it, and see whether it can.

Three ways to change it

The Ollama FAQ documents these:

1. Server default. Set OLLAMA_CONTEXT_LENGTH before starting the server, for example 8192.
2. Interactive session. Inside ollama run, use /set parameter num_ctx 4096 with your own value.
3. Per request. Through the API, pass num_ctx inside the options object.

The per request route is the safest for applications because it makes the setting visible in your code instead of hiding it in a service environment.

OLLAMA_CONTEXT_LENGTH=8192 ollama serve

What it costs

A larger window uses more memory, because the KV cache grows with context. The FAQ states that required RAM scales with the number of parallel requests times the context length. So doubling the context roughly doubles the cache, and doubling parallel requests does too.

Before you jump to a very large window, check these:

• Does the model support the longer window you are asking for? Read its model card, because a runtime will let you set a number the model was not trained for.
• Do you have the memory? See how the KV cache grows.
• Do you need all that text in the prompt? Retrieval of the relevant chunks often beats pasting everything.

A habit that helps

Set the context explicitly in every application that calls the model. Defaults can change between versions, and a hidden default is a source of bugs that look like model quality problems. If you need to compare two runtimes, set the same window on both first.

If the window is large enough and quality is still poor, move on to other causes: quantization level, prompt format, or the model itself. The guide to a small eval helps separate them.

Quick check

Sources

• Ollama FAQ

Last reviewed: 19 October 2026.