How to run a local embedding model with Ollama for private search

To run embeddings locally with Ollama, pull an embedding model, then send text to its /api/embed endpoint on localhost port 11434. You get back normalised vectors you can store in any index. Your source text never leaves your machine during embedding.

Why embed locally

Embedding a codebase means sending every file to a model. With a hosted model, that is your source code leaving your network. A local model removes that step. For many teams with private repositories, this is the deciding factor, and it is worth confirming with your own policy.

Local embedding is also free per call and works offline.

The basic call

Ollama's documentation lists the embed endpoint and uses embeddinggemma as its example model. Vector length depends on the model, typically 384 to 1024 dimensions.

curl http://localhost:11434/api/embed -d '{"model": "embeddinggemma", "input": "def resend_pending_files(queue):"}'

From Python:

import ollama
r = ollama.embed(model="embeddinggemma", input=["first chunk", "second chunk"])
vectors = r["embeddings"]

Batch your input

The documentation says you can pass an array of strings as input and receive one vector per string. Batching is far faster than one request per chunk. Start with batches of a few dozen chunks and adjust to your memory.

Normalised output

The endpoint returns L2 normalised vectors, so each has unit length. The documentation recommends cosine similarity for most semantic search. With unit vectors, cosine and dot product agree, which lets you use whichever your store handles fastest.

The one rule that matters

Use the same model for indexing and querying. The documentation warns that mixing models produces incompatible vectors. Write the model name into your index metadata and refuse to query if the names differ.

Putting it together

1. Chunk your code, see chunking source code.
2. Send chunks in batches to the local endpoint.
3. Store vectors with path, symbol and model name.
4. At query time, embed the question with the same model and search.
5. Add a keyword index alongside, per hybrid retrieval.

Keep the local server local

Check what the server listens on. A service bound to localhost is reachable only from your machine. If you change it to listen on all interfaces so a teammate can use it, you have created a network service that needs authentication and patching. Ollama has had published vulnerabilities, for example the issues covered in our write up of a pull path traversal bug, so keep it updated.

Test the output once

Before indexing thousands of chunks, embed two sentences you know are related and one that is not, then compute cosine similarity for each pair. The related pair should score clearly higher. If it does not, check that you are sending the text you think you are sending, and that the model name is spelled correctly. A quick sanity test like this catches wrong endpoints and truncated input early.

Quick check

Sources

• Ollama embeddings documentation

Last reviewed: 19 October 2026.