A private code question tool on your laptop: the local stack explained

A private code question tool needs five parts: a chunker, a keyword index, an embedding model, a vector index and a local model that writes the answer. All five can run on one laptop, so your source never leaves the machine. Each part is also software you must keep patched.

The five parts

1. Chunker. Splits files into pieces along function and class boundaries. A parser such as tree-sitter handles many languages and tolerates syntax errors. See chunking source code.

2. Keyword index. SQLite's FTS5 module gives full text search with bm25 ranking inside a single database file. See the FTS5 how to.

3. Embedding model. Turns chunks and questions into vectors. A local runner such as Ollama serves embeddings from an endpoint on localhost. See running embeddings locally.

4. Vector index. Stores vectors and finds nearest neighbours. At laptop scale, exact search is usually fine. See exact versus approximate search.

5. Answer model. A local language model reads the retrieved chunks and writes the response. llama.cpp is an inference engine for running models locally, in a quantised GGUF format, and it can start an OpenAI compatible API server, so many tools can talk to it. See GGUF and quantization.

The flow

1. Index: chunk, store text in the keyword index, embed, store vectors.
2. Ask: embed the question, run keyword and vector search, merge, optionally rerank.
3. Answer: send the top chunks and the question to the local model, and ask it to cite file paths.

What stays local, and what might not

If every part is local, no source code crosses the network. Check that this is true. Some tools phone home for updates or telemetry, and some default to a hosted model when a local one is not configured. Watch network connections during a first run, and read each tool's settings.

Hardening checklist

• Bind servers to localhost only. A model server listening on all interfaces becomes a network service anyone on the same network can reach.
• Put the index files in a directory with user only permissions.
• Exclude secrets and environment files from indexing. See what to scan before you index.
• Download models and tools from the original publishers, and record versions.
• Update the runner and libraries on a schedule. Local AI runners have had published vulnerabilities, including the issues in our write up of a memory leak in a popular local runner.
• Do not give the model tools that can write files or run commands unless you have to. A question tool needs read access only.
• Encrypt the disk. The index is a searchable copy of your code.

Know your dependencies

This stack pulls in a parser, a database, an embedding library, a model runner and possibly a vector extension, each with its own dependencies. Safeguard can scan a project's dependencies and produce an SBOM, and the free scan at gold.safeguard.sh/scan is a quick way to check what you are about to rely on.

When to stop and use something else

If you need team wide access with permissions, a laptop tool is the wrong shape. You would run a shared service with access labels on every chunk, see permission aware retrieval.

Quick check

Sources

• llama.cpp README
• Ollama embeddings documentation
• SQLite FTS5 documentation
• Tree-sitter documentation

Last reviewed: 19 October 2026.