Force valid JSON from a local model with llama.cpp grammars

llama.cpp can constrain generation with a GBNF grammar or a JSON schema, so the model can only produce text that matches the format. That gives you parseable output without retry loops. It does not make the content correct, and the grammar docs list features that are unsupported or slow.

What a grammar does

The llama.cpp grammar docs describe GBNF as a BNF extension with regex like features that constrains model output to a formal grammar. You can use it to force valid JSON or other fixed formats. It works in the command line tools and the server.

The simplest example from the docs accepts only yes or no followed by a newline:

root ::= ("yes" | "no") "\n"

Run it with a grammar file:

./llama-cli -m model.gguf --grammar-file my.gbnf -p 'Answer yes or no.'

Swap in your own model path.

Where to pass it

Per the docs:

• llama-server: pass a grammar field in the completion request body.
• Command line tools: use the grammar option for inline text or the grammar file option for a file.
• JSON schema: in the server, use the json_schema body field for completion endpoints. For the chat completions endpoint, put the schema in the response_format field.
• Testing: a validator tool checks grammars against sample strings.

Why this beats prompting alone

Asking a model nicely for JSON works most of the time. In a pipeline, the rare failure breaks a parser at 3 a.m. With a grammar, every token is chosen from the set the format allows at that point, so malformed output is ruled out by construction. Use it for classification labels, extraction into fixed fields and tool call arguments.

Gotchas from the docs

1. The schema is not shown to the model. The docs say the JSON schema constrains output and is not injected into the prompt. Describe the structure in your prompt too, or the model may fill the shape with weak content. Tool calling is the exception.
2. Objects disallow extra properties by default. The default for additional properties is false. Set it to true if you want extras.
3. Unsupported features are skipped silently. Run the validator on what was generated.
4. Known limits. The docs say prefixItems and nested references are broken, patterns must start with a caret and end with a dollar sign, numeric minimum and maximum work only for integers, and several keywords including patternProperties, uniqueItems and not are unsupported.
5. Performance. The docs note grammars have performance problems and that a bounded repeat is faster than chaining optional items.

Valid does not mean true

A grammar guarantees shape, not correctness. A model can return perfect JSON with a wrong value. Validate the content in your code: check ranges, known identifiers and cross field rules. Treat model output as untrusted input before it reaches a database, shell or another service. See which LLM output sinks need which controls.

A small workflow

1. Write the schema.
2. Describe it in the prompt.
3. Pass the schema or grammar to the server.
4. Validate the result in code.
5. Log failures and review them.

Quick check

Sources

• llama.cpp grammars README

Last reviewed: 19 October 2026.