Open-Weight Model Families: Licences, Sizes, Formats, Provenance

An open-weight model publishes trained parameters you can download and run, under whatever licence the publisher chose. Open source AI, as the OSI defines it, also requires training code and detailed data information. To adopt one safely, read the licence file, prefer safetensors or GGUF over pickle, pin a commit, check hashes and record an AIBOM entry.

This guide goes one level deeper than our history of the Hermes model family, our walkthrough on how to generate an AIBOM with open source tools and our comparison of model cards, dataset cards and AIBOMs. Here we read the primary documents themselves: the Open Source Initiative's Open Source AI Definition 1.0, the safetensors and GGUF format specifications, the PyTorch and Python documentation on pickle, the OpenSSF model signing project, the CycloneDX and SPDX AI specifications, the European Commission's July 2025 guidelines for general purpose AI providers, NIST AI 600-1, and the model cards, configuration files and licence texts that publishers put on Hugging Face themselves.

This post describes model families only by licence, size, format and provenance. It does not rank models or compare benchmarks between labs. Every parameter count, context length and licence term comes from the publisher's own card, config or licence, fetched on 23 October 2026.

Key takeaways

• "Open weights" means the parameters are downloadable. It says nothing on its own about the licence, the training data or the training code. The OSI Open Source AI Definition 1.0 requires all three: data information, code and parameters.
• One family can carry several licences. In the Qwen2.5 instruct line, the 3B checkpoint is research only, the 7B is Apache 2.0 and the 72B has a 100 million monthly active user clause. Read the licence for the exact checkpoint.
• File format is a security decision. Pickle files can run code when loaded. Safetensors is a JSON header plus raw bytes; GGUF is a typed header plus tensors, though it can carry a Jinja chat template your runtime renders.
• Memory scales with bits per weight: a 7.24 billion parameter model needs about 14.5 GB at 16 bits and about 4.1 GB at Q4_K, before the KV cache. Mixture of experts models need memory for every expert.
• Provenance is a chain: pinned commit, per file SHA-256, a signature (OpenSSF model signing) and an AIBOM entry in CycloneDX ML-BOM or the SPDX 3.0 AI profile.
• Keep local servers on 127.0.0.1 (the documented default for llama.cpp's server and Ollama) and keep trust_remote_code off.
• The EU AI Act's open source exception skips some documentation duties, never the copyright policy or training summary, and not for systemic risk models. User thresholds and non-commercial clauses disqualify a licence.

What is the difference between open-weight, open source AI and API-only models?

Three ways a model reaches you

An API-only model runs on the publisher's infrastructure. You never hold the weights, so you cannot inspect them, run them offline or pin a byte-exact copy. Provenance becomes a contract question.

An open-weight model publishes its trained parameters, usually on Hugging Face, under a licence the publisher writes or selects. You can run, quantise and usually fine-tune it. What you may do commercially, at what scale and for which purposes is set by the licence. Training data and code may or may not be published.

An open source AI system, as the Open Source Initiative defines it, goes further: it grants the freedoms to use, study, modify and share, plus what the OSI calls the preferred form for making modifications.

What the OSI Open Source AI Definition 1.0 actually requires

The OSI announced version 1.0 of the Open Source AI Definition on 28 October 2024 at All Things Open, after a year long community process. It grants four freedoms derived from the Free Software Definition: use for any purpose without asking permission, study and inspect, modify for any purpose including changing output, and share with or without modifications.

The precondition is access to the preferred form for making modifications, which for machine learning has three required elements:

• Data information. Information about the training data detailed enough that a skilled person could build a substantially equivalent system: a complete description of all training data (including unshareable data), its provenance, how it was obtained, selected, labelled and filtered, plus listings of public and third party data, including data available for a fee.
• Code. The complete source code used to train and run the system, from data processing and training settings to tokenisers, inference code and architecture, under OSI approved licences.
• Parameters. The weights or other configuration settings, under OSI approved terms; the definition gives intermediate checkpoints and final optimiser state as examples.

The definition adds that "Open Source models" and "Open Source weights" must include the data information and code used to derive the parameters. It does not require the training data itself to be released, only detailed information about it.

A worked contrast from two model cards

The OLMo 2 7B card from Ai2 says code and model are under Apache 2.0, that the model was trained on the Dolma dataset (its metadata names allenai/olmo-mix-1124), links the training repository, and publishes intermediate checkpoints as revisions named stepXXX-tokensYYYB.

The Mistral 7B v0.1 card publishes weights under Apache 2.0 with a config and tokenizer, enough to run and fine-tune. The Hub card does not publish a training data listing or training code.

Both are open weight under a permissive licence; only the first sets out to supply the data information and code the OSI definition asks for. We describe what each card says; the OSI judges conformance with its own checklist.

Comparison: API-only, open-weight, open source AI

How do model families work?

A model "family" is a set of checkpoints that share an architecture and a training lineage but differ in size, training stage or packaging. Four distinctions explain most of what you see on a model page.

Base versus instruct

A base checkpoint is trained to predict the next token and continues text. An instruct or chat checkpoint is further trained to follow a conversational format.

Hub metadata records this lineage in base_model. Qwen2.5 7B Instruct lists Qwen/Qwen2.5-7B. Nous Research's Hermes 3 8B lists meta-llama/Meta-Llama-3.1-8B, which is why its licence tag points to the Llama family. Our Hermes history traces that lineage version by version.

Instruct checkpoints depend on a chat template, the exact token layout for each turn. OpenAI's gpt-oss card says both checkpoints "should only be used with the harmony format". Templates ship in tokenizer_config.json, a template file, or GGUF metadata as tokenizer.chat_template, which the GGUF specification defines as a Jinja template. That makes the template part of the supply chain.

Distillation and cross-family lineage

Some families are built from other families. DeepSeek's R1 card describes six dense "distill" checkpoints fine-tuned on reasoning data generated by R1, built on Qwen2.5 and Llama bases. The card's licence section is explicit that the Qwen based distills derive from models originally under Apache 2.0, the 8B Llama distill derives from Llama 3.1 8B under the Llama 3.1 licence, and the 70B Llama distill derives from Llama 3.3 70B Instruct under the Llama 3.3 licence. One family name, three licence ancestries. Our post on whether a fine-tuned model inherits its base model licence covers how to reason about that.

Dense versus mixture of experts

In a dense model, every parameter participates in every token. Mistral 7B, Llama 3.1 8B, Qwen2.5 and OLMo 2 are dense; their configuration files contain no expert settings.

In a mixture of experts (MoE) model, some layers contain several parallel feed-forward "experts" and a router picks a few of them per token. The configuration file says how many:

• Mixtral 8x7B v0.1: num_local_experts 8, num_experts_per_tok 2. The Hub's safetensors metadata totals 46,702,792,704 parameters.
• gpt-oss-20b: 32 local experts, 4 per token. OpenAI's card gives 21B total parameters with 3.6B active.
• gpt-oss-120b: 128 local experts, 4 per token; the card gives 117B total with 5.1B active.
• DeepSeek V3 and R1: n_routed_experts 256, num_experts_per_tok 8. The Hub's safetensors metadata for the published files totals about 684.5 billion stored parameters.

The practical rule: memory scales with total parameters, compute per token scales with active parameters. All experts must be loaded because any of them may be selected for the next token.

Parameter counts and size labels

Size labels in names ("7B", "8x7B", "72B") are rounded. The exact count is in the Hub's safetensors metadata, which the API exposes per checkpoint. Mistral 7B v0.1 is 7,241,732,096 parameters; Qwen2.5 7B Instruct is 7,615,616,512; Llama 3.1 8B is 8,030,261,248. The GGUF specification formalises the label as <expertCount>x<count><scale-prefix>, so "8x7B" means eight experts of roughly seven billion parameters each, not 56 billion parameters in total (shared layers are counted once).

One trap: for checkpoints stored in packed low bit formats, the Hub's stored element count is not the model's parameter count. gpt-oss stores its MoE weights in MXFP4 inside U8 tensors, so the Hub metadata reports a number that differs from the 21B and 117B the publisher states. Use the publisher's card for parameter counts in that case.

Context length

The max_position_embeddings field in config.json is the starting point, but read it with the card:

• Mistral 7B v0.1's config says 32,768 but also sets sliding_window to 4096, so attention looks back over a 4,096 token window per layer.
• Qwen2.5 7B Instruct's card advertises a full context of 131,072 tokens, while noting that the shipped config.json is set for 32,768 and that longer inputs need YaRN rope scaling added to the config.
• Hermes 3 8B's config says 131,072, with rope scaling settings recording an original 8,192 position base and a factor of 8, inherited from its Llama 3.1 base.

Longer context costs memory through the KV cache, which we quantify in the quantisation section.

Example families by licence, size and format

The table below is in alphabetical order by family and is not a ranking. Every value comes from the publisher's Hub card, licence or configuration file as fetched on 23 October 2026.

A small provenance observation from the same data: the Hermes 3 8B card's licence tag reads llama3 while its declared base is Llama 3.1, which has its own licence text. Metadata tags are a convenience and can be imprecise. The licence file and the base model's licence are what bind you.

What licence types apply to model weights?

Permissive software licences: Apache 2.0 and MIT

Several publishers simply apply a standard OSI approved software licence to the weights. On the Hub today that includes Mistral 7B and Mixtral 8x7B, Qwen2.5 7B Instruct, OLMo 2 and gpt-oss (Apache 2.0), and DeepSeek R1 (MIT). OpenAI's gpt-oss card describes its choice as a "permissive Apache 2.0 license" suitable for "commercial deployment". DeepSeek's R1 card states that the series supports commercial use and allows modifications and derivative works, "including, but not limited to, distillation".

Apache 2.0 adds an explicit patent licence and a requirement to state changes in modified files; MIT is shorter and asks for the copyright and permission notice to be kept. Both permit commercial use, modification and redistribution without user caps.

Remember the distill caveat above: an MIT tag on a fine-tune does not erase the obligations of the base it was trained from.

Custom community licences

Other publishers write a licence for the family. The Llama 3.1 Community License (release date 23 July 2024 in its text) grants broad rights to use, reproduce, distribute and modify, with conditions:

• Distribution. Include the agreement and "prominently display 'Built with Llama'". If you use the materials or their outputs to create or improve an AI model you distribute, include "Llama" at the beginning of its name. Keep the attribution notice in a NOTICE file.
• Acceptable use. Use must follow the Llama 3.1 Acceptable Use Policy, incorporated by reference.
• Scale. If your products and your affiliates' had more than 700 million monthly active users in the month before the release date, you must request a separate licence from Meta.

The Qwen licence on Qwen2.5 72B Instruct has the same shape with a lower threshold: commercial users whose product has more than 100 million monthly active users must request a licence.

The Gemma Terms of Use (last modified 1 April 2026 when read) require redistributors to include the section 3.2 use restrictions as an enforceable provision in their own terms, pass on the agreement, mark modified files and ship a NOTICE file. Use must not breach the Gemma Prohibited Use Policy. The publisher also "reserves the right to restrict (remotely or otherwise)" usage it reasonably believes violates the agreement, and claims no rights in outputs.

Research and non-commercial licences

Some checkpoints are not licensed for commercial use at all. The licence on Qwen2.5 3B Instruct defines "Non-Commercial" as "for research or evaluation purposes only" and says that if you are using the materials commercially, you must request a licence. That sits in the same family, under the same family name, as the Apache 2.0 7B checkpoint. Size by size licence checks are not optional.

OpenRAIL and use-based restrictions

Responsible AI Licences (RAIL) pair broad permissions with a list of use-based restrictions that must travel with every derivative. BLOOM is published under the BigScience RAIL License v1.0; its card points to Attachment A of the licence for detailed usage restrictions.

DeepSeek V3's model licence (the LICENSE-MODEL file, separate from the MIT licence on the code) follows the same pattern. It lets you host the model for third party remote access and distribute derivatives, but says the use-based restrictions in Attachment A "MUST be included as an enforceable provision" in any agreement governing use or distribution of the model or its derivatives, and it reserves DeepSeek's right to restrict usage "remotely or otherwise" in violation of the licence. Our explainer on OpenRAIL licences and commercial use covers the family of RAIL variants.

Gated access is not a licence, but it is a signal

Hugging Face lets authors gate a repository so that users must submit an access request, which shares their Hugging Face profile information with the author, before downloading files. Approval can be automatic or manual. The Llama 3.1 and Gemma 2 checkpoints shown above are gated with manual approval. Gating changes how your build pulls the files (you need an authenticated token with access), and it records who accepted the terms. It does not change the licence.

Licence comparison

The last column summarises the Commission's guidelines, discussed in the EU section below; it is not legal advice on any specific licence.

What are the file formats, and why does the format matter for safety?

A downloaded model is a directory, not a single file: weight shards, an index, config.json, tokenizer files, a generation config, often a chat template, and sometimes Python files. Each one is an input your runtime trusts. Our model file extension risk table lists the wider set; here we go deep on the three you meet most often.

PyTorch pickle files (.bin, .pt, .pth)

PyTorch's traditional checkpoint format stores tensors inside a Python pickle. Python's documentation warns that the pickle module "is not secure" and that malicious pickle data can execute arbitrary code during unpickling. Hugging Face's security documentation describes the same risk for pickled models and suggests trusted publishers, signed commits and safer formats.

PyTorch's mitigation is weights_only in torch.load, which restricts the unpickler to tensors and an allowlist; current documentation shows weights_only=True as the default and warns against setting it to False for untrusted data. The Hub also runs a pickle import scan that lists the imports a pickled file references. Both are defences, not guarantees.

Many repositories ship both formats. Mistral 7B v0.1 holds two pytorch_model-*.bin shards and two model-*.safetensors shards of the same weights. You choose which your pipeline downloads.

Safetensors

Safetensors exists, in its README's words, "to remove the need to use pickle on PyTorch". The layout is:

1. 8 bytes: an unsigned little endian 64 bit integer N, the header size.
2. N bytes: a UTF-8 JSON header starting with {, mapping each tensor name to dtype, shape and data_offsets, plus an optional __metadata__ map of strings only.
3. The raw tensor bytes.

Duplicate keys are disallowed (the README notes not every parser enforces this), and the byte buffer must be fully indexed with no holes, which "prevents the creation of polyglot files". Tensor values are not checked, so NaN and infinity can appear. Because the header is plain JSON at a fixed offset, you can inspect a file without loading it, as the walkthrough shows.

GGUF

GGUF is the single file format of GGML and llama.cpp runtimes, successor to the GGML, GGMF and GGJT layouts. Its header holds the ASCII magic GGUF, a version that must be 3 for the current specification, a tensor count and a metadata count, then typed key and value pairs, tensor information and aligned data (32 bytes if general.alignment is absent).

For provenance the metadata matters. The specification defines optional keys including general.license (an SPDX expression), general.repo_url, general.uuid, a general.source.* group for the model a file was converted from, and general.base_model.N.* keys for parents. When a converter fills them in, the file carries its lineage. When it does not, record the source yourself.

The GGUF naming convention, <BaseName><SizeLabel><FineTune><Version><Encoding><Type><Shard>.gguf, is in the specification's words "not intended to be perfectly parsable". Do not treat a filename as the source of truth.

GGUF can also hold tokenizer.chat_template. llama.cpp's server documentation says its chat template defaults to the one "taken from model's metadata" with the Jinja engine enabled by default, and offers --chat-template-file to supply your own. Review the embedded template or override it.

Remote code: the format around the format

Hugging Face Transformers can load architectures that are not built into the library by downloading Python files from the model repository. The Transformers documentation says that because a custom model does not use the library's own modeling code, you need to pass trust_remote_code=True to from_pretrained() to load it. That flag tells your process to import and run Python from the repository. A safetensors weight file in a repository that also needs remote code is not a code free download.

Format comparison

What do quantisation levels change for size and quality?

Bits per weight is the number to think in

Quantisation stores weights with fewer bits, usually in blocks that each carry a scale, so effective bits per weight sit slightly above the nominal width. Hugging Face's GGUF documentation gives these figures:

Weight memory is parameters x bits per weight / 8 bytes. GB here means 10^9 bytes.

Worked example 1: a 7 billion parameter dense model

Mistral 7B v0.1 has 7,241,732,096 parameters according to the Hub's safetensors metadata.

• At 16 bits: 7,241,732,096 x 2 bytes = 14,483,464,192 bytes, about 14.48 GB. The two safetensors shards on the Hub are 9,942,981,696 and 4,540,516,344 bytes, a total of 14,483,498,040 bytes. The 34 KB difference is the JSON headers. If a "16 bit" file is far off this number, it is not what its name says.
• At Q8_0 (about 8.5 bits): about 7.69 GB.
• At Q6_K (6.5625 bits): about 5.94 GB.
• At Q4_K (4.5 bits): about 4.07 GB.
• At Q2_K (2.625 bits): about 2.38 GB.

Now add the KV cache: 2 (keys and values) x layers x KV heads x head dimension x tokens x bytes per value. Mistral 7B's config gives 32 layers, 32 attention heads, 8 KV heads and a hidden size of 4,096, so the head dimension is 128. At 16 bit cache values:

• 4,096 tokens: 2 x 32 x 8 x 128 x 4,096 x 2 = 536,870,912 bytes, about 0.54 GB.
• 32,768 tokens: 2 x 32 x 8 x 128 x 32,768 x 2 = 4,294,967,296 bytes, about 4.29 GB.

A runtime with a rolling cache for the 4,096 token sliding window needs only the first figure; without one, a Q4_K copy at full 32K context needs roughly 4.07 + 4.29 = 8.4 GB before overhead. The cache can outweigh the weights.

Worked example 2: a 72 billion parameter dense model

Qwen2.5 72B Instruct has 72,706,203,648 parameters on the Hub. Its config gives 80 layers, 64 attention heads, 8 KV heads and a hidden size of 8,192, so the head dimension is 128.

• At 16 bits: about 145.4 GB. That is more than one 80 GB accelerator.
• At Q8_0: about 77.3 GB.
• At Q4_K: about 40.9 GB.
• At IQ4_XS: about 38.6 GB.
• KV cache at 32,768 tokens and 16 bit values: 2 x 80 x 8 x 128 x 32,768 x 2 = about 10.7 GB.

So a 4 bit copy at full configured context is about 51.6 GB plus overhead. Remember also that this checkpoint's licence has a 100 million MAU clause, which does not change with quantisation. A quantised copy is a derivative; it keeps the base model's licence obligations.

Worked example 3: mixture of experts memory

gpt-oss-20b has 21B total parameters and 3.6B active per token according to OpenAI's card. If every weight were stored at 16 bits, the weights alone would need about 21 x 10^9 x 2 = 42 GB. OpenAI's card states that the MoE weights were post-trained with MXFP4 quantisation, which lets the 20b model "run within 16GB of memory" and the 120b model fit a single 80 GB GPU. The active parameter count drives speed; the total parameter count, at whatever precision each part is stored, drives memory.

What quantisation changes for quality and for provenance

Fewer bits means more rounding error, and loss grows as bits fall. IQ types use an importance matrix computed from sample text, so two files of the same type can differ by calibration data. Test a quantised file on your own evaluation set; we quote no quality figures here.

For provenance, quantisation produces new bytes, so the publisher's hashes no longer apply. A third party GGUF is a different artefact from a different supplier, whatever its name says. Quantise from verified originals and record the step, or track the converter as its own supplier.

How do you verify provenance?

Provenance for a model is the same question as for any software artefact: where did these exact bytes come from, who vouches for them, and can you show it later?

Layer 1: pin a commit, not a branch

Every Hugging Face repository is a Git repository. The model API returns the current commit as sha; for Mistral 7B v0.1 on 23 October 2026 that was 27d67f1b5f57dc0953326b2601d68371d40ea8da. The main branch can move; a commit cannot. Pin the commit in your download command and in your code (revision= in Transformers and huggingface_hub, --revision on the hf CLI).

Layer 2: per file hashes

Large files on the Hub are stored with Git LFS, and the API's tree listing returns each LFS file's SHA-256 as its oid. For Mistral 7B v0.1, the first safetensors shard's SHA-256 is 9742cb4764964155b7a5f35eefad651f590006091ddeb536863d6c5865cca1b9. Record those hashes at the time you approve the model, and verify the files you deploy against your record, not against whatever the API says on deploy day.

Layer 3: signatures with OpenSSF model signing

Hashes show bytes have not changed since you recorded them; a signature shows who vouched for them. The OpenSSF model-transparency project under Sigstore ships the model-signing library and CLI (version 1.0.0 reached PyPI on 4 April 2025; 1.1.1 is current). Its README says:

• model_signing sign signs a model directory, ignoring Git paths by default. The default method is Sigstore keyless signing, recorded in Sigstore's append-only transparency log; key, certificate and PKCS #11 methods are also supported.
• The output is a Sigstore bundle wrapping a DSSE envelope with an in-toto statement whose subjects are (file path, digest) pairs and whose predicate type is https://model_signing/signature/v1.0.
• model_signing verify checks the signature, then recomputes every file hash against the signed list.

Are signed model cards a thing yet?

Partly. The model signing README says the statement's predicates are intended to store, and therefore sign, model card information "in the future". Today the signature covers the files you include, and if the README is in the directory you sign, its hash is in the signed list. That signs the card's bytes; it does not make the card's claims true. Hugging Face's security documentation separately recommends relying on signed Git commits, which tells you which account pushed a change but not that the account is the publisher you expect.

Layer 4: an AIBOM entry

The last layer is a record other tools can read.

CycloneDX ML-BOM. CycloneDX has a machine-learning-model component type and a modelCard object (present in the 1.5, 1.6 and 1.7 schemas). modelCard.modelParameters covers approach, task, architectureFamily, modelArchitecture, datasets, inputs and outputs; modelCard.considerations covers users, use cases, limitations, trade-offs and ethical, environmental and fairness considerations. As an ordinary component, the model also gets hashes, licenses, externalReferences and pedigree.

SPDX 3.0.1 AI profile. An AIPackage is a subclass of the Software Package, with properties such as autonomyType, domain, energyConsumption, hyperparameter, informationAboutTraining, limitation, metric, modelDataPreprocessing, safetyRiskAssessment and standardCompliance. The profile requires exactly one hasConcludedLicense and exactly one hasDeclaredLicense relationship per AIPackage, which suits weights: the declared licence (the card tag) and the concluded one (after you read the base model's licence) can differ.

For field guidance see our SPDX 3.0 AI profile guide, what to record for each model and the AIBOM explainer.

The NIST view

NIST AI 600-1, the Generative AI Profile of the AI Risk Management Framework (July 2024), lists "Value Chain and Component Integration" among the risks unique to or exacerbated by generative AI: "Non-transparent or untraceable integration of upstream third-party components", improper supplier vetting across the AI lifecycle, and other issues that reduce transparency or accountability for downstream users. Pinning, hashing, signing and recording a model is how you turn that risk into a list of checkable facts.

Step by step: adopt an open-weight model with a provenance trail

The walkthrough uses Mistral 7B v0.1 because it is ungated, Apache 2.0 licensed and ships both pickle and safetensors files, which makes the choices visible. Replace the repository and commit with your own. Commands assume a Unix shell with curl, jq, python3 and shasum.

Step 1: read the card, licence and current commit from the API

REPO="mistralai/Mistral-7B-v0.1"
curl -s "https://huggingface.co/api/models/${REPO}" \
  | jq '{sha, gated, license: .cardData.license, base_model: .cardData.base_model,
         params: .safetensors.total}'

Then open the repository's LICENSE file (or the licence the card links) and read it. The tag is a summary; the file binds you. For a fine-tune, repeat this for every model in its base_model chain.

Step 2: list files and record their hashes at a pinned commit

SHA="27d67f1b5f57dc0953326b2601d68371d40ea8da"
curl -s "https://huggingface.co/api/models/${REPO}/tree/${SHA}" \
  | jq -r '.[] | select(.lfs != null) | "\(.lfs.oid)  \(.path)"' \
  | grep -E '\.safetensors$' > approved-hashes.txt
cat approved-hashes.txt

Keep approved-hashes.txt with your approval record. It is the reference you verify against later.

Step 3: download only the formats you intend to load

pip install -U "huggingface_hub[cli]"
hf download "$REPO" --revision "$SHA" \
  --include "*.safetensors" "*.json" "tokenizer.model" \
  --local-dir ./models/mistral-7b-v0.1

The --include patterns keep the .bin pickle shards off your disk entirely. For a gated model, run hf auth login with a token that has been granted access first.

Step 4: verify the hashes

cd ./models/mistral-7b-v0.1
shasum -a 256 -c ../../approved-hashes.txt

Every line should end in OK. A mismatch means the file is not the one you approved; stop and investigate before loading anything.

Step 5: inspect the safetensors header without loading tensors

This reads the 8 byte length and the JSON header only. It never deserialises tensor data and imports nothing from the model directory.

# inspect_safetensors.py  (run with: python3 -I inspect_safetensors.py FILE)
import json, struct, sys

MAX_HEADER = 100 * 1024 * 1024  # our own sanity cap, not part of the spec

with open(sys.argv[1], "rb") as f:
    (n,) = struct.unpack("<Q", f.read(8))
    if n > MAX_HEADER:
        sys.exit(f"header length {n} exceeds sanity cap")
    header = json.loads(f.read(n))

meta = header.pop("__metadata__", {})
dtypes = {}
for name, t in header.items():
    dtypes[t["dtype"]] = dtypes.get(t["dtype"], 0) + 1
print("metadata:", meta)
print("tensors:", len(header), "dtypes:", dtypes)

Check that the dtypes match what the card claims and that the tensor count is plausible for the architecture.

Step 6: inspect a GGUF header the same way

If you also use quantised GGUF files, read the fixed header first:

# inspect_gguf.py  (run with: python3 -I inspect_gguf.py FILE)
import struct, sys

with open(sys.argv[1], "rb") as f:
    magic = f.read(4)
    version, = struct.unpack("<I", f.read(4))
    tensors, kvs = struct.unpack("<QQ", f.read(16))
print("magic:", magic, "version:", version, "tensors:", tensors, "metadata keys:", kvs)
if magic != b"GGUF":
    sys.exit("not a GGUF file")

To read the metadata keys (licence, base model, chat template), use the gguf Python package maintained in the llama.cpp repository, and look specifically at general.license, general.source.repo_url, general.base_model.* and tokenizer.chat_template.

Step 7: sign the approved directory with your own key

Signing internally means every later deployment can verify against your approval, independent of the upstream repository.

pip install model-signing
openssl ecparam -name prime256v1 -genkey -noout -out model-signing.priv
openssl ec -in model-signing.priv -pubout > model-signing.pub

model_signing sign key ./models/mistral-7b-v0.1 \
  --private-key model-signing.priv \
  --signature ./signatures/mistral-7b-v0.1.sig

model_signing verify key ./models/mistral-7b-v0.1 \
  --signature ./signatures/mistral-7b-v0.1.sig \
  --public-key model-signing.pub

Store the private key the way you store any release signing key, ideally on a hardware token (the library supports PKCS #11 URIs) or use Sigstore keyless signing from your CI identity. Put the verify step in the deployment job, not only on a laptop.

Step 8: record it in your AIBOM

A minimal CycloneDX component for the approved model looks like this. Hashes are shortened here for layout; record the full values from step 2.

{
  "bomFormat": "CycloneDX",
  "specVersion": "1.6",
  "components": [
    {
      "type": "machine-learning-model",
      "bom-ref": "model:mistralai/Mistral-7B-v0.1@27d67f1b",
      "group": "mistralai",
      "name": "Mistral-7B-v0.1",
      "version": "27d67f1b5f57dc0953326b2601d68371d40ea8da",
      "licenses": [ { "license": { "id": "Apache-2.0" } } ],
      "hashes": [
        { "alg": "SHA-256", "content": "9742cb47...cca1b9" },
        { "alg": "SHA-256", "content": "9bcf5635...01cb" }
      ],
      "externalReferences": [
        { "type": "distribution", "url": "https://huggingface.co/mistralai/Mistral-7B-v0.1" }
      ],
      "modelCard": {
        "modelParameters": {
          "task": "text-generation",
          "architectureFamily": "mistral",
          "modelArchitecture": "MistralForCausalLM"
        }
      }
    }
  ]
}

For a fine-tune, add the base model as its own component and link it through pedigree.ancestors, so the licence chain is visible to anyone reading the BOM. The AIBOM generation guide shows open source tools that produce these entries for you.

How do you run an open-weight model locally with safe defaults?

Bind to localhost

Both of the most common local servers bind to loopback by default, and both document how to change it. Leave it alone unless you have a reason.

• llama.cpp server. The --host option's documented default is 127.0.0.1. The server supports --api-key and --api-key-file for authentication, with no key set by default.
• Ollama. Its FAQ states that it "binds 127.0.0.1 port 11434 by default" and that the bind address is changed with OLLAMA_HOST. It allows cross-origin requests from 127.0.0.1 and 0.0.0.0 by default, with more origins configurable through OLLAMA_ORIGINS.

# llama.cpp: loopback only, with an API key, and a chat template you control
llama-server -m ./models/mistral-7b-v0.1-Q4_K.gguf \
  --host 127.0.0.1 \
  --api-key-file ./secrets/llama-keys.txt \
  --chat-template-file ./templates/reviewed.jinja

# Ollama: confirm the bind address before you start it
echo "${OLLAMA_HOST:-127.0.0.1:11434 (default)}"
ollama serve

Containers change the meaning of "localhost"

Inside a container, a server must listen on 0.0.0.0 to be reachable through a published port, and the llama.cpp README's Docker example does exactly that with -p 8080:8080. On the host, -p 8080:8080 publishes on every interface. Publish to loopback instead:

docker run --rm -p 127.0.0.1:8080:8080 \
  -v "$PWD/models:/models:ro" \
  ghcr.io/ggml-org/llama.cpp:server \
  -m /models/mistral-7b-v0.1-Q4_K.gguf --host 0.0.0.0 --port 8080

Mount the model directory read only, pin the image by digest rather than tag in production, and add --api-key-file if anything other than you will call it.

No remote code, safetensors only, pinned revision

For Transformers, make the safe choices explicit rather than relying on defaults that may change:

from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "mistralai/Mistral-7B-v0.1"
SHA = "27d67f1b5f57dc0953326b2601d68371d40ea8da"

tok = AutoTokenizer.from_pretrained(REPO, revision=SHA, trust_remote_code=False)
model = AutoModelForCausalLM.from_pretrained(
    REPO,
    revision=SHA,
    use_safetensors=True,      # refuse to fall back to pickle shards
    trust_remote_code=False,   # never import Python from the repo
)

Once the files are in the local cache and verified, set HF_HUB_OFFLINE=1 in the serving environment so nothing is fetched at start-up. If a model genuinely needs remote code, read the Python files at the pinned commit, vendor them into your own repository and review changes like any other dependency.

Treat the runtime as a dependency too

The weights are data. The runtime that parses them (llama.cpp, Ollama, Transformers, PyTorch, the tokenizer library, the Jinja engine) is software with its own release history and its own vulnerabilities. Pin those versions, scan them and upgrade them on the same schedule as the rest of your stack. A model directory you verified last month is only as safe as the parser that opens it today.

What do the EU AI Act general purpose AI obligations say about open models?

Timeline

What providers must do

Article 53 sets four core duties for providers of general purpose AI models: (a) technical documentation for the AI Office and national authorities, (b) documentation for downstream integrators, (c) a policy to comply with EU copyright law, and (d) a public summary of training content. Article 54 requires providers outside the EU to appoint an authorised representative.

The Commission's guidelines give an indicative criterion for a general purpose AI model: training compute above 10^23 FLOP and the ability to generate language, images or video. Above 10^25 FLOP a model is presumed to have high impact capabilities and becomes a model with systemic risk under Article 51, with extra Article 55 duties.

The open source exception and its conditions

Articles 53(2) and 54(6) exempt providers of models released under a free and open source licence allowing "access, usage, modification, and distribution", with weights, architecture information and usage information public, from duties (a), (b) and the authorised representative, unless the model has systemic risk. The guidelines then set the conditions:

• Still applies: the copyright policy and the training content summary, because an open licence "does not necessarily reveal substantial information on the data used for training".
• All four rights are required. Access must be free of payment (non-discriminatory user verification is acceptable). Use may be limited only by attribution and same-terms conditions. Recipients must be free to modify and redistribute, including under different terms.
• Disqualifying restrictions include non-commercial or research only use, distribution bans, "usage restrictions triggered by user scale thresholds", and separate commercial licences for specific use cases.
• Allowed restrictions: proportionate, safety oriented terms based on objective, non-discriminatory criteria.
• No monetisation. Paid commercial tiers, mandatory paid support, access only through a paid or advertising funded platform, or access conditioned on collecting personal data (other than strictly for security) remove the exception. Optional paid services alongside a free model do not.

Against the licences above, Apache 2.0 and MIT meet the licence condition on their face; MAU thresholds and research only terms do not; RAIL style and custom use restrictions need a case by case look. The licence is one condition among several.

When a fine-tuner becomes a provider

The guidelines' indicative criterion is that a downstream modifier becomes the provider of the modified model if the modification used more than one third of the original model's training compute. If that figure is unknown and cannot be estimated, the threshold becomes one third of 10^25 FLOP for a systemic risk original, or one third of 10^23 FLOP otherwise. Most small fine-tunes fall far below this; record the compute you used so you can show it.

Where the Code of Practice fits

The Commission published the General-Purpose AI Code of Practice on 10 July 2025 and calls it an adequate voluntary tool for showing compliance. Its transparency and copyright chapters apply to all providers; its safety and security chapter applies only to systemic risk models.

Decision table: picking a licence and format combination

Common mistakes

• Reading the tag, not the file. The Hub licence tag is typed metadata. The LICENSE file and the base model's licence bind you.
• Assuming one licence per family. Qwen2.5 has three licences across three instruct sizes.
• Forgetting inherited obligations. An MIT or Apache tag on a fine-tune does not remove a Llama, Gemma or RAIL base's naming rules or pass-through restrictions.
• Loading pickle when safetensors is right there. Exclude *.bin at download and pass use_safetensors=True.
• Enabling trust_remote_code to silence an error. It runs Python from the repository.
• Pinning main. Pin a commit and record file hashes.
• Treating a third party quantisation as the publisher's file. New bytes, new supplier.
• Sizing for weights only, or MoE by active parameters. The KV cache can exceed a small model's weights, and every expert must be in memory.
• Publishing a container port on all interfaces. Use -p 127.0.0.1:8080:8080.
• Assuming "open" means exempt under the EU AI Act. Thresholds, research only clauses and monetisation disqualify, and the copyright and training summary duties apply regardless.

FAQ

What does open-weight mean?

It means the trained parameters of a model are published for download, so you can run the model on your own hardware. It does not by itself tell you the licence, whether training code is released, or what data was used. Those are separate questions answered by the licence file, the model card and any dataset documentation.

Is an open-weight model the same as an open source model?

Not under the OSI's definition. Open Source AI Definition 1.0 requires the parameters under OSI approved terms, the complete training and inference code under OSI approved licences, and detailed data information sufficient for a skilled person to build a substantially equivalent system. Many open-weight releases provide only the first.

Can I use Llama models commercially?

The Llama 3.1 Community License allows commercial use with conditions: include the licence, display "Built with Llama", start the name of any distributed model trained with Llama materials or outputs with "Llama", follow the acceptable use policy, and request a separate licence from Meta if your products had more than 700 million monthly active users in the month before the release date. Read the licence for the specific Llama version you use.

Are safetensors files safe to load?

The format is designed so loading does not execute code: an 8 byte length, a JSON header and raw bytes. That removes pickle's main risk. It says nothing about model behaviour, and the repository can still hold Python that runs if you enable remote code. Keep the parser and runtime patched.

How much memory do I need to run a 7B model?

For weights alone, multiply parameters by bits per weight and divide by 8. A 7.24 billion parameter model needs about 14.5 GB at 16 bits, about 7.7 GB at 8 bits and about 4.1 GB at Q4_K's 4.5 bits per weight. Then add the KV cache: about 0.54 GB at 4,096 tokens and about 4.3 GB at 32,768 tokens for Mistral 7B's configuration with 16 bit cache values.

How do I verify a model downloaded from Hugging Face?

Pin a commit, take each large file's SHA-256 from the tree API at that commit, download only the formats you will load and check them with shasum -a 256 -c. For more assurance, sign the approved directory with OpenSSF model-signing and verify in your deployment pipeline. Record model, commit, hashes, licence and base chain in your AIBOM.

Does quantising a model change its licence?

No. A quantised copy is a derivative of the original weights, so the original licence's conditions still apply, including attribution, naming, use restrictions and user thresholds. Quantisation does change the bytes, so the original file hashes no longer apply and the quantised file needs its own provenance record.

Are open-weight models exempt from the EU AI Act?

Partly, and only sometimes. Providers whose licence freely allows access, use, modification and distribution, without monetisation and with weights and architecture and usage information public, skip the technical documentation, downstream documentation and authorised representative duties. They still need a copyright policy and a training content summary, and models with systemic risk get no exemption.

Where Safeguard fits

Safeguard's fit here is real but bounded. Weights are data files, and choosing a licence, quantising and signing them are steps this post walks through by hand. Safeguard does not certify licences, rank models or vouch for model behaviour. It covers the software around the model: the services, inference servers, container images and lockfiles that load and serve weights. safeguard scan --dir . scans a repository's dependencies, --image scans a container image such as the one serving llama.cpp, and --sbom takes an existing SBOM, with JSON results under ./output. Reachability analysis ranks the vulnerable paths your code actually calls, and platform policies and gates can stop a build that pulls a known vulnerable runtime.

For the model side, AI-BOM discovery is in early access. It inventories AI assets, such as model SDK imports in code and managed AI services, and produces CycloneDX ML-BOM and SPDX AI profile records. Ask which sources it covers before you plan around it.

Software supply chain hygiene still decides most of the risk in this stack. The tokenizer library, inference runtime, Jinja engine and container base image are ordinary open source dependencies with ordinary vulnerabilities, and every rebuild or runtime upgrade pulls them in again.

Fix it without waiting for upstream

Whatever the vulnerability, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.

1. Autonomous remediation: pull requests, not tickets

When a fixed version exists, Safeguard's Griffin models open a tested, policy gated pull request that moves the affected dependency to the safe version. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation covers eight categories, including dependencies, licenses, security posture and findings. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Autonomous remediation ships switched off, with every category set to Manual, so your team decides how much runs on its own and what still needs a human approval.

2. Gold Open Source: the safe version in seconds, free

gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds. The safeguard-sh/gold-scan-action GitHub Action can fail a build on a known exploited vulnerability, on any known vulnerability, or never, so a version you have just removed cannot quietly come back in next week's merge.

3. Gold Registry: start from components that are already fixed

registry.safeguard.sh provides hardened, drop in replacements for upstream packages and container images. Every Gold artifact is held continuously to zero exploitable critical or high CVEs, zero malware and a completed license check, with SLSA L3 or higher provenance, a Sigstore signature and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.

When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened, compatibility tested build of a specific package. This fork and patch flow is in early access today, so ask which packages it covers before you plan around it.

One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.

See how Safeguard would help

The demos show Safeguard on ordinary software projects, not on model weights. The same flows apply to the repositories that serve your models: the Python service, its pinned runtime libraries and the inference server image.

The first demo is a real Safeguard CLI session. After install, safeguard opens the interactive terminal, you ask it to fix the lodash CVE in a repository, and it reads the finding and advisory, asks before each command, edits package.json to lodash 4.17.21, runs npm install and npm test with your approval, prints a summary and pushes nothing. It ends with /permissions, /model and /scan. Against an inference service repository, the upgrade would be a Python runtime library instead of lodash, with your tests run before anything is committed.

Click through it at your own pace, or open it full screen.

Many teams run open-weight models on a workstation or an isolated GPU host precisely so that prompts and data stay local. The local runner demo shows how to set up the runner on your own machine, approve it with its device code, and then scan from the terminal. A runner on that same host can scan the serving code and images that live there without moving them anywhere else.

Click through it at your own pace, or open it full screen.

If your engineers already use Claude, the third demo shows how to connect Claude to the Safeguard MCP server with OAuth and choose which tools Claude can use. An engineer can then ask about vulnerabilities in the projects that host your model serving code, using only the tools you have allowed.

Click through it at your own pace, or open it full screen.

To try it on your own repositories, download Safeguard and follow the CLI documentation.

Quick check

Sources

• Open Source Initiative, The Open Source AI Definition 1.0
• Open Source Initiative, announcement of the Open Source AI Definition 1.0 (28 October 2024)
• Hugging Face, safetensors README and format specification
• Hugging Face, safetensors documentation
• GGML project, GGUF specification
• Hugging Face, GGUF on the Hub (quantisation types)
• Python documentation, pickle module (security warning)
• PyTorch documentation, torch.load
• Hugging Face, Pickle scanning
• Hugging Face Transformers, Customizing models (trust_remote_code)
• Hugging Face, Gated models
• Hugging Face, Licenses on the Hub
• Hugging Face, Command line interface (hf download)
• OpenSSF and Sigstore, model-transparency README (model signing)
• PyPI, model-signing release history
• CycloneDX, Machine Learning Bill of Materials (ML-BOM)
• CycloneDX, bom-1.6 JSON schema
• CycloneDX, bom-1.7 JSON schema
• SPDX 3.0.1, AI profile
• SPDX 3.0.1, AIPackage class
• European Commission, Guidelines on the scope of obligations for providers of general-purpose AI models (18 July 2025)
• European Commission, Guidelines text (English PDF)
• European Commission, Overview of the guidelines for GPAI providers
• European Commission, The General-Purpose AI Code of Practice
• NIST AI 600-1, Generative Artificial Intelligence Profile (July 2024)
• llama.cpp, server README
• Ollama, FAQ
• Meta, Llama 3.1 Community License
• Gemma Terms of Use
• Alibaba Cloud Qwen, Qwen2.5-72B-Instruct LICENSE
• Alibaba Cloud Qwen, Qwen2.5-3B-Instruct LICENSE
• Alibaba Cloud Qwen, Qwen2.5-7B-Instruct model card
• DeepSeek, DeepSeek-V3 LICENSE-MODEL
• DeepSeek, DeepSeek-R1 model card
• OpenAI, gpt-oss-20b model card
• Mistral AI, Mixtral-8x7B-v0.1 model card
• Mistral AI, Mistral-7B-v0.1 repository files
• Ai2, OLMo-2-1124-7B model card
• BigScience, BLOOM model card
• Nous Research, Hermes-3-Llama-3.1-8B model card
• Hugging Face Hub API, model metadata endpoint for Mistral-7B-v0.1 (parameter counts, commit and file hashes)

Last reviewed: 23 October 2026. Model licences, cards and Hub metadata change; check the linked licence file for the exact checkpoint and commit you use.