Treat an offline model transfer as a software import. Download on a connected staging machine, choose safe formats, record hashes and licenses, scan the runtime and its dependencies, carry everything on reviewed media, and verify the hashes again on arrival. Local inference needs no network once the files are in place.
What you need to carry
An offline machine needs three groups of files:
- Model weights. Prefer GGUF or safetensors. See model file formats.
- A runtime. For example llama.cpp binaries or an Ollama installer, matching your operating system and hardware backend.
- Dependencies. Python packages, drivers and libraries the runtime needs, if any. Native runtimes carry fewer than a Python serving stack, but check.
llama.cpp lists CUDA, Metal, Vulkan and CPU backends, so confirm you take the build that matches the target hardware.
Checklist on the staging machine
- Download from the publisher's official source. Check the account name for lookalikes.
- Record the source URL, license text, version and the date.
- Compute and store a SHA-256 hash of each file. Compare with the publisher's value when one is provided.
- Avoid pickle based checkpoints. The Hugging Face docs warn that unpickling can execute code. If you must convert one, do it in a sandbox.
- Scan the runtime and any packages for known vulnerabilities, and write down the versions.
- Check licenses. See open weights vs open source AI.
- Put everything on dedicated media, and scan the media with your usual tools.
Checklist on the offline machine
- Verify hashes of every file before use.
- Install the runtime from the carried package only.
- Keep the model server on loopback. See binding and exposure.
- Check where the runtime keeps models and set it if needed. See where models are stored.
- Turn off any update or telemetry options your runtime offers, and confirm by checking outbound connections. I did not verify per tool telemetry settings, so check each tool's own docs.
- Run a short eval to confirm the files behave as expected. See a small eval.
Updates are the hard part
Offline machines miss security fixes unless you plan for them. Set a schedule to repeat the staging process for runtimes, and subscribe to advisories for the engines you use. Past advisories exist for local inference engines, so patching matters here as much as anywhere.
Records to keep
A simple manifest per machine: file name, hash, source, license, date imported, who approved it. When an advisory lands, you can answer which machines are affected in minutes.
Safeguard's SBOM and dependency scanning can help inventory the packages on your staging machine before they cross the gap, and the free scan at https://gold.safeguard.sh/scan is a quick starting point.
Quick check
Does a local model need internet once it is installed?
Inference itself runs on the machine. Ollama's FAQ says that when run locally it does not see your prompts or data. Downloads, updates and optional cloud features are what need a network.
Is a hash check enough?
It proves the file matches what you recorded. It does not prove the file was safe to begin with, so vet the source and format on the staging side.
Sources
Last reviewed: 19 October 2026.
