Run the container as a non-root user, mount its root filesystem read-only, drop Linux capabilities, set no-new-privileges, limit memory and processes, and avoid privileged mode. Each is a documented docker run option. Model servers need a writable scratch area and GPU access, so plan for those.
Why inference containers deserve this
An inference server parses large untrusted inputs, loads big binary files and often runs with access to expensive hardware. If a bug in the server or a loaded file gives an attacker code execution, the container settings decide how far they get. Hardening flags reduce what a compromised process can touch.
These flags are standard in Docker's command reference. None of them is specific to AI, which is the point: a model server is a normal service and gets normal hardening.
The flags, one by one
| Flag | What the reference says it does | Inference note |
|---|---|---|
| user | Sets the user the container runs as, by name or UID | Pick a non-root UID that owns the model directory |
| read-only | Mounts the container root filesystem as read only | Add writable mounts for cache and temp |
| tmpfs | Mounts a temporary filesystem at a path | Good for temp files and shared memory style needs |
| cap-drop | Drops Linux capabilities | Drop all, add back only what you can name |
| security-opt no-new-privileges | Stops processes gaining new privileges | The reference notes tools like su and sudo then stop working |
| memory | Sets a memory limit | Prevents one container taking the host |
| pids-limit | Limits the number of processes | Caps fork based abuse |
| privileged | Gives extended privileges | Avoid it |
A starting command
docker run --rm \
--user 10001:10001 \
--read-only \
--tmpfs /tmp \
--cap-drop ALL \
--security-opt no-new-privileges=true \
--memory 48g --pids-limit 512 \
-v /srv/models:/models:ro \
-p 127.0.0.1:8000:8000 \
your-inference-image
Read the choices. The model directory is mounted read-only, so a compromised process cannot rewrite weights. The port is published on localhost, so a proxy on the host decides who gets in. Memory and process limits cap damage from abuse. Adjust sizes to your hardware.
What will break, and how to fix it
- Write errors on startup. Servers write caches and temp files. Find the paths in the error, then give those paths a tmpfs or a dedicated writable volume. Do not remove the read-only flag.
- Permission errors on the model directory. Make the directory readable by the UID you chose.
- GPU access. GPU runtimes need device access. Grant what your runtime documents, test it, and do not reach for privileged mode as a shortcut.
- A binary that needs a capability. Add back the specific capability, record why, and review it.
Build the image for it too
Hardening at run time works better with a lean image. Use a pinned base, install only what the server needs, and set a non-root user in the Dockerfile so the default is safe. Scan the image for vulnerable packages before it runs. A separate post on this blog covers scanning a GPU inference image.
Try it yourself
Click through it at your own pace, or open it full screen. Look at how image findings are grouped, then decide which a run time flag cannot fix.
Quick check
Does a read-only root filesystem stop all attacks?
No. It limits persistence and tampering inside the container. A bug can still leak data or abuse memory, so pair it with network and image controls.
Can I just use privileged mode for the GPU?
Avoid it. Grant the specific device access your runtime documents, because privileged mode removes most of the isolation you are trying to add.
Where Safeguard fits
Safeguard scans container images and dependencies and applies policies and gates in CI, so an image with known vulnerable packages can be stopped before it reaches a cluster. A free scan is available.
Sources
- docker container run reference, read 19 October 2026.
Last reviewed: 19 October 2026.
