Give the server an allowlist of media domains, turn off HTTP redirects for media fetches, and cap download and decode sizes. Without these, a user can submit a URL that makes your inference server call internal services or cloud metadata addresses. vLLM documents each setting in its security page.
Why a URL in a prompt is an outbound request
Multimodal models accept images, video and audio. Many clients pass these as URLs instead of uploading bytes. The inference server then fetches that URL itself, from inside your network.
That is server-side request forgery, usually shortened to SSRF. The attacker does not need network access to your internal services. They ask your server to make the request and read what comes back, or at least cause it to be made.
The vLLM security documentation names two outcomes if domains are not restricted. A user could target internal services, including cloud metadata endpoints such as 169.254.169.254. Or a user could point at very large files or slow endpoints so the server downloads unbounded data and exhausts memory, disk or bandwidth.
The three controls the docs describe
| Control | What it does | Setting named in the docs |
|---|---|---|
| Domain allowlist | Only listed domains can be fetched for media | allowed media domains option |
| Redirect block | Stops a redirect from escaping the allowlist | VLLM_MEDIA_URL_ALLOW_REDIRECTS set to 0 |
| Size limits | Rejects oversized downloads while streaming | VLLM_MAX_MEDIA_DOWNLOAD_SIZE_MB and related variables |
The docs give 256 MB as the default for a single remote media response, and they say oversized responses are rejected while streaming, before the full body sits in memory. They also say the allowlist covers both the online serving API and the batch runner.
A launch line shaped like the docs example looks like this. Replace the domains with your own.
vllm serve MODEL --allowed-media-domains media.example.com cdn.example.org
export VLLM_MEDIA_URL_ALLOW_REDIRECTS=0
Steps to apply this
- List where your media legitimately comes from. Often it is one object store or CDN host. Keep the list short.
- Set the allowlist to exactly those hosts. Do not use a wildcard for a large public domain.
- Disable redirects, since a trusted host that redirects elsewhere can defeat the allowlist.
- Lower the size limits if your workload never needs 256 MB of media in one request.
- Block egress at the network layer too. The server should not be able to reach the metadata address or internal admin ports even if a setting is missed.
- Test with a URL that points at an internal address and confirm the request is refused.
Step five matters because configuration drifts. A network rule is a second fence that does not depend on one flag staying set.
What this does not cover
An allowlist stops the fetch, not what a model does with a permitted image. Prompt injection hidden inside an image is a separate problem, handled by limiting what the model's output can trigger. The docs also note that per-request multimodal arguments are rejected by default on API endpoints, and they advise against the option that restores them on any endpoint exposed to untrusted clients.
If you use a different inference server, look for the same three questions. Can it fetch URLs for me? Can I restrict where? Are sizes capped?
Try it yourself
Click through it at your own pace, or open it full screen. Watch how a dynamic scan probes an application from the outside, which is the same view an attacker has of your endpoint.
Quick check
What is SSRF in one sentence?
It is when an attacker gets your server to send a request on their behalf, often to an internal address they could not reach directly.
Is a domain allowlist enough on its own?
No. Pair it with a redirect block, size limits and network egress rules, so one missed setting does not open the door.
Where Safeguard fits
Safeguard offers dependency scanning and policies and gates that run in CI, so a vulnerable inference server version can be caught before release. A free scan is available if you want to see your dependency inventory first.
Sources
- vLLM security documentation, read 19 October 2026. This is the developer preview version of the docs.
Last reviewed: 19 October 2026.
