On July 29, 2019, Capital One disclosed that attacker Paige Thompson had exploited a misconfigured web application firewall to make it issue a server-side request to 169.254.169.254 — the AWS EC2 instance metadata service — retrieve temporary IAM credentials, and use them to pull roughly 106 million customer records out of more than 700 S3 buckets. The bug class was server-side request forgery, and the fallout was significant enough that AWS shipped IMDSv2, which requires a session token on every metadata request specifically to blunt SSRF-driven credential theft. Six years later, SSRF and its close cousin, the open redirect, remain routine findings in Python web applications, and the reason is almost always the same: a URL-validation check that looks correct in a code review but can be defeated by how differently two pieces of code parse the same string. CVE-2021-29921, a real bug in Python's own ipaddress standard-library module, shows how deep this problem goes — even the interpreter's built-in IP parser mishandled a class of malformed input for years. This post walks through building URL validation in Python that actually holds up: allowlisting, IP-range checks, redirect revalidation, and the specific parser tricks attackers use to slip past all three.
Why is naive string matching not enough to validate a URL?
Naive string matching fails because URL parsers disagree with each other, and an attacker only needs your validator and your HTTP client to disagree once. Security researcher Orange Tsai's 2017 Black Hat talk, "A New Era of SSRF — Exploiting URL Parser in Trending Programming Languages," catalogued how characters like @, #, backslashes, and unusual Unicode dot variants get interpreted differently by different parsers and libraries, letting a URL that looks like it points to safe-domain.com actually resolve somewhere else entirely — for example a host string like https://safe-domain.com@attacker.com/ where everything before the @ is just userinfo, not the host. A regex or str.startswith() check against an allowed prefix is exactly the kind of validation this defeats, because it inspects the raw string rather than the structured, correctly parsed result. The fix is to parse the URL exactly once, with the same library your outbound HTTP client uses (Python's urllib.parse.urlsplit), and validate the parsed .hostname attribute rather than substrings of the original text.
How should you build a host allowlist in Python?
An allowlist should compare parsed, lowercased hostnames against an exact set or a validated suffix, never a substring check. "trusted.com" in url is not a security control — it matches evil-trusted.com.attacker.net just as happily as it matches the real thing. The safer pattern parses the URL first: from urllib.parse import urlsplit then host = urlsplit(url).hostname and checks host in ALLOWED_HOSTS for an exact-match set, or, for subdomains, confirms the parsed host ends with "." + "trusted.com" (never a bare .endswith("trusted.com"), which still matches nottrusted.com). OWASP's SSRF Prevention Cheat Sheet recommends allowlisting as the primary defense over denylisting precisely because the set of things to block (internal ranges, cloud metadata IPs, alternate encodings) is open-ended, while the set of destinations a given feature legitimately needs to reach is usually small and enumerable.
How do you block internal IP ranges without breaking on inconsistent parsing?
Block internal ranges by resolving the hostname to its actual IP and checking that IP with Python's ipaddress module — not by pattern-matching the hostname string, and not by trusting whatever IP a downstream library re-parses later. import ipaddress then ip = ipaddress.ip_address(resolved_ip) lets you check ip.is_private , ip.is_loopback , and ip.is_link_local in one pass, covering RFC 1918 ranges, 127.0.0.0/8, and the 169.254.0.0/16 block that includes the cloud metadata address Capital One's attacker reached. But the check is only as good as the parser underneath it: CVE-2021-29921 documented that Python's ipaddress module, before it was patched, accepted IPv4 octets with leading zeros — like 010.0.0.1 — inconsistently, which matters because some libraries treat a leading zero as octal and others as decimal, so the same string can resolve to two different addresses depending on which code reads it. Run patched Python (3.9.5+, 3.8.12+, or later) and validate the IP that will actually be connected to, not a string that merely looks like one.
Why isn't disabling the first redirect enough?
Disabling redirect-following on the initial request isn't enough because SSRF and open-redirect protections have to apply to every hop, not just the one your code explicitly requested. A common half-measure is setting allow_redirects=False (requests) or configuring a PoolManager to reject redirects in urllib3, checking the first destination against the allowlist, and calling it done — but if the application ever does follow a redirect chain (directly, or through a proxy layer that re-enables it), each subsequent Location header is attacker-influenced data that hasn't been checked. urllib3's own documentation notes that redirect handling configured at the pool or session level doesn't substitute for revalidating the destination on every hop. The robust pattern is to fetch with redirects disabled, inspect the Location header yourself, re-run the full validation (allowlist plus IP-range check) against that new URL, and only then issue the next request manually — repeating until you hit a non-redirect response or a hop limit.
What's the safest way to actually connect once a URL passes validation?
The safest approach is to resolve the hostname to an IP, validate that specific IP, and then connect to the validated IP directly — rather than validating a hostname and letting the HTTP client re-resolve DNS moments later. Between your check and the actual connection, DNS can answer differently a second time; an attacker controlling a domain's DNS records can serve a public IP to your validator and a private one to your HTTP client, a timing gap known as DNS rebinding. Pinning the resolved IP for the connection (for example by passing it explicitly or using a custom transport adapter) closes that window. Combined with allowlisting, ipaddress-based range checks on the resolved address, and per-hop redirect revalidation, this closes the three gaps — parser disagreement, incomplete range coverage, and resolve-time-of-check-to-time-of-use drift — that let SSRF and open-redirect bugs survive despite validation code that looks reasonable on first read.