CWE-174 Double Decoding of the Same Data: How It Bypasses Filters

CWE-174 happens when a product decodes the same input twice. Any protection that runs between the two decodes inspects an intermediate form, and the second decode turns it into something the protection never saw.

It affects web apps, frameworks and gateways, anywhere a value can pass through more than one layer that percent decodes or unescapes. The impact range in the record is wide: bypass of protection mechanisms, code execution, and effects on access control, confidentiality, availability and integrity, varying by context.

How it happens

Take a request parameter. A web server decodes it once. A framework decodes it again when it builds its parameter map. Your handler calls a decode function on the value for good measure. Each layer thinks it is the only one.

Meanwhile a filter sits somewhere in that chain looking for dangerous characters. The attacker encodes the dangerous character twice. After the first decode, the filter sees a harmless percent sequence. After the second decode, the dangerous character appears.

Here are two layers that each look reasonable.

from urllib.parse import unquote

def handler(raw: str):
    once = unquote(raw)                 # layer 1
    if "/" in once or ".." in once:     # filter checks the intermediate form
        raise ValueError("bad name")
    name = unquote(once)                # layer 2 decodes again
    return open("/srv/files/" + name, "rb").read()

A raw value of %252e%252e%252fsecret becomes %2e%2e%2fsecret after the first decode, which has no dot or slash and passes the filter. The second decode turns it into ../secret.

The observed examples in the record match this closely.

• CVE-2004-1315 is forum software that improperly URL decodes the highlight parameter when extracting text to highlight. Double encoding the value let remote attackers run arbitrary PHP code.
• CVE-2004-1939 is an XSS protection mechanism that removes the slash that could close tags, but is bypassed using double encoded slashes, %252F.
• CVE-2001-0333 is directory traversal using double encoding.
• CVE-2004-1938 uses %2527, a double encoded single quote, in SQL injection.

Every one of those is a protection that looked at the wrong form of the data.

What an attacker gains

The record lists the consequences as bypass of protection mechanisms, execution of unauthorized code or commands, and impact on access control, confidentiality, availability and integrity, depending on context.

The attacker rarely needs a new bug. The attack is an existing one, such as traversal, cross-site scripting or SQL injection, hidden from the filter that should have stopped it. Double decoding is the disguise.

How to find it

MITRE lists no detection methods for this entry. The checks below work.

• Send the double encoded form of every character you filter: dot, slash, backslash, quote, angle brackets and percent itself. Look at the response and at server logs to see which form arrived at the handler.
• Draw the decode chain for each parameter, from the edge to your code. Count how many layers decode. If your own code calls a decode function on a value that arrives from a framework, that is a red flag.
• Grep for repeated decode calls and for decode calls inside loops.
• Check scanners and web application firewalls on your own staging copy. A filter placed before the final decode may be blind to the real value.

How to fix it

1. Decode exactly once, as early as possible, into the internal form of the application. Validate that form and use that form.
2. Do not let the same data be decoded by later layers. If a library offers raw and decoded accessors, choose one and use it consistently.
3. Assume all input is malicious. Use an accept known good strategy: define what acceptable input looks like, reject what does not conform, and consider length, type, range, syntax and consistency across fields.
4. Avoid making decisions on names of resources if they can have alternate names. Prefer identifiers that map to resources on the server.
5. After the one decode, reject values that still contain percent sequences when none are expected. That catches double encoding.

from urllib.parse import unquote
import os, re

def open_named(raw: str) -> bytes:
    name = unquote(raw)                       # exactly once
    if "%" in name or not re.fullmatch(r"[A-Za-z0-9_.-]{1,64}", name) or name.startswith("."):
        raise ValueError("bad name")
    base = os.path.realpath("/srv/files")
    full = os.path.realpath(os.path.join(base, name))
    if os.path.commonpath([base, full]) != base:
        raise ValueError("bad name")
    return open(full, "rb").read()

6. Output encode for the context where data lands, so that if a stray encoded character slips through it is rendered as text.

Dependencies that decode twice

Two of the observed cases are in software people install rather than write. If a component in your stack decodes its input twice, the exposure comes with it. The free in-browser dependency scan reads a manifest or lockfile with up to 100 dependencies, needs no login and keeps the file on your machine. Reachability analysis then shows whether your code calls the affected function.

Quick check

Sources

• MITRE CWE-174