CWE-176, Improper Handling of Unicode Encoding, describes software that does not properly deal with input containing Unicode encoding. It hits anything that accepts text from outside: web servers, APIs, file handlers, command parsers.
The danger is rarely Unicode itself. It is disagreement. One component sees a string one way, the next component sees it another way, and the security check only looked at the first view.
How it happens
A typical request passes through several layers. A server checks the URL for forbidden patterns, then hands it to a file system or script engine that decodes it further. If the check runs on the encoded form and the decoder runs afterward, an attacker can write the same dangerous path in a form the check does not recognise.
MITRE lists real examples. In CVE-2000-0884, a server let remote attackers read documents outside the web root through malformed URLs containing Unicode encoded characters. In CVE-2001-0709, a URL encoded with Unicode let an attacker obtain the source code of ASP files.
What an attacker gains
MITRE lists the common consequence as integrity: the product ends up in an unexpected state. In the examples above that meant reading files that were meant to be off limits.
How to find it
The record has no listed detection methods, so rely on basics. Send the same forbidden value in several encodings (plain, percent encoded, overlong or alternate Unicode forms) and compare the responses. If any encoding gets a different answer from the plain form, a layer disagrees with another.
How to fix it
The record lists no mitigations, so these are general practice rather than MITRE guidance.
- Decode and normalise input once, as early as possible, into one canonical form.
- Run every security check on that canonical form, and use only that form afterward.
- Reject input that is not valid in the expected encoding instead of guessing.
- Keep the framework's decoding path current, since many of these bugs live in server and library code.
Quick check
Why is a check on the raw URL not enough?
A later layer may decode the URL again. The check approved one string and the system acted on another.
