Safeguard
Open Source Security

Recurring vulnerability patterns in the PyPI ecosystem

PyYAML shipped two rounds of deserialization fixes in under two years — CVE-2017-18342 and CVE-2020-14343 — because the underlying pattern kept resurfacing.

Safeguard Research Team
Research
6 min read

In January 2021, PyYAML 5.4 shipped a fix for CVE-2020-14343 — the second deserialization patch the library had needed in under two years. The first, CVE-2017-18342, closed off yaml.load()'s default behavior of fully reconstructing arbitrary Python objects from untrusted input, which let a crafted YAML document instantiate any class the interpreter could reach, including ones that shell out to the OS. PyYAML's maintainers responded by deprecating plain load() in favor of safe_load() and SafeLoader. But CVE-2020-14343 showed the fix from CVE-2020-1747 — shipped in PyYAML 5.3.1 back in March 2020 — was incomplete: even the supposedly safer full_load() and FullLoader path could still be abused through the python/object/new tag to instantiate dangerous classes, subprocess among them, and execute arbitrary code. Two patches, one root cause, under two years apart. That is the pattern worth studying: certain vulnerability shapes — deserialization, path handling — don't get fixed once and disappear. They recur across libraries and across versions of the same library, because the underlying language feature that creates the risk never goes away. This post walks through the three shapes that show up most often in PyPI packages and what actually separates a scanner that finds them from one that tells you which ones matter.

Why does pickle deserialization keep showing up as a vulnerability class?

Pickle deserialization keeps showing up because pickle.load() isn't a data parser in the way json.loads() is — it's a bytecode interpreter for a small stack-based virtual machine, and Python's own documentation states plainly that unpickling data from an untrusted source is not secure. A pickled object can carry a __reduce__ method that gets called during reconstruction, and that method can return any callable with any arguments, including os.system or subprocess.Popen. This isn't a bug in pickle — it's the intended behavior of a format designed to serialize live object graphs, not sanitized data. The failure mode is always the same: a library caches, queues, or transmits pickled data (Celery task results, cache backends, ML model checkpoints saved with torch.save or joblib.dump) and later unpickles it without verifying the source was trusted. Bandit's B301 check exists specifically because this pattern recurs across unrelated projects rather than being one library's mistake — it's a structural hazard baked into how the format works.

Why did PyYAML need two separate CVEs for the same underlying problem?

PyYAML needed two CVEs because YAML's tag system is deliberately extensible to represent arbitrary Python types, and closing off one instantiation path left another open. CVE-2017-18342 fixed the most direct route: yaml.load() without an explicit loader used Loader, which supported the !!python/object/apply tag to call arbitrary constructors. The fix pushed users toward full_load() and FullLoader, which blocked function calls but still permitted certain object construction. CVE-2020-14343 then showed full_load() could still reach subprocess through python/object/new, because the safety boundary was drawn around "which tags require calling a function" rather than "which classes are safe to instantiate at all" — a narrower fix than the vulnerability warranted. The lesson for defenders isn't specific to PyYAML: when a security patch narrows one path to a dangerous primitive, verify the fix addresses the primitive itself, not just the path your proof-of-concept used. yaml.safe_load() remains the only loader in PyYAML that restricts construction to a fixed allowlist of basic types, and it's the one the maintainers now document as the default recommendation.

What makes path traversal such a persistent bug class in Python packages?

Path traversal persists because the primitives Python offers for combining paths — string concatenation, os.path.join(), and zipfile.extractall() — don't reject ../ segments by default; validating them is a step the developer has to remember to add every single time. os.path.join("/uploads", user_filename) happily returns /uploads/../../etc/passwd if user_filename contains traversal sequences, because join()'s job is string construction, not security enforcement. The same gap shows up in archive extraction: a .zip or .tar file can contain an entry named ../../../etc/cron.d/evil, and code that calls extractall() without inspecting member names will write it there. This shows up repeatedly in file-upload handlers, log-processing tools, and package/archive utilities across the PyPI ecosystem precisely because the vulnerable call looks identical to the safe one — there's no syntactic marker distinguishing extractall() on a trusted internal archive from extractall() on an attacker-supplied one. That distinction only exists in the data flow, which is exactly what a lexical or pattern-matching scanner cannot see.

Why do these three bug classes need different detection strategies than a typical CVE scan?

These bug classes need different detection strategies because a standard software composition analysis (SCA) scan tells you a vulnerable version of a package is installed, but not whether your code — or a dependency's code — actually calls the dangerous function with attacker-influenced data. Grepping a codebase for yaml.load, pickle.loads, or extractall produces a source list, not a risk list: most matches turn out to be operating on trusted, internally generated files rather than untrusted input. The higher-signal approach starts with the same sink list — pickle.load/loads, yaml.load on anything but SafeLoader, zipfile.extractall/tarfile.extractall without a member filter — and then traces backward through the call graph to determine whether that sink is reachable from a network handler, a file upload, a queue consumer, or another attacker-influenced entry point. A sink invoked only on a hardcoded configuration file at startup is a very different finding than the same sink invoked on the body of an incoming HTTP request.

How does Safeguard help teams triage these specific patterns?

Safeguard's reachability analysis is built for exactly this triage problem: rather than stopping at "package X contains a known-dangerous pattern," it builds a static call graph from your source, lockfile, and bytecode and classifies each finding as reachable, conditionally reachable, unreachable, or unknown, then layers in dynamic reachability from production runtime data to confirm which of those paths are actually exercised. A pickle.loads() call flagged as CWE-502 in a caching layer gets ranked very differently depending on whether the call graph shows it fed only by internally signed cache entries or by a queue message an external client can influence. Griffin can then explain the specific taint path in plain language and, where a safe replacement exists — swapping yaml.load() for yaml.safe_load(), or adding a path-traversal filter before extractall() — propose the corrected code as a pull request instead of leaving a generic CWE-502 or CWE-22 ticket for an engineer to research from scratch.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.