Safeguard
Application Security

CTF writeup patterns: serialization and cryptographic puzzles, decoded

CVE-2013-0156 let attackers RCE Rails by feeding YAML into a parameter parser — the same insecure-deserialization pattern CTF players train on every weekend.

Safeguard Research Team
Research
7 min read

On January 8, 2013, Aaron Patterson disclosed CVE-2013-0156: Ruby on Rails' XML parameter parser would happily decode an incoming request body as YAML or as Ruby Symbols, and both paths let an attacker instantiate arbitrary objects on the server. The result was unauthenticated remote code execution against every unpatched Rails app on the internet, fixed only in 2.3.15, 3.0.19, 3.1.10, and 3.2.11. If that flaw sounds familiar to anyone who has played a CTF, it should — "insecure deserialization" is one of the most reliable challenge categories in competitive hacking, precisely because it maps so cleanly onto real incidents. Two years later, at AppSecCali 2015, Gabriel Lawrence and Chris Frohoff showed the Java world its own version of the same bug class with the ysoserial tool and the Apache Commons Collections gadget chain, which Stephen Breen of FoxGlove Security then used to demonstrate zero-day RCE against WebSphere, WebLogic, JBoss, and Jenkins. This post walks through the puzzle patterns CTF authors keep reusing — deserialization, gadget chains, padding oracles, and JWT confusion — and the production vulnerabilities each one is standing in for.

Why is insecure deserialization such a common CTF category?

Insecure deserialization is a common CTF category because the exploit primitive is simple to teach but the consequences are maximal: turning a data format into a code-execution vector. In Python, pickle.loads() on attacker-controlled bytes can execute arbitrary code because pickle streams can embed instructions to call any importable function, not just reconstruct data — the Python documentation itself warns never to unpickle untrusted input. PHP's parallel is unserialize() acting on user input, known as PHP Object Injection, where a crafted serialized string reconstructs an object whose magic methods (__wakeup, __destruct) run attacker-chosen logic on instantiation. CVE-2013-0156 is the canonical real-world version: Rails' XML-to-YAML coercion meant any endpoint accepting XML params was, in effect, an unauthenticated YAML.load() on the whole internet. CTF challenges strip away the framework plumbing and hand you a raw serialized blob, but the skill transfers directly: recognize the format, find the sink, and reconstruct the object graph that gets you execution.

What is a gadget chain, and why does Commons Collections keep showing up?

A gadget chain is a sequence of otherwise-harmless method calls, already present in a target's dependencies, that an attacker stitches together into a payload that achieves code execution when deserialized — without needing to inject any new code onto the system. Java's native serialization calls readObject() on every deserialized class, and Lawrence and Frohoff's 2015 ysoserial research found that Apache Commons Collections' InvokerTransformer class — designed to invoke an arbitrary method via reflection — could be chained through a LazyMap and an AnnotationInvocationHandler to turn any ObjectInputStream.readObject() call into remote code execution. Because Commons Collections shipped inside countless enterprise Java applications, this single gadget chain became a universal skeleton key; Breen's FoxGlove Security research showed it working unmodified against completely unrelated products. CTF "gadget chain" challenges usually hand you a smaller, custom class hierarchy and ask you to build the equivalent chain by hand — the same reflective-call-graph reasoning attackers used against WebLogic and Jenkins, just scoped to a puzzle instead of a Fortune 500 app server.

Why do padding oracle challenges keep appearing in crypto CTF tracks?

Padding oracle challenges persist because the underlying flaw — a server that reveals, even indirectly, whether decrypted ciphertext padding was valid — turned out to be devastatingly practical against real, widely deployed protocols. CBC-mode block ciphers pad plaintext to a fixed block size, and if an application returns a different error (or even a measurably different response time) for "bad padding" versus "bad content," an attacker can decrypt an entire ciphertext byte-by-byte without ever knowing the key, by resubmitting billions of tampered ciphertexts and watching which ones the oracle accepts. Google's security team publicly documented exactly this against SSL 3.0's CBC handling in 2014, naming the attack POODLE and tracking it as CVE-2014-3566; it forced browsers to deprecate an entire protocol version. CTF versions of the puzzle usually give you a decryption endpoint and a padding-error signal and ask you to script the byte-recovery loop yourself — which is precisely the automation POODLE required at scale, just against a toy oracle instead of a production TLS stack.

How does the classic "penguin" image expose ECB mode's weakness without breaking any math?

The ECB "penguin" is famous because it demonstrates a real cryptographic weakness using nothing but visual pattern recognition: Electronic Codebook mode encrypts each identical plaintext block into an identical ciphertext block, so encrypting a bitmap image with ECB leaves large uniform regions (like a penguin's black-and-white silhouette) clearly visible in the "encrypted" output, since repeated plaintext blocks always produce repeated ciphertext blocks. No key recovery or math is required to see the flaw — you just look at the image. CTF crypto tracks use this exact property in more subtle ways: an ECB-encrypted cookie or session token that repeats a known plaintext block (like a username field) lets a player detect and manipulate block boundaries, then reorder or duplicate ciphertext blocks to forge a token — a "cut-and-paste" attack. Production analogues are why NIST's SP 800-38A guidance and virtually every modern security standard treat ECB as inappropriate for anything beyond a single, fixed-size, non-repeating block, pushing real systems toward authenticated modes like GCM instead.

What makes length-extension attacks a recurring MAC puzzle?

Length-extension attacks recur in CTF MAC puzzles because they exploit a structural property of Merkle–Damgård hash functions — including MD5, SHA-1, and SHA-256 — that is easy to state but non-obvious the first time you see it: given H(secret || message) and the length of secret, an attacker can compute H(secret || message || padding || attacker_data) for a chosen extension, without ever learning secret itself, because the hash's internal state after processing secret || message is exactly the state a legitimate second call would resume from. Any application that authenticates requests as hash = MD5(secret + data) and checks the hash server-side is vulnerable to an attacker appending extra parameters (like &admin=true) and forging a valid hash for the extended message. This is precisely why HMAC exists as a standard, rather than naive keyed hashing: HMAC's double-hash construction breaks the length-extension property entirely. CTF challenges that hand you an oracle producing H(secret||msg) pairs are training the exact reasoning needed to identify a homemade MAC scheme in a code review, before it ships.

Why do JWT "alg:none" and key-confusion challenges keep coming back?

JWT puzzle challenges persist because the standard's flexibility — letting the token itself declare which algorithm verified it — became a well-documented attack surface the moment implementations trusted that declaration blindly. A JSON Web Token's header specifies its signing algorithm, and early or misconfigured verification libraries would honor an attacker-set "alg": "none" and accept an unsigned token as valid, or accept an RS256 token re-signed with HS256 using the server's public RSA key as the HMAC secret — a key-confusion attack made possible because the public key is, by definition, not secret. Both patterns are documented extensively in JWT library security advisories and OWASP's JSON Web Token guidance, and both have shown up as authentication bypasses in production APIs, not just puzzles. CTF web tracks reproduce them by handing you a token, a known-public key, and an endpoint to forge — the same three ingredients present whenever a real API validates JWTs without pinning the expected algorithm.

How Safeguard helps

None of these classes disappear because a team "knows about CTFs" — they disappear when the artifacts that make them exploitable never reach production in the first place. A large share of real incidents in this family, from the Rails YAML bug to homemade HMAC schemes, trace back to a hardcoded key, secret, or credential sitting somewhere it shouldn't. Safeguard's secrets scanning covers exactly that root cause: it scans source, Git history, container layers, and build logs for known-issuer patterns and high-entropy strings, and verifies live credentials against the issuing service before raising an alert, so a hardcoded signing key or MAC secret — the kind of thing a length-extension or key-confusion attack depends on — gets flagged and revoked before it becomes the missing ingredient in someone's exploit chain.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.