Safeguard
Developer Tools•

How do you pin a known failure as a regression case in Hypothesis?

Add the failing input with the example decorator. Hypothesis runs it first on every run, so a fixed security bug cannot return unnoticed. Steps inside.

Safeguard Security Team
4 min read

Pin a known failure in Hypothesis by adding the failing input to the test with the @example decorator. Hypothesis runs explicit examples before random generation on every run, so a fixed bug gets rechecked each time. You do not depend on the local example database.

Why the database is not enough

Hypothesis saves failures it finds in an example database and replays them later. That is handy, but the docs are plain about its role: it is best thought of as a cache that you never need to invalidate, and you should not rely on it for correctness. By default it sits under .hypothesis/examples in the working directory, which most teams do not commit.

So a bug found in CI may never be replayed on a developer laptop, and a cleaned working directory forgets everything. For a security regression, forgetting is not acceptable. The docs give the answer: to ensure Hypothesis always tries an input, use @example.

How explicit examples behave

From the reference documentation:

  • Explicit examples run before random generation, in their own phase.
  • They do not count toward max_examples.
  • If one fails, Hypothesis stops and reports that failure without generating random inputs.
  • They do not shrink, so the input you pin is the input that runs.

The last point matters. Pin the already shrunk, readable input, not the raw first failure.

Steps

  1. Reproduce the failure and take the shrunk input from the output.
  2. Write a short comment naming the ticket and what the input proves.
  3. Add the example above the test, matching the argument names of the test function.
  4. Run the test against the unfixed code and confirm it fails at the explicit phase.
  5. Apply the fix and rerun. The pinned case passes and the random search continues.
  6. Commit the test and the fix together.

For a bug reported as a list containing a not a number float, the docs show this style of example: an @example carrying the reported values, so the reported failure is reproduced on purpose.

Where to put the pinned cases

OptionGood forWatch out for
Decorator on the property testOne property, a few casesLong lists clutter the file
Separate regression test per bugClear ticket mappingMore files over time
Parametrized list fed to the decoratorMany cases, one placeNeeds care with argument shapes

Pick one pattern for the team. Consistency makes audits and reviews faster, since reviewers know where to look for the pinned case when a security fix lands.

Common mistakes

Pinning the unshrunk input. It is long, hard to read, and may include irrelevant data.

Pinning private data. If the failing input came from production, replace names, emails and tokens with fake values, then confirm it still fails.

Pinning without a comment. In a year nobody will know whether the case can be deleted.

Skipping the failing run. Always see the test fail before the fix, or you cannot be sure the pinned case exercises the bug.

Try it yourself

Click through it at your own pace, or open it full screen. Look at how a fix is tied to the finding it resolves, which is the same link your pinned case should keep to its ticket.

Safeguard offers autonomous remediation with fix pull requests for dependency findings, off by default and set to manual for every category. A free scan shows what it would flag.

Quick check

Do explicit examples slow the suite down?

Barely. They are a handful of fixed inputs run once, and they do not count toward max_examples.

Can I rely on the example database instead?

Not for correctness. The documentation says to use the example decorator when an input must always be tried.

Sources

Last reviewed: 19 October 2026.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.

Self-healing security runs on Safeguard.

Your first fix PR is minutes away.

No sales call required, even your agent can complete the purchase over MCP.