You can, if you use it carefully. In pytest, xfail runs the test and expects it to fail, so the bug stays visible in reports. Turn on strict mode so a fix makes the suite fail until the marker is removed, and always attach the ticket that tracks the defect.
xfail versus skip
The pytest documentation explains the difference. A skipped test does not run. An xfail test does run and is expected to fail. If it fails, no traceback is shown and the result is reported as XFAIL. If it unexpectedly passes, the result is XPASS.
For security work this distinction matters a lot. A skipped security test is invisible. An xfail test is a standing, running statement: the exploit still works today.
Why strict mode is the important part
By default, neither XFAIL nor XPASS fails the suite. That is a trap. Someone fixes the bug, the test starts passing, the run says XPASS, and nobody notices. The marker stays forever and the test no longer protects anything, because a regression would flip it back to XFAIL silently.
Setting strict=True on the test makes an XPASS count as a failure. You can set the default for all tests with the xfail_strict ini option. With strict on, the fix commit breaks the build, which prompts the developer to delete the marker and turn the test into a normal regression test. That is exactly the lifecycle you want.
Steps
- Write the security test as if the bug were fixed: it asserts the safe behavior.
- Mark it xfail with strict enabled and a reason that names the ticket.
- Run the suite and confirm it reports XFAIL, not a pass.
- Add a due date in the ticket, not in the code.
- When the fix lands, the strict failure appears. Remove the marker in the same pull request.
- Confirm the test now passes as a normal test.
When it is acceptable
| Situation | xfail OK? |
|---|---|
| Confirmed bug, fix scheduled, ticket open | Yes |
| Confirmed bug, no owner | No, assign an owner first |
| Flaky test you do not understand | No, investigate it |
| Risk formally accepted by the owner | Yes, with the acceptance recorded in the ticket |
| You simply want a green build | No |
The last row is the honest one. Xfail is for documenting a known gap, not for hiding one.
Individual cases inside a parametrized test
The pytest docs show that a single case in a parametrized test can be marked by wrapping it with pytest.param and an xfail mark, so only that case reports as xfailed while the rest pass. That is useful when one attacker input still works and the others are blocked.
Reporting
Count xfailed security tests in your release notes or dashboard. A rising number is a risk signal. Run pytest with the option that lists xfail results so reviewers see them, rather than scrolling past a summary line.
Review the list monthly. Anything older than your agreed fix window needs an escalation, not another month of silence.
Try it yourself
Click through it at your own pace, or open it full screen. Look at how a gate turns a quiet signal into a visible decision, which is what strict mode does for xfail.
Safeguard offers policies and gates that can block merges on rules you set. A free scan shows the dependency findings you would be gating on.
Quick check
Does xfail still run the test?
Yes. Unlike skip, xfail runs the test and expects a failure, unless you set run to false.
What does strict do?
It makes an unexpected pass fail the suite, so the marker cannot linger after the bug is fixed.
Sources
Last reviewed: 19 October 2026.
