Plant fake secrets with a unique marker, push them through your application the way real data would travel, then search every place logs end up. If a canary appears anywhere in plain text, redaction has a gap. Run the test in CI so it keeps working.
What a canary value is
A canary is a fake sensitive value that you can recognize later. It looks like the real thing, a password, an API key, a card number in a test range, an email, but carries a marker such as CANARY-7f3a so a search finds it instantly. It is not a real credential and gives access to nothing.
The OWASP Logging Cheat Sheet lists testing steps for logging, including checking that logging works as specified, testing that mechanisms are not open to injection, verifying access controls on log data, and testing the effect of logging failures. Redaction testing fits alongside those checks.
Where to look
| Destination | Why it matters |
|---|---|
| Application stdout and local files | Raw value exists before any pipeline |
| Central log store | Wider access, longer retention |
| Trace backend | Attributes and URLs |
| Error reports | Request details and local variables |
| Metrics labels | High cardinality values pasted in by mistake |
| Backups and exports | Copies you forget |
Steps
- Pick canaries per data type: a password field value, a bearer token, a fake card number, an email, a query string token. Give each a distinct marker.
- Send them through real entry points: login form, API call with an Authorization header, a request with a token in the URL, a form with a long free text field.
- Trigger failure paths on purpose. Errors are where the full request object tends to get logged. Send malformed input and force an exception.
- Wait for the pipeline to flush, then search each destination in the table for the marker.
- Record results as pass or fail per data type and destination. A small grid is enough.
- For every fail, fix at the source first, then add a pipeline rule as backup. See redact at the source or in the pipeline.
- Automate. Run the canary suite in staging on each release, and fail the build when a marker is found.
Pitfalls
Do not use real credentials, even expired ones. Do not leave canaries in production data, since they pollute analytics and can trigger detections. Mark the test accounts clearly. Also search for transformed forms: a Base64 copy, a URL encoded copy and a truncated copy often survive redaction that catches the plain string.
Beyond the pass or fail result
Count how many destinations each canary reached before it was caught. A canary that was masked in the central store but sits in local files tells you the pipeline works and the source does not. Add the grid to your release checklist, and rerun after any logging library upgrade, since a changed default can quietly undo a fix.
Quick check
Can I use a real expired key as a canary?
No. Use a fabricated value with a marker. Real keys, even expired ones, can cause confusion and policy problems.
How often should the test run?
On every release in staging, and after any change to logging libraries or collector configuration.
Safeguard's dependency scanning flags vulnerable logging and telemetry libraries in your SBOM. Try a free scan at gold.safeguard.sh/scan.
Sources
Last reviewed: 19 October 2026.
