Spend ten minutes each month on five checks: did every collector run, are the files non-empty and readable, are retention settings unchanged, did any integration token expire, and are gaps written down. Automation fails quietly, so the check is what makes the evidence trustworthy.
Why automated evidence needs a checker
A script that stopped in February looks identical to a script that is working until someone opens the folder. In an audit, that discovery happens in front of the auditor. A monthly check moves it to you.
The five checks
| Check | How to do it | Failure sign |
|---|---|---|
| Collectors ran | Open the CI schedule history | A missing or red run |
| Files are usable | Open one file from each source, check it has content and a collected-at note | Zero bytes, error text in JSON |
| Retention unchanged | Read the organization retention value | Different from last month |
| Credentials alive | Look for authentication errors in the job log | 401 or 403 responses |
| Gaps recorded | Read the evidence log | A missing month with no note |
Steps
- Open your evidence folder and sort by date. The newest month should have a file for every source.
- Open one file per source. Check the size and look at the first few lines.
- Read the retention setting. GitHub's documentation says the default is 90 days and that a changed value applies only to new objects, so a quiet change can shorten what you will have later.
- Check the token or app expiry date for each collector and renew any within thirty days.
- If something failed, rerun it now. Write in the log what happened, when you reran it and what the gap is. A recorded gap is much better than an unexplained one.
- Sign off with your name and date in the log.
Handling a gap honestly
Sometimes data cannot be recovered. A 180 day audit log window, for example, cannot be reopened after it passes. Write the gap down, state the compensating evidence, such as the ticket history or a pull request list from the same period, and move on. Do not backfill a record and imply it was collected on time. Dates on evidence must be true.
A short log format
Keep the log as a simple table in the evidence folder with one row per month: date checked, checker, sources verified, problems found, action taken. Ten rows into the year it becomes its own evidence that monitoring of your controls happens, which is often what an auditor wants to see beyond the raw files.
Who should own it
The person responsible for compliance, with a backup. If the owner is also the person who writes the collector, ask someone else to do the check once a quarter.
Link it to your yearly audit calendar.
Where Safeguard fits
Safeguard offers compliance and evidence automation and continuous scanning, so scan evidence arrives with dates. Try the free scan.
Quick check
How often should the check run?
Monthly suits most small teams. Run it sooner after any change to tokens, tools or CI configuration.
Can I backfill a missed month?
Only with data that still exists, and the file must show when it was really collected. Never imply it was collected earlier.
Sources
- GitHub Docs: retention period configuration
- GitHub Docs: reviewing the audit log for your organization
Last reviewed: 19 October 2026.
