SOC 2 Evidence and Sampling: What the Auditor Actually Pulls

On 14 May 2026 the AICPA's peer review team warned CPA firms about SOC 2 engagements that come out with identical reports, risk assessments, sample sizes and testing procedures for every client. Peer reviewers have been told to look for that pattern. For a company preparing a first audit the lesson runs the other way: a good auditor will size samples to your controls, so you need to understand how sampling works before fieldwork starts.

What's affected

Every control in a Type 2 examination that runs more than once. The auditor tests operating effectiveness by choosing items from a population and checking each one against the control description. Evidence is whatever shows the control happened: a ticket, an approval, a screenshot with a date, an exported log, a signed form.

Three terms matter.

• Population. The complete list of times the control applied in the period. For offboarding, it is every person who left. For change management, it is every production change. The AICPA definition quoted by one CPA firm is "a population of audit relevance."
• Sample. The items the auditor selects. Methods include simple random, systematic, haphazard and block sampling. A statistical approach requires random selection. A non-statistical approach lets the auditor apply judgment.
• Exception. A sampled item that does not meet the control. One exception on a small sample can mean the control failed.

How to check if you're exposed

For each control, answer four questions. If you cannot, you are not ready for fieldwork.

1. What is the population, and where does the list come from? Auditors need to confirm the list is complete. One CPA firm says auditors may review the query used to pull it, so nothing was filtered out, or watch an administrator generate the user list.
2. How often does the control run? A control that runs daily or per change has a large population. A quarterly control may have four items in a year.
3. Where does the evidence live, and for how long? One CPA firm describes a company that kept evidence as email attachments, and the attachments were purged after six months. Older samples then had no support.
4. Who produces it, and could they produce it today? If the answer needs a week, the control is not ready.

A rough map from control to population to evidence:

What to do now

1. Build a one page evidence calendar: control, owner, frequency, system of record, and the date it next runs. Keep it in a shared place.
2. Pull each population yourself before the auditor does, and check it against a second source. Compare the leaver list from HR with deactivations in your identity provider.
3. Test your own samples. Pick five items per control and gather the evidence as if the auditor asked. Anything that takes more than an hour to find is a gap.
4. Fix retention. Move evidence out of mailboxes and chat into a system with a fixed retention period that is longer than your observation window.
5. Tell the auditor about system changes. One CPA firm describes a company that moved data to a new system without telling the auditor, while the old system was still in scope for part of the period.

A starter evidence calendar:

Weekly:    review failed backup jobs, record outcome in ticket
Monthly:   reconcile HR leavers to identity provider deactivations
Quarterly: user access review for production, source control, cloud
Quarterly: vendor list review, new vendors assessed
Annually:  policy review and approval, risk assessment, tabletop exercise
Per event: change approval, incident record, new hire and leaver access

How to avoid this trap

• Do not assume a sample of one stays small. A CPA firm reports that a Type 1 sample of one new hire can become 25 in a Type 2. Plan for the larger number.
• Do not rely on a control that runs only when someone remembers. The firm's examples show that missed runs are what show up as exceptions.
• Do not edit evidence after the fact. Dated, system generated records beat hand made ones.
• Do not let the population list come from the person who owns the control. A second source is what makes the list credible.

Why it matters

One CPA firm's published example is useful. For a control tested on a population of 389 employees, with zero deviations expected, the initial sample would be 25. After one deviation the auditor expands to 40, after a second to 60, and after a third the control is judged not operating effectively. For a monthly control the auditor selects three months, and any deviation is a failure of operating effectiveness.

Those numbers are one firm's method. There is no single table that every auditor must use, which is exactly why you ask your own firm for its approach before the window opens. The rule that holds everywhere is that small populations leave no room for a miss, and large populations reward a process that runs the same way every time.

Try it yourself

Pick one control from your list, such as removing access when someone leaves. Write the population query, pull the last 10 items, and attach the evidence for each. Time how long it takes. If it takes more than an afternoon, automate the pull or change where the evidence is stored.

Click through it at your own pace, or open it full screen.

Quick check

Sources

• Linford and Company: Audit sampling in SOC examinations
• Linford and Company: Moving from a Type I to a Type II SOC report, what to avoid
• Linford and Company: What Is SOC 2? (updated 13 October 2026)
• Journal of Accountancy: AICPA guides peer reviewers to address SOC 2 risks (14 May 2026)