SA-11, Developer Testing and Evaluation, is in the System and Services Acquisition family of NIST SP 800-53 Rev 5. It tells the organization to require the developer of a system, component or service to do five things at every stage after design: plan ongoing security and privacy control assessments, perform testing and evaluation at a depth and coverage the organization chooses, produce evidence that the plan was executed and what the results were, run a verifiable flaw remediation process, and correct the flaws that testing finds.
800-53 binds US federal systems, and reaches other companies through contracts and flow down clauses. For everyone else it is a reference. If you sell software to a customer who asks how you test it, SA-11 is a clean way to organize your answer.
What it means in practice
Notice that the control is aimed at "the developer." If you build your own product, you are the developer. If you buy a system or hire a contractor to build one, SA-11 becomes a contract requirement you place on them.
The guidance in the record is specific about method. Testing custom software may involve manual code review, security architecture review and penetration testing, along with static analysis, dynamic analysis, binary analysis or a hybrid. It also names security instrumentation and fuzzing. The assessment plan should say which activities are planned, how rigorous they are, how often they run, and what artifacts they produce.
Two words carry weight: depth and coverage. Depth is how rigorous each assessment is. Coverage is how much of the system is included. A scan that touches one repository out of forty has shallow coverage, however good the tool is.
The enhancements show how far the control can go: SA-11.1 Static Code Analysis, SA-11.2 Threat Modeling and Vulnerability Analyses, SA-11.3 Independent Verification of Assessment Plans and Evidence, SA-11.4 Manual Code Reviews, SA-11.5 Penetration Testing, SA-11.6 Attack Surface Reviews, SA-11.7 Verify Scope of Testing and Evaluation and SA-11.8 Dynamic Code Analysis. You pick the ones your risk calls for. Few small companies need all eight.
Evidence to collect
- A written assessment plan that lists test types, frequency, depth and coverage for each system.
- Pipeline configuration showing static analysis, dependency scanning and dynamic testing running on real branches.
- Scan results with dates, tool versions and the commit or build they ran against.
- The latest penetration test report, its scope statement and the retest after fixes.
- A flaw register or ticket export showing each finding, severity, owner, fix date and verification.
- Code review records for security sensitive changes, such as pull request approvals with the reviewer named.
- If you outsource development, the contract clause that requires the above and the evidence the vendor gave you.
A worked example
A small team ships a web service from twelve repositories. Their plan says static analysis and dependency scanning run on every pull request, dynamic testing runs nightly against staging, and an outside penetration test happens yearly. Each finding becomes a ticket with a severity based due date. Once a quarter the lead compares the repository list with the scanned list and finds two new repositories nobody added. They fix the pipeline, note the miss, and keep the comparison as evidence. That loop, plan then run then fix then verify, is the control.
Common gaps
Plans that exist only in someone's head. The tests run, but nothing says what is supposed to run, so an auditor cannot tell if a skipped step is a gap or a choice.
Results without remediation. Scanners produce lists, and the list grows. SA-11 asks for a remediation process that can be verified, which means a record that a finding was fixed and the fix was checked.
Coverage drift. A new service ships and nobody adds it to the pipeline. Compare your repository list against the list of scanned repositories every quarter.
Noise. When most findings are unreachable or irrelevant, people stop reading them. Reachability analysis, which checks whether vulnerable code is actually called, helps keep the queue honest.
How to implement it
- Write a one page assessment plan per product. List the test types, the trigger for each (every commit, nightly, every release, yearly), and who owns the result.
- Wire the automated tests into the build so a failing result is visible and blocks when the risk calls for it.
- Schedule manual work you cannot automate: a yearly penetration test, and manual review of authentication, payment or data export code.
- Send every finding to one tracker with a severity, an owner and a due date based on your policy.
- Retest fixed findings and record the result. Close the ticket only after verification.
- Keep the evidence where you can export it by date range. Screenshots decay. Exports and logs do not.
Click through it at your own pace, or open it full screen.
Try it yourself
Paste a dependency manifest into the free in browser scan at gold.safeguard.sh/scan and see which known vulnerabilities your dependencies carry. It gives you a first data point for the plan above.
Quick check
An auditor asks for proof of SA-11. What is the minimum set?
A plan that states what is tested and how often, results from actual runs, and a record that findings were fixed and verified. Plan, execution evidence and remediation evidence map directly to the control text.
Does SA-11 require a penetration test?
Not by itself. Penetration testing appears as the enhancement SA-11.5. The base control requires testing at a depth and coverage the organization defines. Mappings to other frameworks vary, so check the mapping your auditor uses.
