A penetration test of an LLM app should cover the conventional surface (API, authentication, authorization, web front end) plus model specific areas: prompt injection, agent tool abuse, output handling, tenant data leakage, and cost abuse. Agree rules of engagement first, and ask for findings that include reproduction steps.
Start from the two OWASP lists
Use the OWASP API Security Top 10 (2023) for the service and the OWASP Top 10 for LLM Applications (2025) for the model layer. Together they give a scope document that a tester and a buyer will both recognise.
Scope table
| Area | What testers should attempt | Source of ideas |
|---|---|---|
| Object level access | Read another tenant's conversations or files by changing IDs | API1 |
| Authentication | Token reuse, weak sessions | API2 |
| Resource limits | Long prompts, large outputs, many calls | API4, LLM10 |
| Prompt injection | Direct and indirect, through documents and tool results | LLM01 |
| Excessive agency | Make an agent call tools beyond the user's rights | LLM06 |
| Output handling | Model output into HTML, queries, or commands | LLM05 |
| Information disclosure | System prompt and data leakage | LLM02, LLM07 |
| Retrieval layer | Cross tenant retrieval from the vector store | LLM08 |
| SSRF | URL fetch features and media inputs | API7 |
Rules of engagement
- Test a staging environment with production like data that is not real customer data.
- Set spend caps on provider keys so a test cannot run up cost.
- Define which agent tools are in scope and whether testers may trigger real side effects.
- Agree how found secrets and personal data are handled.
- State the retest policy up front.
Ask for the right output
A useful report lists each finding with severity, steps to reproduce, affected component, and a fix suggestion. Ask the tester to separate model behavior findings from conventional ones, since they get fixed by different owners. Turn each finding into a regression test. See pentest scoping checklist.
What a pen test does not replace
It is a point in time check. Keep scanning dependencies and images continuously, since new advisories appear after the test ends.
Try it yourself
Open the web app scan demo to see how an automated test reports issues, then compare that with what a human tester adds for business logic and model behavior.
Click through it at your own pace, or open it full screen.
A short note on Safeguard
Safeguard offers web and API scanning plus dependency scanning to run between pen tests. Try a free scan at gold.safeguard.sh/scan.
Quick check
Do we need a pen test before our first enterprise deal?
Some buyers ask for a summary. Check the questionnaire, and have a scoped test planned if you do not yet have one.
Can an automated scan replace a human tester for prompt injection?
Automated tools help with breadth, but novel injection paths and business logic still benefit from a human.
Sources
Last reviewed: 19 October 2026.
