An AI package confusion tabletop tests what happens when a coding assistant suggests a package name that did not exist, someone registers it with malicious code, and a developer installs it. This script runs 45 minutes with three injects and a fictional package, acme-fastjson-utils.
The risk in one paragraph
Language models sometimes suggest package names that look plausible but are not real. If an attacker registers such a name, anyone who installs without checking gets the attacker's code. This is a form of the lookalike risk SLSA lists under threat H, using a compromised package, which covers tricking users into a lookalike package. The drill is not about blaming the tool. It is about whether your process checks names before installs happen.
Setup
- Facilitator, scribe, a developer who uses an assistant, a platform engineer, a security lead.
- Premise: a developer accepted an assistant's suggestion and ran an install. The package is new, has a handful of downloads and a postinstall script.
Injects
Inject 1, minute 5. A review comment says "I do not recognise this dependency." The pull request adds acme-fastjson-utils.
Ask:
- Who checks new dependencies in review, and against what?
- What signals would you look for: age, maintainers, download history, repository link?
- Is there a rule requiring approval for new packages?
Inject 2, minute 18. The package was already installed on the developer's laptop and in a preview build yesterday.
Ask:
- What could the install script have read on a laptop, and in CI?
- How do you search machines and pipelines for the name?
- Which credentials do you rotate first?
Inject 3, minute 32. Three other developers say the assistant suggested the same name to them.
Ask:
- How do you tell the team without causing panic?
- Do you block the name at your registry proxy?
- Do your assistant settings allow commands that install packages without human approval?
Controls to discuss
| Control | What it does |
|---|---|
| Registry proxy with an allow list | Blocks names nobody approved |
| New package review rule | Forces a human check on unfamiliar names |
| Install scripts disabled by default | Limits what a bad package can do on install |
| Assistant permissions | Requires approval before shell commands run |
| Lockfile only installs in CI | Prevents surprise additions |
Which of these exist today? Record the answer for each. See also prompt injection policy controls for coding agents and how to check a new dependency during pull request review.
Debrief
Choose three fixes and record them in an after-action report. A good minimum is a new package rule, install script limits and assistant command approval.
Safeguard provides dependency scanning, policies and gates, and MCP Guardrail for agent tool use, which can enforce some of these checks. A free scan shows the packages your project already depends on.
Try it yourself
Click through it at your own pace, or open it full screen. Watch how an assistant gets a check on a dependency before it is used, then compare with your own workflow.
Quick check
Is this scenario realistic or theoretical?
It is a plausible attack path with a simple cause, an unchecked name, so it is a good rehearsal even if you have not seen it yourself.
Should we ban AI assistants?
Banning is rarely necessary. Add review, registry controls and command approval so a wrong suggestion cannot run unattended.
Sources
Last reviewed: 19 October 2026.
