To judge malicious package detection, ask which ecosystems it covers, how it reaches a verdict, how quickly you learn about a new malicious release, whether it can block an install, and how you handle false positives. Insist on evidence behind each verdict and a machine readable feed you can audit.
Why is this different from vulnerability scanning?
A vulnerability is an honest bug that someone found later. A malicious package is built to harm. Legacy software composition analysis was designed around the first case, matching versions against a list of known CVEs, so it can say nothing about a package that was never a vulnerability record. Detection of malicious packages needs its own data and its own methods.
That is why the question list matters. Two products can both say "malicious package detection" and mean very different things.
What are the ten questions?
- Which ecosystems and artifact types are covered? The OSV schema defines ecosystem values for npm, PyPI, Maven, NuGet, crates.io, Go, RubyGems, GitHub Actions, VSCode extensions and many others. Ask for the exact list you depend on, including CI actions and editor extensions, not only language packages.
- How is a verdict made? Static analysis of code, execution in a sandbox, behavior signals such as new network calls or install scripts, publisher reputation, or a mix. Ask for the mix and what each part catches.
- What is the evidence for a single verdict? A good report shows what the package does, not just a score. You need this to decide quickly and to explain a block to a developer.
- How fast do you learn of a new malicious release? Ask for the measured time from publication to your alert, and how it is measured. Be skeptical of an answer without a method.
- Can it block an install, or does it only alert? Detection after install is a different outcome from refusing the install. Ask where the control sits: developer machine, CI, proxy or registry.
- How are false positives handled? Ask how a developer disputes a block, how long a review takes and who can approve an exception.
- Do you publish findings in an open format? The OpenSSF Malicious Packages repository publishes reports in the OSV format, and its aim is to give the community and researchers shared data. A vendor that can import and export in that format makes you less dependent on any one feed.
- How do you handle typosquats and dependency confusion? Ask for how lookalike names and internal names are checked.
- Where does the analysis run and what leaves our network? Some teams need to run it locally or on their own infrastructure. Ask for the data flow in writing.
- What does it miss? The honest limits of the method. A vendor that cannot name any is not telling you something.
What are the honest limits?
No detector catches everything. A package can behave well in a sandbox and misbehave later, and a new technique can pass until it is seen once. Detection narrows the window. It does not remove the need for the basics: pin versions, use lockfiles, review new dependencies and keep install time privileges low.
GitHub's own guidance for Actions shows the same thinking in a related area. It advises pinning actions to a full length commit SHA, noting that a tag can be moved or deleted, and auditing the source of any action you use.
What to do
- List the ecosystems you depend on, including CI actions and editor extensions.
- Send the ten questions to each vendor and ask for answers in writing.
- Run a test using a package you know is on a public malicious list, and see where in your pipeline it is stopped.
- Time the full path from a new report to a developer seeing a block.
- Decide who approves exceptions and write it down before rollout.
- Re-ask question four every six months, since attackers change methods.
Where Safeguard fits
You can try a free in-browser scan of up to 100 dependencies at gold.safeguard.sh/scan. Use it to see what a dependency check looks like on your own manifest, then apply the ten questions above to any tool, ours included.
Quick check
Is a CVE scanner enough to find malicious packages?
No. A malicious package usually has no CVE, so a version match against known vulnerabilities finds nothing. You need dedicated detection data and methods.
What is the single most useful question?
Ask what the product misses. A specific, honest answer tells you more than any coverage claim.
Sources
- OpenSSF, Malicious Packages repository
- OSV schema, ecosystems and identifiers
- GitHub Docs, Secure use reference for Actions
Last reviewed: 18 October 2026.
