The npm registry now hosts more than 3.5 million packages, and a typical mid-size JavaScript application pulls in 800-1,200 transitive dependencies it never directly chose. Since 2021, tools like Socket.dev have popularized the idea of a single "package health score" — a number meant to tell a developer, in the two seconds before npm install, whether a dependency is safe. It's a compelling pitch: reduce an entire supply chain risk assessment to a traffic-light number between 0 and 100.
But the methodology behind these scores matters more than the score itself. A package can score 95/100 on maintenance activity and popularity while shipping a malicious postinstall script added six hours ago. This post breaks down how npm health and quality scores are actually calculated, where Socket.dev's model excels and where it falls short, and what a more complete scoring approach — the one Safeguard builds toward — needs to account for.
What is an npm package health score, exactly?
An npm package health score is a composite number, usually 0-100, that blends maintenance activity, popularity, and code-level risk signals into a single ranking. Most scoring engines pull from three buckets: quality (test coverage, linting, documentation, README completeness), maintenance (commit frequency, release cadence, open issue ratio, time since last publish), and popularity (weekly downloads, GitHub stars, dependent package count). npms.io pioneered this three-pillar model around 2017, and it still weights npm's own registry metadata heavily — a package last published 14 months ago with 40 open issues scores lower on maintenance regardless of whether its code is safe. That's the core tension: "health" in this original sense measures software engineering hygiene, not security. A well-maintained package updated weekly by five active contributors can still contain a critical vulnerability, and an abandoned single-maintainer utility with 200 million weekly downloads (like several is-* or lodash.* micro-packages) can be perfectly safe despite scoring poorly on maintenance.
How does Socket.dev calculate its Socket Score?
Socket.dev's score is built from roughly 70+ static analysis detections grouped into categories like Supply Chain Risk, Vulnerability, License, Quality, and Maintenance, each scored 0-100 and then combined. Founded in 2022, Socket differentiates itself from npms.io-style scoring by running static analysis on package source and behavior at publish time — flagging install scripts, use of eval(), filesystem or network access, obfuscated code, and telemetry collection. This is a meaningful advance: it looks at what code does, not just how often it's committed. Socket assigns letter-style category grades and an overall score, and it will flag a package red for something as simple as a preinstall script that shells out to curl a remote URL. The catch is that static, signature-style detection is inherently reactive — Socket's own blog has documented catching malicious packages within hours of publication, but "hours" is still a window during which a CI pipeline can pull the poisoned version. In fast-moving incidents, hours is exactly when the damage happens.
Why did compromised packages still look "healthy" right before they were exploited?
Because health and popularity scores are trailing indicators, and several of the worst npm supply chain incidents happened to trusted, high-scoring packages the moment they were compromised, not gradually. event-stream had accumulated over 1.9 million weekly downloads and years of clean history before a new "maintainer" added a malicious dependency (flatmap-stream) in November 2018 to target a Bitcoin wallet app — it sat in the dependency tree for roughly two months before detection. ua-parser-js, downloaded over 7 million times weekly, was compromised in October 2021 when an attacker gained access to the maintainer's npm account and pushed versions containing a cryptominer and password stealer directly to the registry. In both cases, every registry-metadata signal — download count, star count, release history — looked excellent right up to the compromised version. A score built primarily on historical reputation cannot flag a reputation being actively abused; it needs to evaluate the specific version being installed, not the package's brand.
What actually predicts supply chain risk versus just measuring code quality?
Behavioral and provenance signals predict risk far better than commit cadence, and the industry's shift toward this since 2023-2024 reflects lessons from real incidents. OpenSSF Scorecard, for comparison, runs 18 automated checks (branch protection, signed releases, dependency pinning, fuzzing, SAST usage) and produces a 0-10 score per repository — useful, but it scores the source repo, not the published tarball, and those two can diverge when a maintainer's local machine or CI token is compromised and a malicious version is published without a matching commit. The March 2024 xz-utils backdoor (CVE-2024-3094) is the clearest example: the malicious code was hidden in build scripts and test artifacts, not in reviewable source diffs, specifically to defeat exactly this kind of static and repo-level scrutiny. Real risk prediction requires diffing what's published against what's in source control, checking for install-time network calls, and tracking maintainer account changes (new publisher, new 2FA registration, ownership transfer) as first-class risk events rather than footnotes.
Can a health score catch a live, in-progress supply chain attack?
Rarely, and two 2024-2025 incidents show why. When the polyfill.io CDN domain was sold to a new owner in February 2024 and began injecting malware into responses served to over 100,000 sites by June 2024, no npm package health score moved at all — the compromise happened at the CDN/domain layer, entirely outside package registry metadata. Similarly, the March 2025 tj-actions/changed-files GitHub Action compromise (CVE-2025-30066) exfiltrated CI secrets from an estimated 23,000 repositories by tampering with a git tag to point to malicious code, a supply chain vector that neither npm scoring nor most SCA tools were built to watch. A static health score, refreshed periodically and scoped to registry packages, structurally cannot cover CDN dependencies, GitHub Actions, or build-time tag manipulation. Treating a single npm score as a supply chain risk answer leaves these entire attack classes uncovered.
How Safeguard Helps
Safeguard treats "package health" as one input into supply chain risk, not the whole answer, because the incidents above show that reputation-based scoring alone misses exactly the attacks that matter most. Safeguard continuously monitors the full dependency graph — not just direct npm dependencies but transitive packages, CI/CD actions, and build pipeline components — and correlates behavioral signals (new install scripts, unexpected network calls, sudden maintainer or publisher changes, ownership transfers) with source-to-publish diffing, so a version that doesn't match its tagged commit gets flagged before it reaches a build.
Rather than a single static number recalculated on a schedule, Safeguard evaluates the specific version pinned in your lockfile at the moment it changes, and re-evaluates on every dependency update, not just on a periodic crawl. That closes the "hours of exposure" window that affects reactive scanners: a compromised publish is assessed against behavioral and provenance signals as soon as it enters your build, rather than waiting for a signature or community report to catch up.
Safeguard also extends coverage beyond the npm registry itself to the adjacent surfaces attackers have increasingly targeted — GitHub Actions and CI/CD dependencies (the class of attack behind tj-actions/changed-files), third-party script and CDN dependencies (the class behind polyfill.io), and build artifacts versus source diffs (the class behind xz-utils). For teams that have relied on a single package health score as their supply chain security control, that's the practical gap: a health score tells you about a package's history, while Safeguard is built to catch the moment history stops predicting behavior.
If your team is currently gating dependency approvals on a single vendor's score, it's worth asking what happens in the hours between a compromise and its detection — and whether your current tooling watches anything outside the npm registry at all. That gap is exactly where Safeguard is designed to sit.