In March and April 2023, engineers at Samsung's Device Solutions division in Hwaseong, South Korea, pasted proprietary semiconductor source code, internal test sequences, and confidential meeting notes into ChatGPT — three separate incidents in the space of about 20 days after the company lifted an internal ban on generative AI tools. Samsung's response, documented by Forbes and logged as Incident #768 in the AI Incident Database, was to ban generative AI outright company-wide by May 2023. That single case captures the core tension every engineering org now faces: AI coding assistants measurably speed up development, but the same paste-in, autocomplete-everywhere workflow that makes them useful also creates three distinct new risk surfaces — secrets leaking into prompts, licensed code leaking into your codebase, and code provenance you can no longer trace to a human author or a verified source. None of these risks are hypothetical, and none require exotic tooling to manage. This guide walks through what's actually happened, what the standards bodies recommend, and what controls close the gap — grounded in OWASP's LLM Top 10, NIST's AI Risk Management Framework, and the SLSA supply-chain framework.
How does sensitive data actually leak through AI coding assistants?
It leaks the same way Samsung's did: through the prompt itself. When a developer pastes a stack trace, a config file, or a "why is this failing" snippet into a chat-based assistant, anything embedded in that text — API keys, database connection strings, internal hostnames — travels to the model provider's infrastructure as plain input. Unlike a git commit, a prompt isn't scanned by any pre-commit hook by default; it's just text in a chat window. The Samsung incidents involved source code and equipment measurement data, not credentials specifically, but the mechanism is identical for secrets: an assistant with autocomplete access to your working directory, or a developer copy-pasting a .env file to "help debug" a connection issue, exposes whatever's in that context window. The fix is to treat AI prompts as an untrusted egress channel and scan for secrets before they leave the machine — via pre-commit and pre-push hooks — rather than relying on developers to self-censor under deadline pressure.
Why does AI-generated code create license contamination risk?
Because code-completion models are trained on public repositories, and a suggestion can reproduce a licensed snippet without carrying its license or attribution along with it. GitHub has stated that exact matches to training data occur in under 1% of Copilot suggestions, but "under 1%" across millions of daily suggestions is still a meaningful volume, and the suggestion arrives in your editor with no SPDX tag, no NOTICE entry, no indication it originated in a GPL-licensed project. This gap underpinned Doe v. GitHub, a class action filed in the Northern District of California in November 2022 alleging Copilot's output violated open-source license and attribution terms. GitHub responded with a duplication-detection filter in 2022 that blocks suggestions longer than 150 characters matching public code, and a 2023 "code referencing" feature that surfaces the matching public source and its license when a suggestion is used. The practical takeaway for engineering teams: don't assume assistant output is license-clean by default, and don't rely on the vendor's filter as your only control — enforce your own allow/deny license policy at the point code enters a PR.
What is "slopsquatting" and why does it matter for provenance?
Slopsquatting is what security researchers call it when an AI coding assistant hallucinates a plausible-sounding but nonexistent package name in a dependency list, and an attacker registers that exact name on PyPI or npm preloaded with malware — so the next developer whose assistant "remembers" the same hallucination pulls in an attacker-controlled package via a normal pip install or npm install. It's a direct descendant of dependency-confusion attacks, and it matters because it breaks the basic assumption that a human reviewed and intentionally chose every line in a manifest file. When an assistant proposes a new dependency, that proposal needs the same provenance check a human-added dependency gets — resolved version, publisher identity, and signed build attestation — not a rubber stamp because it appeared in a diff.
What does NIST and OWASP guidance actually recommend for AI-generated code?
Both frameworks converge on one principle: treat AI output as untrusted input requiring the same gates as code from an unknown external contributor, not as trusted first-party work. OWASP's LLM Top 10 categorizes insecure output handling and supply-chain risk as top-tier concerns for LLM-integrated systems, and NIST's AI Risk Management Framework (AI RMF 1.0, published January 2023) frames this under "Manage" — requiring documented provenance and continuous monitoring of AI-influenced artifacts rather than one-time approval. Concretely, this means every AI-suggested change should pass through the same SAST, secret-scanning, and dependency-review gates a human PR does; no assistant should get a bypass lane just because a person clicked "accept." Teams that skip this step are effectively code-reviewing 1% of what they used to, because the review habit that catches a human's mistake doesn't fire when the code "looks right" and arrived instantly.
How should teams enforce provenance on AI-touched builds?
By attaching cryptographic attestation to the build pipeline itself, not to the individual suggestion — since there's no reliable way to tag which lines in a file an assistant wrote versus a human. SLSA (Supply-chain Levels for Software Artifacts) and in-toto define a standard for this: every build produces a signed provenance statement naming the source commit, the build steps, and the materials consumed, published to a transparency log via Sigstore. That gives you an answer to "was this artifact built by our pipeline from a reviewed commit" independent of whether an AI assistant touched any given line. Practically, this closes the gap that pure code-review can't: even a careful human reviewer can't verify a hallucinated dependency was never substituted between review and build without a provenance chain proving what actually shipped.
How Safeguard Helps
Safeguard applies its existing supply-chain controls directly to AI-assisted workflows rather than treating them as a separate problem. Secrets scanning covers source, git history, and — specifically relevant here — model weights and fine-tunes, verifying each finding against the issuer's API (sts:GetCallerIdentity for AWS keys, /v1/models for OpenAI keys) so a prompt-leaked credential gets caught and revoked, not just flagged; a safeguard install-hook --hook pre-push git hook blocks known secret patterns before anything leaves a developer's machine. License compliance runs SPDX-based detection on every dependency an assistant proposes, enforcing your allow/deny YAML policy at the PR and CI gate so a copyleft-licensed suggestion never merges silently. Attestation & Signing produces SLSA provenance and Sigstore-signed, Rekor-logged attestations for every build, so you can prove what actually shipped regardless of whether a human or an assistant wrote a given line. And for teams running or fine-tuning their own models alongside AI-assisted coding tools, the AI-BOM tracks weight provenance and training-data license risk with the same rigor applied to open-source dependencies — one governance model across code and models instead of two disconnected ones.