Safeguard
AI Security

A risk framework for enterprise AI coding and agent tool rollouts

Samsung banned ChatGPT company-wide after three leaks in 20 days. A working framework for data exposure, model supply chain, and access control risk.

Safeguard Research Team
Research
7 min read

In April 2023, Samsung engineers pasted proprietary semiconductor source code, internal test sequences, and confidential meeting notes into ChatGPT — three separate incidents inside a 20-day window. Samsung's response was blunt: a company-wide ban on public generative AI tools. That single case captures why enterprise AI rollouts need a structured risk framework rather than a one-line acceptable-use policy. Two years later, in February 2025, security researchers at ReversingLabs found malicious models on Hugging Face using a technique nicknamed "nullifAI" — compressing a pickle payload with 7z instead of ZIP and placing the malicious opcode early in the stream, specifically to slip past Picklescan's blacklist-based detection. And in December 2025, JFrog disclosed three Picklescan zero-days rated CVSS 9.3 (CVE-2025-10155, CVE-2025-10156, CVE-2025-10157), showing that even the tool built to catch unsafe model files could itself be bypassed. Layer on agentic coding assistants that read and act on untrusted repository content, and enterprises are now managing three distinct, compounding risk surfaces at once. This post lays out a framework built around those three pillars: data exposure, model supply chain, and access control — with the concrete incidents and numbers behind each one.

What does the data exposure risk actually look like in practice?

Data exposure risk shows up wherever a coding assistant's context window becomes a transmission channel for something that should never leave the building. The Samsung case is the clearest documented example: engineers used ChatGPT to check source code for errors and to summarize meeting notes, and in doing so sent proprietary semiconductor design data to OpenAI's servers, outside Samsung's control, in three incidents inside 20 days (reported by Forbes and logged in the AI Incident Database, entry #768). The failure mode is not exotic — it is the same accidental-exposure pattern security teams have fought for years with email and file-sharing tools, just with a new front door. What changes with AI coding assistants is scale and stickiness: every autocomplete suggestion and every "explain this function" prompt is a potential exfiltration event, and unlike a Slack message, it happens dozens of times per engineer per day without any deliberate "share" action. A framework has to treat the assistant's input stream as a DLP surface, not assume employees will self-police.

Why does the model supply chain matter as much as the code supply chain?

It matters because a model file can execute code the moment it loads, and the tooling built to catch that is demonstrably imperfect. PyTorch's default serialization format for .pt and .bin files is Python's pickle, and deserializing a pickle file runs arbitrary code as it unpickles — a well-established weakness class (CWE-502, unsafe deserialization, often paired with CWE-94, code injection), not a single patchable bug. Picklescan was built to catch this by blacklisting dangerous opcodes and module references before a model loads. But ReversingLabs' February 2025 "nullifAI" research showed models on Hugging Face evading it simply by using 7z compression instead of standard ZIP and front-loading the malicious payload in the pickle stream — Picklescan didn't recognize the archive format and never got far enough into the file to see the payload. Separately, Sonatype researchers disclosed four distinct bypass techniques of their own, and JFrog's December 2025 disclosure of CVE-2025-10155 through CVE-2025-10157 (extension evasion via non-standard file suffixes, ZIP CRC error-handling tricks, and submodule-import blacklist evasion) each independently defeated the scanner. The lesson for a framework: model files need the same provenance, signing, and pre-load static verification that source dependencies get — never trust one scanner's blacklist as the sole control.

How much does an AI coding assistant actually increase secret leakage?

The measured increase is real and non-trivial, not theoretical. One study comparing repositories with GitHub Copilot enabled against a baseline found Copilot-enabled repos leaked secrets at a 6.4% rate versus 4.6% for non-Copilot repos — roughly a 40% relative increase. A separate extraction study went further, harvesting 8,127 code suggestions from Copilot and finding 2,702 valid-looking secret patterns among them, of which 200 turned out to be live, working credentials, not placeholders. The mechanism is straightforward: models trained on public code absorb hardcoded API keys and tokens that were committed by mistake, and then reproduce similar-looking patterns — or worse, complete a half-typed real secret sitting in the current file — as a "helpful" suggestion. This is why secret-scanning in pre-commit and CI needs to be positioned as a check specifically on AI-assisted commits, since the attack surface changes from "a developer occasionally pastes a key" to "an autocomplete engine synthesizes plausible ones every session."

Can an AI agent be manipulated through the code it's asked to review?

Yes, and this is the access control risk that's newest and least understood. Reports in mid-2026 described "Comment and Control"-style attacks in which a single malicious pull request title or GitHub issue body contains instructions that an agentic coding tool — Claude Code, Gemini CLI, or Copilot-driven workflows among those named — reads as part of its normal context and follows, including instructions to locate and exfiltrate API keys or tokens from the repository or environment (reported by The Hacker News and Techzine, July 2026). The root problem is architectural: agents that read repository content to do their job cannot easily distinguish "instructions from my operator" from "text an attacker planted in an issue." Traditional access control assumes a human reviews every action before it runs; an agent with tool-calling permissions collapses that review step unless it's rebuilt deliberately — through allow-listed tool surfaces, scoped credentials per task, and logging every tool call against a declared expected set.

What should a rollout framework actually require before agents get write access?

A practical framework treats each pillar as a gate, not a checkbox. For data exposure: route assistant traffic through logging/DLP before it reaches a third-party model endpoint, and default new tools to the most restrictive data-sharing tier until reviewed. For model supply chain: require weight-file provenance and signature verification before any model — vendor or internally fine-tuned — loads into a serving environment, and don't rely on a single static scanner given the Picklescan bypass history. Safeguard's AI-BOM tooling, for instance, scans model weights for embedded executable code before load and enforces signed-weights and hash-verification policies at load time, which is the kind of control this pillar needs. For access control: scope every agent's credentials to the narrowest task-relevant permission set, log tool calls against a declared allow-list, and flag any call outside it — because "the agent read a comment and did what it said" is now a documented attack pattern, not a hypothetical one.

How Safeguard Helps

Safeguard maps directly onto the model supply chain pillar of this framework: AI-BOM discovery finds every model artifact across Hugging Face, MLflow, SageMaker, Vertex AI, Azure ML, and internal registries, and Safeguard's model loader verifies weight hashes and signatures and blocks disallowed code execution at load time — closing exactly the gap that let nullifAI-style payloads and Picklescan bypasses through in 2025. Policies can require signed weights and block any model flagged with unsafe embedded code before it ever reaches a production serving workload. The data exposure and agentic access-control pillars are organizational and architectural problems as much as tooling ones — but a complete AI-BOM inventory is the prerequisite for answering the first question any incident review asks: which models, from where, are running in your environment right now.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.