AI Security

Fix Explanation Quality: Griffin AI vs Mythos

A remediation PR explanation is either evidence or storytelling. Griffin AI attaches taint paths and disproof attempts; Mythos-class tools attach plausible prose.

Shadab Khan
Security Engineer
7 min read

The explanation attached to a remediation PR is not decoration. It is the document a reviewer uses to decide whether to merge, to push back, or to escalate. A good explanation turns a diff into a justified change. A bad one turns it into a guessing game.

Griffin AI's explanations are structured evidence. Mythos-class pure-LLM tools produce fluent prose that often reads well and rarely stands up to inspection. The difference matters in three practical ways: merge rate, incident rate, and audit posture.

What an explanation has to contain

A reviewer looking at a remediation PR needs four things from the explanation: what was vulnerable, how the attacker reaches it, why this change blocks the attack, and what the change does not touch.

What was vulnerable includes the advisory identifier, the affected code location, and the class of issue. This is the baseline that a PR title alone does not carry.

How the attacker reaches it is the taint path. A clean explanation starts at an attacker-controlled source, walks forward through the intermediate frames, and ends at the vulnerable sink. Each step should be supported by call graph evidence, not asserted.

Why this change blocks the attack is the core of the justification. It should connect the lines modified in the diff to the specific step in the taint path that the modification neutralizes. It should also state whether the block is defensive, such as adding a guard, or transformational, such as upgrading a dependency.

What the change does not touch is the scope statement. It explicitly says that unrelated functionality was preserved, that adjacent code was not modified, and that the diff is minimal. This is what tells a reviewer the change is surgical rather than speculative.

How Griffin AI structures explanations

Griffin explanations are generated from the same grounded context the patcher uses. The taint path is not a narrative. It is a list of frames pulled from the call graph, each with a link to the file and line. The exploit hypothesis is stated as a testable claim: given this source, this precondition, and this sink, an attacker can trigger this behavior.

The disproof attempt is attached next. Griffin enumerates the project's own defensive mechanisms that could already block the exploit and reports which ones were checked and what was found. If a sanitizer covered the path, the PR is not opened and the finding is closed. If the sanitizers did not cover it, the gap is stated explicitly.

The fix description connects the diff to the gap. It references the lines modified and the mechanism by which they close the gap. The scope statement lists the files touched and why, and notes the absence of changes to other candidate sites with reasons.

The result is an explanation a reviewer can verify line by line. The evidence is internal to the PR.

How Mythos-class tools approach explanations

Pure-LLM remediation tools in the Mythos class generate explanations as a second prompt: given the diff and the advisory, write a rationale. The output is fluent, well-structured, and often cites the right CVE.

What it does not contain is ground truth. There is no list of call graph frames because the tool does not have a call graph. There is no disproof attempt because the tool has no mechanism to test a hypothesis against the code. The scope statement, if present, is asserted rather than verified.

A reviewer reading the explanation sees words that sound right. The words often are right. They also sometimes refer to functions that do not exist in this codebase, assert guards that are not actually present, or describe a mechanism the diff does not implement. The errors are not obvious unless the reviewer is familiar with the code, which in a remediation workflow is often not the case.

The specific failure modes

Three patterns recur in pure-LLM explanations.

The first is the invented call path. The explanation walks a taint path that sounds plausible and refers to functions by names close to but not identical to names in the repository. The reviewer, looking at the diff alone, cannot easily tell whether the path described is real.

The second is the borrowed defense. The explanation claims the change adds sanitization, and the diff adds a call to a function that looks like a sanitizer but is actually a logger or a formatter. The name fooled the model, and the explanation inherited the error.

The third is the overclaim. The explanation states that the change closes all instances of the vulnerability, when in fact the diff only closes the one site the model saw. Other sites in other files remain exploitable. The explanation's confidence is not supported by the diff's coverage.

Griffin's grounded pipeline eliminates all three because the pipeline generates explanations from real artifacts: real call graphs, real defensive mechanism catalogs, real site coverage from taint analysis. The model contributes phrasing, not facts.

Audit consequences

Explanation quality matters beyond the immediate merge decision. A remediation PR is part of the audit trail for how a vulnerability was handled. Auditors reading these PRs months later need to reconstruct the decision.

A Griffin explanation holds up in audit because it contains verifiable artifacts. An auditor can check the taint path against the code at the time of the fix, verify the disproof attempt, and trace the scope statement against the repository history.

A Mythos-class explanation holds up in audit only as far as its prose. When an auditor asks for evidence of the taint path, the prose is the only artifact. When the prose contains an invented call or a borrowed defense, the audit finding becomes a gap.

Teams in regulated industries feel this acutely. Explanation quality is not a developer experience concern for them. It is a compliance obligation.

Signal versus style

It is easy to confuse explanation style with explanation quality. Pure-LLM tools produce prose that reads professionally. Griffin explanations can look denser and more structured, with numbered frames and explicit hypothesis statements.

The density is the point. A reviewer who wants to merge quickly needs signal, not narrative. Fewer words that point to verifiable artifacts are more useful than more words that describe general principles. Teams that try both styles tend to prefer the dense, structured format after a few weeks because it surfaces the decision-relevant information faster.

What to evaluate

When comparing explanations from different tools, do not read them for fluency. Read them for verifiability. For each claim in the explanation, ask whether an artifact attached to the PR supports the claim. If the explanation says a taint path exists, is the path listed with file and line references? If it says a defense was considered, is the defense named? If it says the scope is minimal, is the scope stated and justified?

A tool whose explanations pass this test is a tool that can run at scale. A tool whose explanations fail it is a tool whose output has to be re-verified by a human on every PR, which erodes the productivity gain the tool was supposed to provide.

The structural difference

Explanation quality reflects pipeline architecture. A grounded pipeline produces explanations full of real artifacts because real artifacts are available upstream. A pure-LLM pipeline produces explanations full of plausible prose because plausible prose is what it can generate from the inputs it has. Griffin AI sits on the first side of that line by construction, and Mythos-class tools sit on the second for the same reason.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.