Continual Learning in Production: Feedback Loops, Forgetting and Drift

Continual learning in production means a system keeps changing after you ship it: a model retrains on new data, an online learner updates on every event, or an agent rewrites its own memory and skills. It fails in four predictable ways: feedback loops where the system shapes its own future training data, forgetting of older behaviour, drift in the data it sees, and drift in the evaluations used to judge it. The defence is the same for all four: a frozen, versioned evaluation set, drift metrics with alert thresholds, a gate before every update goes live, and a rollback you have rehearsed.

Key takeaways

• Three standard settings. Task-, domain- and class-incremental learning differ in whether the system is told which task it faces and whether it must separate new classes from old ones. Class-incremental is the hardest.
• Feedback loops are the production failure benchmarks miss. A model that updates over time influences the data it later learns from, a problem Sculley and colleagues described in 2015.
• Forgetting is measurable. EWC, replay and parameter isolation reduce it at a cost. Measure it per slice on a frozen set of old tasks.
• Drift comes in three kinds. Data, concept and label drift each need a different detector.
• The yardstick drifts too. Benchmarks leak into training data, hosted models change behind a stable name, and judge models carry biases.
• Agents now learn without touching weights. Hermes Agent from Nous Research saves memory entries and writes skill files after tasks. Its docs (checked 23 October 2026) ship write approval gates for both, off by default.
• Regulation names this. EU AI Act Article 15(4) requires high-risk systems that keep learning to address feedback loops; NIST AI RMF asks you to monitor behaviour in production and plan recovery.

What is continual learning?

Continual learning, also called lifelong or incremental learning, is learning from a stream of data whose distribution changes over time while keeping what was learned before. Wang, Zhang, Su and Zhu's 2023 survey (arXiv 2302.00487) frames it as the ability to incrementally acquire, update, accumulate and exploit knowledge over a system's lifetime, and names catastrophic forgetting as its main limit.

In production it takes three forms:

1. Scheduled retraining. A model is retrained nightly or weekly on a rolling window, evaluated and swapped in. Most teams do this without calling it continual learning, yet it has every failure mode below.
2. Online learning. The model updates on each event or small batch. Click prediction, fraud scoring and spam filtering have worked this way for years because the world moves faster than a weekly retrain.
3. Context-level learning. A language model agent keeps its weights fixed but writes notes, memories and procedures that load into future prompts. Nothing is retrained, yet behaviour changes between sessions.

What they share: the thing you evaluated last month is not the thing running today.

Task-, domain- and class-incremental learning

Van de Ven and Tolias proposed three scenarios in 2019 (arXiv 1904.07734) because papers were reporting results under incompatible protocols. The expanded paper, "Three types of incremental learning" by van de Ven, Tuytelaars and Tolias, appeared in Nature Machine Intelligence on 5 December 2022.

Using Split MNIST and Split CIFAR-100, the authors report substantial differences in difficulty and in which strategies work across the three. If your system adds categories over time, you are in the class-incremental setting whether you planned for it or not.

How online learning works in classic production systems

Recommenders and ad ranking. The model predicts clicks; impressions and clicks are logged and fed back into training. The model's own choices decide which items get impressions, and so which examples get labels.

Fraud scoring. Transactions above a threshold are blocked or reviewed. Reviewed ones get labels quickly, approved ones weeks later through chargebacks, blocked ones perhaps never. The model shapes which labels it sees.

Spam filtering. Users correct the filter's choices, so labels come mostly from what the filter already did, while senders adapt to whatever it lets through.

In all three, tomorrow's training data is partly produced by today's model.

What "Hidden Technical Debt in Machine Learning Systems" warned about

Sculley, Holt, Golovin and seven Google colleagues published this paper at NIPS 2015 (Advances in Neural Information Processing Systems 28). Four of its warnings matter most here.

Direct feedback loops. A model influences the selection of its own future training data. The authors note that bandit algorithms are the theoretically correct tool but often do not scale, and suggest some randomisation or isolating part of the data from the model's influence. In practice: a small random holdout of traffic where the model's decision is not applied, so you always have unbiased labels.

Hidden feedback loops. Two systems influence each other indirectly through the world, such as one component choosing products and another choosing reviews on the same page. The authors note these loops can link entirely separate systems.

CACE. "Changing Anything Changes Everything": no input in a learned system is truly independent, so every automatic update is a change to the whole system.

Fixed thresholds and monitoring. A retrained model can silently invalidate a hand-tuned threshold. The paper recommends live monitoring with automated response, starting from prediction bias (predicted label distribution should match observed), action limits, and monitoring of upstream data producers.

Those ideas map onto the setup at the end of this post: prediction bias becomes a label drift check, action limits become thresholds on what an agent writes, and upstream monitoring becomes input drift checks.

What is catastrophic forgetting, and how do EWC, replay and parameter isolation address it?

Catastrophic forgetting is the drop in performance on earlier tasks when a network trains on new ones: gradient descent has no reason to protect weights that mattered for data it no longer sees. We cover forgetting after a single fine-tune in catastrophic forgetting after fine-tuning; here the focus is systems that keep learning.

Regularisation: elastic weight consolidation. Kirkpatrick and colleagues at DeepMind introduced EWC in arXiv 1612.00796 (2 December 2016). After task A, it estimates each weight's importance from the diagonal of the Fisher information matrix; while training task B, a quadratic penalty pulls important weights back towards their task A values. The authors demonstrated it on sequences of MNIST tasks and on learning several Atari 2600 games in turn. It needs no stored old data, but the penalty is an approximation and it was shown on sequences with clear task boundaries.

Replay: rehearse old data. Keep a buffer of past examples and mix them into new training. Rolnick and colleagues (arXiv 1811.11682, November 2018) report that replay buffers for all past events substantially reduce forgetting in Atari and DMLab reinforcement learning tasks without task identities, and that a randomly discarding, size-limited buffer performs almost as well as an unbounded one. Gradient Episodic Memory (GEM, Lopez-Paz and Ranzato, arXiv 1706.08840) uses stored examples as constraints so an update may not increase loss on earlier tasks. iCaRL (Rebuffi and colleagues, arXiv 1611.07725) keeps exemplars per class for the class-incremental setting, tested on CIFAR-100 and ImageNet. Replay is the most common production answer; its costs are storage, retention rules for old personal data, and choosing what to keep.

Parameter isolation. Progressive Neural Networks (Rusu and colleagues, arXiv 1606.04671) add a new column per task, freeze old columns and connect them laterally; the authors describe them as immune to forgetting. PackNet (Mallya and Lazebnik, arXiv 1711.05769) prunes a network to free weights for the next task and reports adding three fine-grained tasks to one ImageNet-trained VGG-16. Production analogues are per-task adapters, per-tenant heads or separate models behind a router.

How to measure forgetting

The GEM paper proposes metrics for task sequences, including backward transfer: the average change in accuracy on each earlier task between when it was learned and the end of training. Negative backward transfer is forgetting. In production: keep a frozen evaluation slice for every capability you ever shipped and track each candidate's score against the score that slice had at release. Aggregate metrics hide the loss of a rare pattern.

What are data drift, concept drift and label drift, and how are they detected?

Drift is a change in the joint distribution of inputs X and labels y. Lu and colleagues' review (arXiv 2004.05785, IEEE TKDE) splits its sources into a change in P(X) alone, a change in P(y given X) alone, and a mix. Gama, Žliobaitė, Bifet, Pechenizkiy and Bouchachia's survey (ACM Computing Surveys 46(4), 2014) is the other standard reference.

Lu and colleagues also classify drift by shape: sudden, gradual, incremental and reoccurring (an old concept returning, as with seasons). A sudden drift trips a short-window test quickly; an incremental one may never cross it and needs a longer reference window.

Data drift. Rabanser, Günnemann and Lipton's "Failing Loudly" (arXiv 1810.11953) compared shift detectors and report that two-sample testing on representations from a pre-trained classifier performed best across the shifts they studied. The lesson: reduce dimensions first (model scores or embeddings), then test, with a multiple-testing correction, instead of testing hundreds of raw features. The population stability index (PSI) bins a variable and sums (live share minus reference share) times the log of their ratio. Practitioners often read below 0.1 as stable and above 0.25 as a significant shift; that is industry rule of thumb, not a standard, so calibrate it on your own history.

Concept drift. This needs labels, which usually arrive late. Lu and colleagues survey error-rate detectors such as DDM, which tracks the online error rate and raises a warning and then a drift signal as it rises, and ADWIN, which keeps a variable-length window and cuts it when two sub-windows' means differ beyond a bound. Open source stream learning libraries implement both. While labels are delayed, use the random holdout, human review samples, or disagreement with a frozen reference model as proxies.

Label drift. Lipton, Wang and Smola (arXiv 1802.03916) introduced black box shift estimation (BBSE): under the label shift assumption, a classifier's confusion matrix on held-out data lets you estimate the new label distribution from its predictions on unlabelled live data, even if the classifier is biased, as long as the matrix is invertible. It is the formal version of Sculley's prediction bias check.

What is evaluation drift, and how do you keep a frozen, versioned eval set?

Evaluation drift is when the yardstick moves. Three causes:

Contamination. When benchmark items reach training data, scores overstate ability. Sainz and colleagues (arXiv 2310.18018) argue the extent is unknown and call for measuring contamination per benchmark. Deng and colleagues (arXiv 2311.09783) propose searching pretraining corpora for overlap and probing models with masked test items. LiveBench (White and colleagues, arXiv 2406.19314) answers with frequently updated questions and objective scoring. For a system that retrains on production data, contamination can happen inside your own pipeline if the evaluation set was sampled from the same logs.

Model versions behind a name. Chen, Zaharia and Zou (arXiv 2307.09009) evaluated March and June 2023 versions of two hosted language model services on seven task types and report that behaviour "can vary greatly over time". Pin dated model versions where offered and record the exact version with every result.

Judge drift and bias. Zheng and colleagues (arXiv 2306.05685) documented position, verbosity and self-enhancement biases in language models used as judges. A judge is a dependency: pin its version and prompt, and keep a small human-labelled set to check agreement whenever it changes.

How to keep a frozen evaluation set:

1. Content-address it. Hash every file into a manifest and record the manifest hash with each result.
2. Version it in git and tag it. evals/v3 means one exact set; new items go into v4.
3. Keep it out of training data, by ID and by near-duplicate match, including any production logs it came from.
4. Slice it by capability, segment, language and rare case.
5. Freeze the scorer with the data: code, judge version, judge prompt.
6. Add a rotating live slice, human-labelled from recent traffic, and report it separately.

More on wiring this into releases: an eval harness as a release gate for AI features.

How do agent memory and skill libraries create a new kind of feedback loop?

Language model agents add continual learning with no gradient at all: the agent writes text that loads into later sessions.

Hermes Agent, Nous Research's open source agent (MIT licence; latest release v0.22.5, published 23 October 2026 per the project's GitHub releases feed, checked that day), documents this clearly. Its memory docs, read on 23 October 2026, describe MEMORY.md (agent notes, 2,200 character limit) and USER.md (user profile, 1,375 characters) under ~/.hermes/memories/, injected into the system prompt as a frozen snapshot at session start, so today's saved fact reaches the next session. Past sessions sit in SQLite at ~/.hermes/state.db and are searchable with session_search. Its skills docs say skills live in ~/.hermes/skills/, that the agent creates, patches and deletes them with skill_manage, and that the system prompt asks it to record a skill after working out a multi-step workflow, after finding a working path past errors, and after a user correction. A background review after a turn can save memories or update skills, and a curator pass consolidates skills.

That is a feedback loop in Sculley's sense: outputs become inputs to future behaviour.

Three failure modes follow. Error compounding: a wrong assumption saved as a general rule spreads to unrelated tasks. Self-confirming procedures: a skill saved after a task the agent judged successful preserves whatever it believed worked. Persistent injection: if untrusted content shapes a saved memory or skill, its influence outlives the session. The AgentPoison paper (Chen and colleagues, arXiv 2407.12784) studied backdoor attacks that poison an agent's long-term memory or retrieval knowledge base across three types of agent. We describe the class only conceptually; the defence is to treat each memory or skill write as a change that needs scanning, review and provenance.

Hermes ships such controls. Its docs say memory entries are scanned for prompt injection, credential exfiltration, SSH backdoor patterns and invisible Unicode before acceptance; memory.write_approval and skills.write_approval stage writes for review and both default to false (write freely, including from the background review); and skills.create_dir can send agent-created skills to another directory, such as a git-tracked repository. Step 8 below uses all three.

What does the EU AI Act say about systems that continue to learn after deployment?

We read these articles of Regulation (EU) 2024/1689 on the European Commission's AI Act Service Desk on 23 October 2026. The site says its text is based on the EUR-Lex consolidated version as at 27 July 2026 and marks passages amended by the Digital Omnibus on AI, so check the current consolidated text before relying on a date.

Article 15(4). High-risk systems that continue to learn after being placed on the market or put into service must eliminate or reduce as far as possible the "risk of possibly biased outputs influencing input for future operations (feedback loops)" and address such loops with appropriate mitigation. Article 15(1) requires consistent performance throughout the lifecycle, Article 15(3) requires accuracy metrics to be declared in the instructions for use, and Article 15(5) lists data poisoning and model poisoning among AI-specific attacks to address where appropriate.

Article 43(4). A substantial modification triggers a new conformity assessment, and Article 3(23) defines one as a change not foreseen or planned in the initial assessment that affects compliance or the intended purpose. For systems that continue to learn, changes pre-determined by the provider at the initial assessment and documented under point 2(f) of Annex IV are not a substantial modification. In practice: describe the learning process, its bounds and its monitoring up front.

Article 72. Providers of high-risk systems must run post-market monitoring that actively and systematically collects and analyses performance data throughout the system's lifetime, based on a plan in the technical documentation. The Service Desk shows Article 72(3) as amended, with the Commission's plan template due by 2 September 2027. See EU AI Act post-market monitoring after conformity.

These duties bind high-risk systems as the Act defines them, which most agents are not. They remain a sound checklist: name the loops, bound planned changes, monitor the rest.

What does NIST AI RMF ask you to measure?

NIST AI 100-1 (AI RMF 1.0, January 2023) is voluntary. Its Appendix B notes that AI systems may need more frequent maintenance and corrective triggers because of data, model or concept drift.

The Generative AI Profile (NIST AI 600-1, July 2024) describes model collapse as training that over-relies on synthetic data making data points disappear from a model's outputs. Under MANAGE 4.1 it suggests post-deployment monitoring procedures for confabulation and cyber risks, and under GOVERN 6.2 vendor terms covering changes over time such as fine-tuning, drift and decay.

That echoes Shumailov and colleagues (Nature 631, published 24 July 2024; first posted as arXiv 2305.17493 in May 2023), who report that indiscriminate training on model-generated content causes irreversible defects in which the tails of the original distribution disappear, across language models, VAEs and Gaussian mixture models. A system retraining on its own outputs runs a small version of that loop. The word "indiscriminate" matters; see model collapse, myth versus fact.

The papers covered, and what they did and did not show

Two cautions. Most forgetting results use task sequences with known boundaries and modest models, so measure in your own setting. The evaluation drift papers measure particular systems at particular times; they show yardsticks move, not by how much yours will.

Step by step: a monitoring setup for a system that keeps learning

This works for scheduled retraining, online learners and agents that write memory and skills. Adapt the thresholds to your data.

Step 1: Pin the evaluation harness

A silent library upgrade can change a metric. Pin exact versions with hashes.

python -m venv .venv && . .venv/bin/activate
pip install pip-tools
printf "numpy\nscipy\n" > requirements.in
pip-compile --generate-hashes requirements.in -o requirements.txt
pip install --require-hashes -r requirements.txt
git add requirements.in requirements.txt && git commit -m "Pin eval deps with hashes"

Step 2: Freeze and version the evaluation set

# freeze_evals.py: hash manifest for every evaluation slice
import hashlib, json, pathlib, sys

root = pathlib.Path(sys.argv[1])
manifest = {f.name: hashlib.sha256(f.read_bytes()).hexdigest()
            for f in sorted(root.glob("*.jsonl"))}
data = json.dumps(manifest, indent=2, sort_keys=True).encode()
(root / "MANIFEST.json").write_bytes(data)
print("manifest sha256:", hashlib.sha256(data).hexdigest())

python freeze_evals.py evals/v3
git add evals/v3 && git commit -m "Freeze evaluation set v3"
git tag -s evals-v3 -m "Evaluation set v3, frozen"

Record the manifest hash, scorer commit and judge model version in every result file.

Step 3: Exclude evaluation items from training

# exclude_evals.py: drop training rows whose normalised text matches an eval item
import hashlib, json, pathlib, re

def h(s):
    return hashlib.sha256(re.sub(r"\s+", " ", s.strip().lower()).encode()).hexdigest()

evals = {h(json.loads(l)["input"])
         for f in pathlib.Path("evals/v3").glob("*.jsonl") for l in f.open()}
dropped = 0
with open("train.jsonl") as src, open("train.clean.jsonl", "w") as dst:
    for line in src:
        if h(json.loads(line)["input"]) in evals:
            dropped += 1
        else:
            dst.write(line)
print("dropped", dropped)

A non-zero count means evaluation items found a route into training. Find it.

Step 4: Compute drift metrics

Compare a fixed reference window (the current model's training data) with a live window (say, the last 24 hours).

# drift.py: data drift (PSI, KS) and label drift (prediction bias)
import numpy as np
from scipy.stats import ks_2samp

def psi(ref, live, bins=10, eps=1e-6):
    edges = np.quantile(ref, np.linspace(0, 1, bins + 1))
    edges[0], edges[-1] = -np.inf, np.inf
    r = np.histogram(ref, edges)[0] / len(ref) + eps
    l = np.histogram(live, edges)[0] / len(live) + eps
    return float(np.sum((l - r) * np.log(l / r)))

def prediction_bias(pred, observed, classes):
    # Sculley et al.: predicted label distribution should match observed
    return max(abs(np.mean(pred == c) - np.mean(observed == c)) for c in classes)

rng = np.random.default_rng(0)
ref, live = rng.normal(0, 1, 50_000), rng.normal(0.3, 1.1, 5_000)
print("psi", round(psi(ref, live), 4), "ks", ks_2samp(ref, live))

Run KS on model scores or reduced embedding dimensions with a multiple-testing correction. For concept drift, feed per-prediction errors into DDM or ADWIN from an open source stream learning library once labels arrive.

Step 5: Gate every update per slice

# gate.py: block a release on any per-slice regression beyond tolerance
import json, sys

TOL = 0.01
base, cand = json.load(open(sys.argv[1])), json.load(open(sys.argv[2]))
fails = [f"{k}: {base[k]:.3f} -> {cand.get(k, float('nan')):.3f}"
         for k in base if k not in cand or base[k] - cand[k] > TOL]
bwt = sum(cand[k] - base[k] for k in base if k in cand) / len(base)
print(f"backward transfer vs baseline: {bwt:+.4f}")
if fails:
    sys.exit("BLOCKED\n" + "\n".join(fails))
print("PASS")

This backward transfer figure is a simplified release-to-release version of the GEM metric. Keep its history: a slow negative trend across releases that each passed is forgetting by a thousand cuts.

Step 6: Run the gate in CI

# .github/workflows/eval-gate.yml
name: eval-gate
on:
  pull_request:
    paths: ["models/current.json", "skills/**", "evals/**"]
jobs:
  gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4   # pin to a full commit SHA in production
      - uses: actions/setup-python@v5
        with: { python-version: "3.12" }
      - run: pip install --require-hashes -r requirements.txt
      - run: python verify_manifest.py evals/v3
      - run: python run_evals.py --model candidate --out candidate.json
      - run: python gate.py results/baseline.json candidate.json

verify_manifest.py recomputes Step 2's hashes and fails on any change; run_evals.py is your scorer.

Step 7: Set alert thresholds with owners

Tune each against months of history so alerts stay rare enough to be read.

Step 8: Gate and version agent memory and skills

For Hermes Agent, these keys come from its memory and skills docs as read on 23 October 2026; check the docs for your installed version.

# ~/.hermes/config.yaml (excerpt)
memory:
  write_approval: true        # docs default: false
skills:
  write_approval: true        # docs default: false; stages every skill write
  create_dir: /srv/agent-skills   # git-tracked directory for agent-created skills

Review staged writes with the documented commands:

/memory pending
/memory approve <id>
/memory reject <id>
/skills pending
/skills diff <id>
/skills approve <id>
/skills reject <id>

cd /srv/agent-skills && git init && git add -A && git commit -m "Baseline agent skills"

The docs note that existing skills are modified where they already live and that the curator and bundled sync work on the profile-local directory, so track ~/.hermes/skills/ too. Run a frozen set of agent tasks before and after approving a batch of skill changes, and reject on any per-task regression.

Step 9: Rehearse rollback

Warning: the blocks after this one replace the live model pointer or revert agent state. Run this backup first and confirm the archives are readable.

ts=$(date +%Y%m%d-%H%M%S)
cp models/current.json "backups/current.json.$ts"
tar -czf "backups/hermes-memories-$ts.tgz" -C ~/.hermes memories
tar -czf "backups/hermes-skills-$ts.tgz" -C ~/.hermes skills
tar -tzf "backups/hermes-skills-$ts.tgz" | head

Warning: this reverts the model pointer to the last release that passed the gate. Take the backup above first.

git log --oneline -- models/current.json | head -5
git revert --no-edit <commit-that-promoted-the-bad-model>

Warning: this reverts agent-created skills. Take the backup above first and stop the agent so it cannot write during the revert.

cd /srv/agent-skills && git log --oneline | head -10
git revert --no-edit <bad-skill-commit>

For memory, prefer /memory reject on staged writes or restoring MEMORY.md and USER.md from the backup over hand-editing, then start a new session, since the snapshot is frozen at session start. For online learners, keep last known good weights as an immutable artifact plus a kill switch that stops updates, which is the capability MANAGE 2.4 describes.

Common mistakes

• Evaluating only on fresh data. A rolling set hides forgetting; keep frozen slices for old capabilities.
• Letting the model choose all its labels. Without a random holdout, a fraud or ranking model learns only from cases it already acted on.
• Thresholds that never move. Re-derive them on held-out data at each release.
• Testing every raw feature for drift. Uncorrected tests produce constant false alarms that people learn to ignore.
• Treating a hosted model alias as fixed. Pin dated versions and record them with results.
• Never checking the judge. Re-check agreement against human labels whenever the judge model changes.
• Evaluation items leaking into training through shared production logs.
• Assuming agent learning needs no evaluation because no weights change. A saved skill changes behaviour as surely as a weight update.
• Leaving write approval off on shared or unattended agents. Hermes defaults both gates to off, which may suit a laptop but not a team chat or server.
• A rollback never rehearsed. It fails when needed, usually because the backup was incomplete.

FAQ

What is continual learning in machine learning?

Training a system on a stream of data whose distribution changes while keeping earlier abilities. It covers scheduled retraining, online updates per event, and agents that learn by writing memory and skill files.

What is the difference between online learning and continual learning?

Online learning describes how updates happen: one example or small batch at a time. Continual learning describes the goal: keep learning across changing distributions without losing old abilities. An online learner can forget like any other.

What is catastrophic forgetting in simple terms?

A network trained on something new gets much worse at something it used to do, because new training overwrote weights the old skill needed. EWC, replay and per-task parameters reduce it; re-testing old tasks measures it.

How do you detect data drift in production?

Compare a recent window of inputs or model scores with a reference window using PSI or a Kolmogorov-Smirnov test, correct for multiple tests, and alert when a shift persists across runs.

What is the difference between data drift and concept drift?

Data drift changes the inputs, P(X). Concept drift changes the relationship between inputs and labels, P(y given X). Data drift is visible without labels; concept drift usually needs labels or a proxy.

What is evaluation drift?

The measuring stick changing: benchmark items leaking into training, a hosted model changing behind a stable name, or a judge model's scoring shifting. A frozen, hashed, versioned evaluation set with a pinned scorer is the anchor.

Does Hermes Agent retrain its model?

Its documented learning loop works at the context level: memory entries, skill files and session search. The memory and skills docs describe no weight updates in that loop.

Can an AI agent's memory be poisoned?

Yes, as a class of attack; AgentPoison (arXiv 2407.12784) studied it. Defences are scanning writes, approval gates for memory and skills, provenance for what the agent read, versioned state and evaluation after changes.

What does the EU AI Act require for AI that keeps learning?

For high-risk systems: Article 15(4) requires reducing and mitigating feedback loops, Article 43(4) treats pre-determined documented learning changes as not a substantial modification, and Article 72 requires lifetime post-market monitoring.

How often should a production model be retrained?

When drift metrics and error-rate detectors say it no longer fits, and only released when it passes per-slice gates. A schedule without a gate automates shipping a worse model.

Where Safeguard fits, honestly

Safeguard does not monitor model drift, score evaluations or detect forgetting, and it is not a Hermes Agent product, integration or partner. The fit is the software around a learning system: training pipelines, evaluation harnesses, drift jobs, serving code and agent skill scripts are built from open source packages and images that can carry known vulnerabilities. Safeguard generates and analyses SBOMs in SPDX and CycloneDX for those repositories and images, evaluates policies and gates in CI, and supports attestation and signing with SLSA, Sigstore and in-toto, all generally available. Its MCP server, with hundreds of MCP tools, can be one of the tools an assistant uses. AI-BOM discovery is in early access.

Fix it without waiting for upstream

A continually learning system rebuilds often, and every retrain or agent skill that installs a package or writes a script pulls dependencies into code that runs in production. Those harnesses, pipelines and skill scripts are open source software that receives CVEs like any other, so the same patching discipline applies.

Whatever the vulnerability, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.

1. Autonomous remediation: pull requests, not tickets

When a fixed version exists, Safeguard's Griffin models open a tested, policy gated pull request that moves the affected dependency to the safe version. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation covers eight categories, including dependencies, licenses, security posture and findings. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Autonomous remediation ships switched off, with every category set to Manual, so your team decides how much runs on its own and what still needs a human approval.

2. Gold Open Source: the safe version in seconds, free

gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds. The safeguard-sh/gold-scan-action GitHub Action can fail a build on a known exploited vulnerability, on any known vulnerability, or never, so a version you have just removed cannot quietly come back in next week's merge.

3. Gold Registry: start from components that are already fixed

registry.safeguard.sh provides hardened, drop in replacements for upstream packages and container images. Every Gold artifact is held continuously to zero exploitable critical or high CVEs, zero malware and a completed license check, with SLSA L3 or higher provenance, a Sigstore signature and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.

When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened, compatibility tested build of a specific package. This fork and patch flow is in early access today, so ask which packages it covers before you plan around it.

One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.

See how Safeguard would help

The first demo is a real Safeguard CLI session: you ask it to fix a lodash CVE, it reads the finding and advisory, asks before each command, edits package.json to lodash 4.17.21, runs npm install and npm test with your approval, and pushes nothing. The same approval-first flow suits the repository holding your evaluation harness or drift jobs, with your frozen evaluation gate run afterwards.

Click through it at your own pace, or open it full screen.

The second demo connects Claude to the Safeguard MCP server with OAuth and shows how you choose which tools Claude can use, after installing Safeguard from Claude's connector directory. Choosing a narrow tool set is the same principle as gating an agent's memory and skill writes: decide in advance what the assistant may do.

Click through it at your own pace, or open it full screen.

The third demo sets up the Safeguard local runner on your own machine, approves it with a device code and scans from the terminal. If your training and evaluation code lives on hardware you control, this is how you would scan it there.

Click through it at your own pace, or open it full screen.

Download Safeguard at safeguard.sh/download and read the CLI docs at docs.safeguard.sh/docs/cli.

Quick check

Sources

• Kirkpatrick et al., Overcoming catastrophic forgetting in neural networks (arXiv 1612.00796)
• Wang et al., A Comprehensive Survey of Continual Learning (arXiv 2302.00487)
• van de Ven and Tolias, Three scenarios for continual learning (arXiv 1904.07734)
• van de Ven, Tuytelaars and Tolias, Three types of incremental learning (Nature Machine Intelligence)
• Rolnick et al., Experience Replay for Continual Learning (arXiv 1811.11682)
• Lopez-Paz and Ranzato, Gradient Episodic Memory (arXiv 1706.08840)
• Rebuffi et al., iCaRL (arXiv 1611.07725)
• Rusu et al., Progressive Neural Networks (arXiv 1606.04671)
• Mallya and Lazebnik, PackNet (arXiv 1711.05769)
• Sculley et al., Hidden Technical Debt in Machine Learning Systems (NIPS 2015)
• Gama et al., A survey on concept drift adaptation (ACM Computing Surveys)
• Lu et al., Learning under Concept Drift: A Review (arXiv 2004.05785)
• Rabanser, Günnemann and Lipton, Failing Loudly (arXiv 1810.11953)
• Lipton, Wang and Smola, Detecting and Correcting for Label Shift (arXiv 1802.03916)
• Sainz et al., NLP Evaluation in trouble (arXiv 2310.18018)
• Deng et al., Investigating Data Contamination in Modern Benchmarks (arXiv 2311.09783)
• White et al., LiveBench (arXiv 2406.19314)
• Chen, Zaharia and Zou, How is ChatGPT's behavior changing over time? (arXiv 2307.09009)
• Zheng et al., Judging LLM-as-a-Judge (arXiv 2306.05685)
• Shumailov et al., AI models collapse when trained on recursively generated data (Nature)
• Shumailov et al., The Curse of Recursion (arXiv 2305.17493)
• Chen et al., AgentPoison (arXiv 2407.12784)
• AI Act Service Desk, Article 15
• AI Act Service Desk, Article 43
• AI Act Service Desk, Article 3
• AI Act Service Desk, Article 72
• NIST AI 100-1, AI Risk Management Framework 1.0
• NIST AI 600-1, Generative AI Profile
• Hermes Agent docs: Memory
• Hermes Agent docs: Skills
• Hermes Agent releases

Last reviewed: 23 October 2026. Hermes Agent ships several releases a week and the AI Act text is being amended; check the linked docs and current consolidated text before relying on any default, key or date above.