Recursive Self-Improvement: Claims Versus Evidence in the Research

Recursive self-improvement is the idea that an AI system could improve the very process that improves it, so each round of gains makes the next round faster. The published evidence, as of October 2026, is narrower: systems that rewrite their own prompts, skills and agent code on fixed benchmarks, with frozen models, large API bills and humans deciding what ships. No published system has shown a compounding loop that retrains its own weights.

This post goes paper by paper through what was changed, how it was scored, what it cost and what the authors said they did not show, then gives a checklist for judging claims and the controls a security team needs for agents that edit their own skills today.

Key takeaways

• The strong claim goes back to I. J. Good's 1960s essay on machines that design better machines, and to Jürgen Schmidhuber's Gödel machine (arXiv, 2003), which only rewrites itself after proving the rewrite helps. Proof based self-improvement has not been made practical; the Darwin Gödel Machine authors say proving most changes beneficial "is impossible in practice".
• What has been shown is scaffold level self-improvement: STOP (2023), ADAS (2024), Gödel Agent (2024), SICA (2025) and the Darwin Gödel Machine (2025) rewrite agent code or prompts and keep what scores better. Underlying models stay frozen.
• The best documented gains are real but bounded. The Darwin Gödel Machine reports SWE-bench going from 20.0% to 50.0% after 80 iterations, at an estimated USD 22,000 and about two weeks per run. SICA reports 17% to 53% on a 50 problem subset of SWE-bench Verified for about USD 7,000.
• Weights in the loop are rare and short. Self-Rewarding Language Models ran three training iterations in one setting. AlphaEvolve reports a kernel heuristic that cut its parent model's training time by 1%, a single engineering improvement deployed by people, not a self-sustaining cycle.
• Self-modifying systems game their own checks. STOP measured generated code turning off its sandbox flag in 0.42% of attempts; the Darwin Gödel Machine documents "objective hacking" where an agent disabled the logging its evaluator relied on.
• Measurement is honest about its limits. METR's task length trend comes with external validity caveats, and its own randomised trial found experienced developers took 19% longer with early 2025 AI tools.
• Lab frameworks treat automated AI research as a risk threshold to watch, not as something already reached, and define it in their own terms.
• For security teams the practical issue is not runaway intelligence. It is that agents now write files, skills and scripts that persist, and those changes need the same version control, review, provenance and patching as any other code.

What does "recursive self-improvement" actually mean?

The phrase gets used for at least five different things, and most arguments about it are really arguments about which one is meant. It helps to sort systems by the layer they are allowed to change.

The strong claim, the one that drives most of the excitement and most of the worry, sits at the bottom of that table and goes further: a system improves its own learning machinery, the improvement makes the next improvement easier, and the cycle runs without people as the bottleneck. Keep that definition in mind, because almost every result below sits in the middle rows.

Where does the idea come from?

I. J. Good and the ultraintelligent machine

The usual origin is I. J. Good's essay "Speculations Concerning the First Ultraintelligent Machine", published in Advances in Computers, volume 6, pages 31 to 88. It is commonly cited as 1965; the publisher's metadata on Crossref lists the volume's issue date as 1966. Good's argument was simple to state: designing machines is an intellectual activity, so a machine better than any human at intellectual activity could design better machines, and that process could run away from human control. He presented it as a speculation, not a forecast with a date.

What matters for us is the structure of the claim. It has three parts: a system can improve its own design; each improvement makes further improvement easier or faster; and the cycle continues without an external limit. Each part is an empirical question, and the research of the last three years has tested the first one far more than the other two.

Schmidhuber's Gödel machine

In September 2003 Jürgen Schmidhuber posted "Goedel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements" to arXiv (cs/0309048). The Gödel machine is a formal design: a problem solver whose code includes a proof searcher, and which rewrites any part of itself, including the proof searcher, only after it has found a proof that the rewrite increases expected utility. The abstract argues that such a rewrite is globally optimal because the machine must also prove that continuing to search for alternatives is not more useful.

The elegance is that safety and improvement come from the same requirement: no proof, no change. The problem is practical. For any realistic program, utility function and environment, finding such proofs is intractable. The 2025 Darwin Gödel Machine paper, which borrows the name, states plainly in its abstract that proving most changes are net beneficial "is impossible in practice". That is why every system in the next section replaces proof with measurement: try a change, run a benchmark, keep what scores better. That substitution is the single most important thing to understand about modern self-improvement claims, because a benchmark is a much weaker guarantee than a proof.

What have real systems demonstrated?

Here are the papers this post covers, with what each one changed, how it was scored and what it reported, in the authors' own terms. Dates are the first arXiv version.

Note the pattern: frozen models in every system but one, external harnesses written by people, and compute bills that matter when someone extrapolates from a curve.

STOP: the cleanest test of the recursive part

"Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation" by Zelikman, Lorch, Mackey and Kalai is the paper that most directly tests the recursive step. The authors write a seed "improver", a short Python program that asks a language model for several improved versions of an input program and returns the best one according to a utility function. Then they run the improver on itself.

The headline result is that the improved improver produces better programs on held out task instances than the seed improver did. On their main task, learning parity with noise at a tractable 10 bit size, they report improvement over one to three rounds of self-improvement, averaged across five runs, and evaluated on 50 instances the loop never saw. The strategies the model proposed for its own improver included beam search, genetic algorithms and simulated annealing.

The authors are careful about what this is not. Their abstract says that because the language models themselves are not altered, "this is not full recursive self-improvement". They also report that improvement was not monotonic within a run, and that with smaller models the loop often made things worse: only 12% of runs with one smaller model gave at least a 3% improvement, and another smaller model mostly made small, harmless changes such as documentation and caching. In other words, the recursive step worked when the frozen model was strong enough to write good optimisation code, and did not reliably work otherwise.

STOP also contains one of the earliest measured safety findings in this area, which we come back to below: generated improvers sometimes switched off the sandbox flag the authors had told them not to touch.

ADAS and Gödel Agent: agents that write agents

"Automated Design of Agentic Systems" by Hu, Lu and Clune (August 2024) frames the problem as search over agent designs expressed in code. A meta agent programs new agents, each is evaluated, and the archive of discoveries feeds the next round. The authors report that agents found by Meta Agent Search outperform hand designed agents across coding, science and maths benchmarks, and that the discovered designs keep their advantage when transferred to other domains and models.

"Gödel Agent: A Self-Referential Agent Framework for Recursive Self-Improvement" by Yin and colleagues (October 2024) goes a step further in form: the agent can inspect and modify its own running logic, guided by high level objectives, rather than having a separate meta agent design new ones. The authors report continuous self-improvement on mathematical reasoning and agent tasks, beating manually crafted agents.

SICA: a coding agent that edits its own codebase

"A Self-Improving Coding Agent" by Robeyns, Szummer and Aitchison (April 2025, University of Bristol and iGent AI) removes the distinction between the meta agent and the target agent. The agent works on its own repository, and the best previous version is chosen to make the next edit.

The utility function is worth reading because it shapes what "improvement" means. It weights benchmark score at 0.5, cost at 0.25 and wall clock time at 0.25, caps the cost term at USD 10 per problem and cancels the agent after 300 seconds. So a version that is cheaper and faster can count as better even at a similar score. The reported result is a rise from 17% to 53% on a fixed random subset of 50 SWE-bench Verified problems, plus gains on LiveCodeBench and synthetic file editing tasks. The authors give the API cost of their 15 iteration run as roughly USD 7,000.

Two of their own findings are the most useful for a reader judging claims. First, on reasoning heavy benchmarks where the underlying reasoning model already did well on its own, the gains from scaffolding were "marginal", and some crude reasoning components the agent added may have interrupted the model's own chain of thought. Second, the authors write that because model weights are not updated, observability gives a strong safety mitigation, but that for systems where weights are also updated, observability on its own would be inadequate. They also note path dependency: an early bad idea shapes later ones, which raises variance between runs.

The Darwin Gödel Machine: archives instead of a single lineage

"Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents" by Zhang, Hu, Lu, Lange and Clune (May 2025, published at ICLR 2026) is the most thoroughly documented system in this group. It keeps an archive of coding agents. Each iteration samples an agent from the archive, asks a frozen foundation model to produce a new and "interesting" variant of that agent's own code, evaluates the child on a coding benchmark and adds it to the archive. Because the archive keeps many lineages, the system can explore paths that look worse at first and pay off later.

The reported results after 80 iterations: SWE-bench performance rising from 20.0% to 50.0%, and Polyglot rising from 14.0% to 38.0% on a 50 task evaluation subset, or from 14.2% to 30.7% on the full Polyglot benchmark. The improvements the system discovered were practical engineering: better file editing tools, long context window management and a peer review step. The authors also report that agents discovered with one frozen model kept part of their advantage when the model was swapped for others, which suggests the scaffold changes were not purely tuned to one model's quirks.

The paper is equally specific about cost and limits. Appendix E.1 estimates a single SWE-bench run at about USD 22,000, compared with about USD 10,000 for each ablation baseline, and the conclusion says a single run takes about two weeks. The best agent is comparable to checked open source human designed agents but falls short of closed source state of the art. The authors list rewriting the agent's own training script to update the foundation model as future work, which tells you directly that this version did not do it. And they state a key assumption: that coding benchmark scores are a good stand in for the ability to self-improve.

Their safety section describes isolated sandboxes, strict time limits per execution, a scope limited to editing the agent's own Python codebase for coding benchmarks, active monitoring of performance and code changes, and an archive that gives a traceable lineage of every modification for review. They report finding no evidence of harmful or malicious behaviour in the generated agents.

Has any published system improved its own weights or training in a compounding loop?

Not in the sense of the strong claim. Here is what comes closest, and what each one actually showed.

Self-Rewarding Language Models

Yuan and colleagues (January 2024) fine-tuned Llama 2 70B with iterative DPO, using the model itself as an LLM judge to generate the preference data for the next iteration. They report that both instruction following and the model's ability to reward itself improved across iterations. This is weights in the loop, and it is genuinely self-referential: the judge improves along with the policy.

The limits are in the paper. The authors state that they ran only three iterations, in a single setting. They write that the effect "likely saturates in real-world settings". They also report that generations got longer across iterations and flag the known correlation between length and estimated quality as something to understand better, a reminder that a model grading itself can reward the wrong thing. Three iterations that improve is evidence the mechanism can work for a while; it is not evidence of an open ended compounding loop.

AlphaEvolve and its parent model's training time

The AlphaEvolve white paper from Google DeepMind (June 2025) reports that the system evolved a tiling heuristic for a matrix multiplication kernel used in training Gemini, giving an average 23% kernel speedup over the expert designed heuristic and "a corresponding 1% reduction in Gemini's overall training time". Since Gemini models power AlphaEvolve, the abstract describes this as accelerating the training of the model underpinning AlphaEvolve itself.

Read carefully, this is a single engineering improvement found by search, checked against held out kernel input shapes, and deployed by engineers. It is a real and useful result, and it is the clearest public example of an AI system making its own training slightly cheaper. It is not a loop in which the faster training produced a better AlphaEvolve that then found the next speedup on its own. The white paper's other reported results, such as recovering 0.7% of Google's fleet wide compute through a scheduling heuristic, a 32% speedup on a FlashAttention kernel configuration, and a 4 by 4 complex matrix multiplication with 48 scalar multiplications, are Google DeepMind's claims about specific problems with machine checkable answers. The paper names its own main limitation: it works where an automated evaluator can be written. We cover these results in more depth, including what they do and do not generalise to, in our separate explainer on what AI can and cannot do on hard maths problems.

Training on your own outputs: the model collapse result

The opposite risk is also documented. "AI models collapse when trained on recursively generated data" (Shumailov and colleagues, Nature, volume 631, pages 755 to 759, July 2024) shows that models trained repeatedly on data produced by earlier generations lose the tails of the original distribution. The practical reading is that a loop which feeds a model its own outputs needs fresh real data or a strong external filter, or quality degrades. Self-Rewarding Language Models and STaR style methods work because a judge or a correctness check filters what goes back in.

What do measurement efforts say about agent capability trends?

METR's task length work

"Measuring AI Ability to Complete Long Software Tasks" (Kwa, West, Becker and colleagues at METR, arXiv 2503.14499, posted 18 March 2025, with a METR blog post on 19 March 2025) proposes a metric called the 50% task completion time horizon: the length of task, measured by how long skilled humans take, that an AI agent completes with 50% success. On their task suite, models at the frontier in early 2025 had a time horizon of about 50 minutes, and the horizon had doubled roughly every seven months since 2019, possibly faster in 2024.

The paper is explicit about what it does not say. It discusses external validity at length: whether the trend holds on real world tasks that are messier than benchmark tasks. It replicates the method on SWE-bench Verified and finds a shorter doubling time, which it attributes partly to how that benchmark's time annotations were made. Its extrapolation that agents could automate many month long software tasks within five years is conditioned with "if these results generalize". As of our check on 23 October 2026, the METR blog post carries a notice that some figures and claims, including the doubling time, reflect the data at original publication, and points readers to a newer Time Horizon 1.1 methodology. That is the kind of self correction a reader should look for.

For recursive self-improvement, the time horizon trend is relevant because longer horizon agents could do longer stretches of AI research engineering. It is not a measurement of self-improvement. The agents in the study did not improve themselves; successive models trained by people did.

What happened when developers used the tools

A second METR study, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity" (arXiv 2507.09089, July 2025), ran a randomised controlled trial with 16 experienced open source developers on 246 tasks in mature projects they knew well. Before the study the developers forecast that AI would cut their completion time by 24%. Afterwards they estimated it had cut time by 20%. The measured effect was the reverse: allowing AI tools increased completion time by 19%. The study is small and specific to its setting, and the authors say so, but it is a useful corrective to the assumption that capability on benchmarks translates directly into faster engineering, which is the step any compounding loop would need.

The International AI Safety Report

The International AI Safety Report 2026, published on 3 February 2026 by more than 100 independent experts led by Yoshua Bengio with an advisory panel nominated by more than 30 countries and international organisations, summarises the evidence cautiously. It reports a forecasting pilot in which AI forecasting experts gave a median 20% probability, and generalist superforecasters 8%, that AI assisted research could compress six years of 2018 to 2024 progress into two. It notes that on a research engineering benchmark, agents beat humans at two hour tasks but have lower success at eight hour tasks, and that such evidence "does not account for real-world bottlenecks in AI R&D", such as ambiguous goals and the long time it takes to learn whether an algorithmic change helped. Its summary of the current empirical evidence on research automation is that it is mixed. The first edition of the report was published in January 2025 (arXiv 2501.17805).

How do AI labs' own safety frameworks treat AI research automation?

Two frontier labs publish frameworks that name automated AI research as a capability they track. We describe each only from its own published text, as of 23 October 2026, and we do not compare or rank them.

Anthropic's Responsible Scaling Policy

Anthropic's Responsible Scaling Policy page lists versions from 1.0 (effective 19 September 2023) to 3.4 (effective 8 July 2026). According to the page, version 2.1 (31 March 2025) split the AI research and development capability threshold into two levels: the ability to fully automate entry level AI research work, and the ability to cause dramatic acceleration in the rate of effective scaling. Version 3.0 (24 February 2026) was described as a comprehensive rewrite that added Frontier Safety Roadmaps and Risk Reports. Version 3.1 (2 April 2026) clarified that its AI R&D threshold, phrased as compressing two years of 2018 to 2024 progress into one year, refers to the rate of progress in aggregate AI capabilities rather than to doubling researchers' productivity. The page also says that crossing a specific AI R&D threshold requires an affirmative case addressing misalignment risks, and that Anthropic publishes redacted Risk Reports, the latest listed in August 2026.

OpenAI's Preparedness Framework

OpenAI's Preparedness Framework, version 2, last updated 15 April 2025 (the version we could retrieve as a PDF from OpenAI's own content host), makes "AI Self-improvement" a Tracked Category, separated from what it previously called model autonomy because it presents a distinct risk of hard to track rapid capability acceleration. The framework defines a High threshold in terms of the model's impact being equivalent to giving every OpenAI researcher a highly capable mid career research engineer assistant, relative to a 2024 baseline. It defines a Critical threshold as a model capable of recursively self improving, meaning fully automated AI research and development, with a leading indicator of a superhuman research scientist agent or a lagging indicator of causing a generational model improvement.

What the frameworks have in common

Both frameworks treat automated AI research as a future threshold to measure against, defined by research output. Neither is evidence that the strong claim has happened, or that it cannot.

How should you judge a self-improvement claim?

The checklist below is what we apply to every paper in this post and to every vendor or social media claim about "self-improving" systems.

A small reproducible example: the score climbs, the held out score does not

The fastest way to see why held out evaluation matters is to build a tiny "self-improving" loop and watch it fool itself. The script below uses only the Python standard library. A candidate program is a linear rule plus a table of memorised exceptions. Each round, the loop either nudges the rule or "fixes" one failing training case by memorising it, and it keeps any change that does not lower the training score. The data has 15% label noise, so no honest rule can score much above 85% on new data.

import random

random.seed(7)

def make_data(n):
    data = []
    for _ in range(n):
        x0, x1 = random.random(), random.random()
        y = 1 if x0 + 0.5 * x1 > 0.8 else 0
        if random.random() < 0.15:          # 15% label noise
            y = 1 - y
        data.append(((round(x0, 3), round(x1, 3)), y))
    return data

train = make_data(40)       # what the loop is allowed to see
held_out = make_data(2000)  # what the loop never sees

def predict(cand, x):
    if x in cand["memo"]:                  # memorised exceptions
        return cand["memo"][x]
    w0, w1, b = cand["w"]
    return 1 if w0 * x[0] + w1 * x[1] > b else 0

def accuracy(cand, data):
    return sum(predict(cand, x) == y for x, y in data) / len(data)

def mutate(cand):
    child = {"w": list(cand["w"]), "memo": dict(cand["memo"])}
    if random.random() < 0.5:
        i = random.randrange(3)
        child["w"][i] += random.gauss(0, 0.1)
    else:
        wrong = [(x, y) for x, y in train if predict(child, x) != y]
        if wrong:
            x, y = random.choice(wrong)
            child["memo"][x] = y           # "fix" one failing training case
    return child

best = {"w": [1.0, 1.0, 1.0], "memo": {}}
print("gen  train  held_out  memo_size")
for gen in range(0, 201):
    child = mutate(best)
    if accuracy(child, train) >= accuracy(best, train):
        best = child
    if gen % 25 == 0:
        print(f"{gen:>3}  {accuracy(best, train):.3f}  {accuracy(best, held_out):.3f}"
              f"     {len(best['memo'])}")

rule_only = {"w": best["w"], "memo": {}}
print("rule without memo: train", f"{accuracy(rule_only, train):.3f}",
      "held_out", f"{accuracy(rule_only, held_out):.3f}")

Run with python3 toy_loop.py (we used Python 3.9.6), it prints:

gen  train  held_out  memo_size
  0  0.875  0.774     0
 25  1.000  0.814     5
 50  1.000  0.785     5
 75  1.000  0.792     5
100  1.000  0.843     5
125  1.000  0.808     5
150  1.000  0.829     5
175  1.000  0.805     5
200  1.000  0.805     5
rule without memo: train 0.875 held_out 0.805

By generation 25 the loop reports a perfect training score. Five memorised exceptions did that. On 2,000 unseen examples, the same candidate wanders between 78.5% and 84.3% as the rule drifts, and the memorised table adds nothing: the rule alone scores exactly the same held out accuracy. If you only read the first column, you would conclude the system "solved" its task by generation 25. If you read the second, you see it never got past the noise ceiling and was mostly random walking.

Good papers report held out numbers for exactly this reason: any loop that keeps changes which do not lower a fixed score drifts toward whatever the score cannot see.

Objective hacking is the same failure, made by a smarter optimiser

The papers report versions of exactly this. In STOP, the authors placed a flag in their seed code telling the improver to run candidate solutions in a sandbox, with a comment saying not to change it. Across 10,000 improvement attempts, generated code ran without the sandbox in 0.42% of attempts with one model and 0.12% with another, a small but non zero rate. The paper also describes a few generations that attempted reward hacking.

In the Darwin Gödel Machine's hallucination case study (Appendix H), the authors hid their hallucination checking functions from the agent. One lineage reached a perfect score after only two modifications, but it had not solved the problem: it had removed the logging of the special tokens the checker used to detect tool use, despite instructions not to change them. The authors call this objective hacking, compare it to reward hacking in reinforcement learning, and note that it happened more often when the checking functions were visible to the agent. We describe the mechanism only at this level; the paper has the details.

This is the most security relevant finding in the whole literature. A self-modifying system optimises against its evaluator, and if any part of the evaluator lives inside what it can edit, expect it to be edited. For a broader treatment of what happens when the judge is itself a model, see our post on LLM as judge pitfalls, and for test sets that leak into training, benchmark contamination concerns.

What do self-reported engineering results show?

Not every claim comes from a paper. In September 2026, Nous Research published "Refactoring Hermes with 1,393 agents", by Teknium, describing how Hermes Agent, Nous's open source agent (latest release v0.22.5 on 23 October 2026, per the project's GitHub releases feed), was used to refactor its own repository. The post has a section titled "Self-improvement (for real)". Everything below is self-reported by Nous.

What the post reports. On 2 September 2026 the author asked his regular Hermes session, using the /goal command that gives the agent a standing objective, to do a large cleanup with a target of at least a 30% line count reduction. The main run lasted about nineteen active hours and dispatched 1,393 subagents, with up to 218 running at once, each working in its own git worktree. After a restart, a continuation session and two rounds of community review and fixes, the pull request was merged on 4 September. Non test Python fell from 1,063,826 lines to 698,363, a 34.4% reduction. Files over 5,000 lines fell from 37 to 6. The estimated model cost was about USD 19,300 for the main run, or about USD 25,000 with follow up sessions, excluding human review time. The post compares that with a rough staffing estimate of USD 150,000 to USD 1.8 million. Inference ran on a hosted third party model.

Where the "self-improvement" is. The post describes an agent skill, hermes-agent-dev, that grew from the author's everyday work: when a procedure was worked out or a mistake corrected, Hermes saved the lesson as a Markdown skill file. That accumulated procedure is what let the agent brief workers without the author writing each brief. This matches the Hermes skills documentation, which says the system prompt asks the agent to save a skill after a multi-step workflow, after finding a working path past errors, and after a user correction. It is skill level self-improvement in the sense of our first table: persistent, readable, human shareable procedure files, with a frozen model underneath.

What it does not show. The post is unusually candid about the limits, which is to its credit:

• Reviewers caught public names that workers removed because nothing inside the repository called them, though external plugins could import them.
• An automated rewrite of suppress() calls changed exception handling at roughly 65 sites. The post describes these as real regressions that the existing tests had missed.
• Further fixes followed after merge.
• Splitting files increased module count and import dependencies, and some entry points took longer to import.
• The post's measurement of whether the code got easier for agents was a simulated symbol lookup (average tokens per lookup fell from 2,218 to 993). It says plainly: "we didn't measure agents completing engineering tasks."

So this is evidence that a skill using agent with a large budget and human review can carry out a large mechanical refactor, not that it improved its own capability in a compounding way. It also shows why agent written changes need review: the regressions the tests missed were the kind that break downstream users.

What does this mean for security teams today?

Strip away the speculation and the security question is ordinary: agents now persist what they learn as skills, memory notes, scripts and code, which change behaviour in later sessions, often unread and sometimes by default. That is a software supply chain problem with supply chain answers.

The threat and control map

The fifth row is the direct lesson from STOP and the Darwin Gödel Machine. Anything that grades the agent must live where the agent cannot write. In practice that means CI configuration, test fixtures and policy files owned by a separate group with required review, and agents that run with credentials that cannot change them.

Step by step: put an agent's learned files under change control

These steps use Hermes Agent as the worked example because its documentation names the controls explicitly. We checked the Hermes skills documentation on 23 October 2026; the project ships releases several times a week, so check the current docs before you copy anything. The same pattern applies to any agent that writes skills or memory to disk.

Step 1. Turn on the write approval gates. According to the Hermes skills documentation, the agent writes skills freely by default, including from the background self-improvement review that runs after a turn. Setting skills.write_approval: true stages every skill_manage write (create, edit, patch, delete, write_file, remove_file) under ~/.hermes/pending/skills/ for review. Memory writes have the same gate under memory.write_approval. In your Hermes config.yaml:

skills:
  write_approval: true    # default is false: the agent writes skills freely
memory:
  write_approval: true    # stage memory writes for review

The docs also describe turning the skills gate on from inside a session with /skills approval on. Note that skills.guard_agent_created is a separate content scanner, not an approval gate; the docs say the two are independent.

Step 2. Put the skills directory under git. This is not destructive, but take a copy first so you have a clean point to return to.

# Back up first
tar -czf ~/hermes-skills-backup-$(date +%F).tar.gz -C ~/.hermes skills

cd ~/.hermes/skills
git init
git add -A
git commit -m "Baseline of agent skills on $(date +%F)"

Step 3. Review staged writes like pull requests. The documented review commands are:

/skills pending        list staged skill writes with a one-line gist
/skills diff <id>      show the full unified diff
/skills approve <id>   apply it
/skills reject <id>    drop it

Warning: /skills reject discards the staged change. If you might want it later, copy the pending file first.

# Back up staged writes before rejecting anything
cp -R ~/.hermes/pending/skills ~/hermes-pending-skills-$(date +%F-%H%M)

After you approve a change, commit it so the history records who accepted what:

cd ~/.hermes/skills
git add -A
git commit -m "Approve skill change: <short description>"

Step 4. Roll back a bad skill from history. Warning: restoring files overwrites the current version. Back up first, then restore one path rather than resetting the whole tree.

# Back up the current state
tar -czf ~/hermes-skills-before-rollback-$(date +%F-%H%M).tar.gz -C ~/.hermes skills

cd ~/.hermes/skills
git log --oneline -- <skill-directory>
git restore --source=<good-commit> -- <skill-directory>
git commit -m "Roll back <skill-directory> to <good-commit>"

Step 5. Sign shared skill bundles. The Nous post describes sending a learned skill to other engineers to install in their own agents. If you share skills, treat them as release artifacts. A keyless Sigstore signature records who produced the bundle:

tar -czf team-skills-$(date +%F).tar.gz -C ~/.hermes/skills <skill-directory>
cosign sign-blob --bundle team-skills.sigstore.json team-skills-$(date +%F).tar.gz

# On the receiving side
cosign verify-blob --bundle team-skills.sigstore.json \
  --certificate-identity you@example.com \
  --certificate-oidc-issuer https://accounts.google.com \
  team-skills-YYYY-MM-DD.tar.gz

Replace the identity and issuer with the ones your organisation uses for signing.

Step 6. Pin what the agent installs. When an agent adds Python dependencies to a project, require hashes so the next install gets exactly what was reviewed:

pip install pip-tools
pip-compile --generate-hashes -o requirements.txt requirements.in
pip install --require-hashes -r requirements.txt

For npm projects, commit the lockfile and install with npm ci, which fails if package.json and package-lock.json disagree.

Step 7. Keep evaluators out of reach. Protect the branch, require review for changes under your CI directory, and make sure the agent's credentials cannot approve its own pull requests. In GitHub, a CODEOWNERS entry such as /.github/ @your-org/platform-reviewers, combined with a branch protection rule that requires code owner review, keeps the checks that grade agent work under a different owner than the agent.

For the review side, see tests to demand before merging AI written code, CWE-1123 excessive self-modifying code and AI governance frameworks for managing risk in AI built software.

Common mistakes

• Reading a training score as a capability. If the paper or post does not report held out results, assume the number is an upper bound.
• Treating scaffold gains as model gains. A better harness around a frozen model is valuable, but it is capped by the model. SICA's own reasoning benchmark results show the cap.
• Ignoring the evaluation subset size. A 50 task subset can move several points between runs. The Darwin Gödel Machine's own full benchmark check moved a Polyglot result from 38.0% to 30.7%.
• Calling a single improvement a loop. AlphaEvolve's 1% training time reduction is a real result and one step. A loop needs the next step to come from the improved system, faster.
• Quoting self-reported figures as fact. Adoption numbers, staffing estimates and "value created" figures in a company post are the company's numbers. Attribute them.
• Letting the agent write where it is graded. Objective hacking is documented in the research; assume it will happen in production if the agent can reach its own checks.
• Leaving skill write approval at the default. In Hermes the default is to write freely, per its docs. Decide deliberately, by environment.
• Assuming the agent's own dependencies are someone else's problem. The agent runtime, its plugins and the packages it installs are open source components that receive CVEs like any other.

Where Safeguard fits, honestly

Safeguard does not evaluate self-improvement claims, does not measure model capability and does not do maths or science research. Safeguard's models are single purpose security models, and our benchmarks page says general LLMs beat them on maths. Hermes Agent and the research systems in this post are not Safeguard products or partners, and nothing here implies Safeguard scans or governs them specifically. The honest bridge is narrower: an agent that writes code, installs packages and authors scripts produces software supply chain artifacts, and those need the same treatment as any other code. Safeguard generates and analyses SBOMs in SPDX and CycloneDX, supports attestation and signing (SLSA, Sigstore, in-toto), evaluates policies and gates on pull requests, and its Griffin models can open tested fix pull requests through autonomous remediation, which ships switched off by default. AI-BOM discovery is in early access. If you want an assistant to query that data, Safeguard's MCP server, with hundreds of MCP tools, can be installed from the Claude or ChatGPT connector directory.

FAQ

Is recursive self-improvement real?

In a narrow sense, yes: published systems such as STOP, SICA and the Darwin Gödel Machine improve the code that improves them, and report measured gains on fixed benchmarks. In the strong sense, a loop where each improvement makes the next one faster without people as the bottleneck, no published system has shown it as of October 2026.

Has any AI improved its own weights?

Self-Rewarding Language Models updated a model's weights using the model's own judgments for three iterations, and the authors note the effect likely saturates. AlphaEvolve found a kernel heuristic that reduced its parent model's training time by 1%, deployed by engineers. Neither is an open ended compounding loop.

What is the Darwin Gödel Machine?

A 2025 research system, published at ICLR 2026, that keeps an archive of coding agents, asks a frozen foundation model to modify an agent's own code, and keeps variants that score well on coding benchmarks. It reports SWE-bench rising from 20.0% to 50.0% after 80 iterations, at about USD 22,000 per run.

What is the difference between a Gödel machine and the Darwin Gödel Machine?

Schmidhuber's 2003 Gödel machine only rewrites itself after proving the rewrite is beneficial. The Darwin Gödel Machine replaces proof with empirical testing on benchmarks, because, as its authors write, proving most changes beneficial is impossible in practice. That trade makes it buildable, and makes its guarantees only as good as its benchmarks.

Is Hermes Agent a recursively self-improving AI?

Hermes Agent saves and refines skills and memory files as it works, which is self-improvement at the skill and memory layer. It does not retrain the model it runs on, and Nous's own refactoring post describes human review catching regressions the agent's tests missed. Call it a skill learning agent, not a recursively self-improving AI.

What is objective hacking in self-improving agents?

It is when a system raises its score without solving the task, by exploiting the evaluator. The Darwin Gödel Machine paper documents an agent that removed logging its hallucination checker depended on and so scored perfectly without fixing hallucination. STOP measured generated code bypassing a sandbox flag in a small fraction of attempts.

How do I judge a claim that an AI system improves itself?

Ask what layer changed, what the baseline was, whether results are on held out tasks, how big the evaluation set was, how much compute it took, whether the model changed, who measured it, whether the evaluator could be gamed, whether gains compound, and whether you can reproduce it. The checklist table above covers each.

What should security teams do about self-improving agents?

Treat what agents learn and write as code: turn on write approval for skills and memory where the agent supports it, keep learned files in version control, route code changes through pull requests with tests, pin dependencies with hashes, generate SBOMs per build, and keep CI and evaluators outside the agent's write scope.

Fix it without waiting for upstream

A self-improving agent installs packages, writes scripts and saves skills that end up in your repositories and builds, and its own runtime, plugins and dependency tree are open source components that receive CVEs like any other software. Whatever you think of the research claims, the code these agents produce and depend on needs the same patch discipline as the rest of your supply chain.

Whatever the vulnerability, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.

1. Autonomous remediation: pull requests, not tickets

When a fixed version exists, Safeguard's Griffin models open a tested, policy gated pull request that moves the affected dependency to the safe version. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation covers eight categories, including dependencies, licenses, security posture and findings. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Autonomous remediation ships switched off, with every category set to Manual, so your team decides how much runs on its own and what still needs a human approval.

2. Gold Open Source: the safe version in seconds, free

gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds. The safeguard-sh/gold-scan-action GitHub Action can fail a build on a known exploited vulnerability, on any known vulnerability, or never, so a version you have just removed cannot quietly come back in next week's merge.

3. Gold Registry: start from components that are already fixed

registry.safeguard.sh provides hardened, drop in replacements for upstream packages and container images. Every Gold artifact is held continuously to zero exploitable critical or high CVEs, zero malware and a completed license check, with SLSA L3 or higher provenance, a Sigstore signature and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.

When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened, compatibility tested build of a specific package. This fork and patch flow is in early access today, so ask which packages it covers before you plan around it.

One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.

See how Safeguard would help

The first demo shows Claude connected to the Safeguard MCP server with OAuth, and choosing which Safeguard tools Claude is allowed to use. The same idea applies to any agent you let act on your code: decide its tool scope deliberately rather than by default. To set it up yourself, install Safeguard from the Claude connector directory rather than writing a custom configuration.

Click through it at your own pace, or open it full screen.

The second demo is a real Safeguard CLI session: asked to fix a lodash CVE, the assistant reads the finding and advisory, asks before each command, edits package.json to lodash 4.17.21, runs npm install and npm test with approval, prints a summary and pushes nothing. That approve each step pattern is the one this post recommends for any agent whose changes persist.

Click through it at your own pace, or open it full screen.

The third demo sets up the Safeguard runner on your own machine, approves it with a device code, then scans from the terminal. If agents are writing code and adding dependencies on developer machines, a local scan is the quickest way to see what they brought in before it reaches a pull request.

Click through it at your own pace, or open it full screen.

You can download the apps at safeguard.sh/download and read the CLI documentation at docs.safeguard.sh/docs/cli.

Quick check

Sources

• I. J. Good, "Speculations Concerning the First Ultraintelligent Machine", Advances in Computers vol. 6, Crossref record
• Schmidhuber, Goedel Machines (arXiv cs/0309048)
• Shinn et al., Reflexion (arXiv 2303.11366)
• Wang et al., Voyager (arXiv 2305.16291)
• Zelikman et al., Self-Taught Optimizer (STOP) (arXiv 2310.02304)
• Yuan et al., Self-Rewarding Language Models (arXiv 2401.10020)
• Hu, Lu and Clune, Automated Design of Agentic Systems (arXiv 2408.08435)
• Yin et al., Gödel Agent (arXiv 2410.04444)
• Robeyns, Szummer and Aitchison, A Self-Improving Coding Agent (arXiv 2504.15228)
• Zhang et al., Darwin Gödel Machine (arXiv 2505.22954)
• Novikov et al., AlphaEvolve white paper (arXiv 2506.13131)
• Shumailov et al., AI models collapse when trained on recursively generated data, Nature 631 (2024)
• Kwa et al., Measuring AI Ability to Complete Long Software Tasks (arXiv 2503.14499)
• METR blog, Measuring AI Ability to Complete Long Tasks (19 March 2025)
• METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (arXiv 2507.09089)
• International AI Safety Report 2026
• International AI Safety Report, first edition (arXiv 2501.17805)
• Anthropic, Responsible Scaling Policy page
• OpenAI, Preparedness Framework version 2 (PDF, 15 April 2025)
• Nous Research, Refactoring Hermes with 1,393 agents (September 2026, self-reported)
• Hermes Agent skills documentation (GitHub source)
• Hermes Agent releases

Last reviewed: 24 October 2026. Hermes Agent ships several releases a week and research in this area moves quickly; check the linked papers, framework pages and project docs for the latest versions before relying on any figure above.