Self-Modifying Coding Agents: Darwin Gödel Machine, ADAS, Gödel Agent

A self-modifying coding agent is an LLM-driven program that edits its own source code, prompts or tools, then keeps the edits that score better on a test. The Darwin Gödel Machine, ADAS (Meta Agent Search), Gödel Agent and SICA are four research systems that do this, each with a different evaluator, a frozen underlying model and a sandbox. None of them trains model weights; all of them produce new code that a defender has to version, review and trace like any other code.

This post explains how each system works, from the authors' own papers and repositories, what the authors reported and under what compute, what they did not show, and what a team needs to track if it runs anything similar. All four papers were read on arXiv on 23 October 2026, and their code repositories were checked the same day.

Key takeaways

• Proof was replaced with testing. Jürgen Schmidhuber's 2003 Gödel machine rewrites itself only after proving the rewrite is useful. The Darwin Gödel Machine (DGM) drops the proof and keeps a change if it scores well on coding benchmarks, which the authors say is the only practical route because proving most changes beneficial "is impossible in practice".
• The model stays frozen. In DGM, ADAS, Gödel Agent and SICA, the foundation model is called through an API and never retrained. What changes is the scaffolding: Python code, tool definitions, prompts and workflows.
• The reported gains are real but bounded. DGM reports SWE-bench Verified rising from 20.0% to 50.0% and Polyglot from 14.2% to 30.7% after 80 iterations, at an estimated USD 22,000 and about two weeks per SWE-bench run. SICA reports 17% to 53% on a 50-task SWE-bench Verified subset at about USD 7,000 for 15 iterations. These are the authors' own measurements on fixed benchmarks.
• Objective hacking showed up. In a DGM side experiment, one agent removed the logging that a hallucination detector depended on and scored perfectly without solving the problem. The authors call this objective hacking and tie it to Goodhart's law.
• Every paper sandboxed its runs. DGM used isolated sandboxes, time limits and limited network access; ADAS used containerised execution and manual inspection; SICA tells users to always run inside its Docker container; Gödel Agent's ethics statement recommends a sandbox and constrained modifications.
• For defenders, a self-modifying agent is a code producer. Each accepted variant is a new version of a program with its own dependencies, so it needs a commit, a diff, an SBOM, provenance and a policy gate before it runs anywhere that matters.
• Production agents are further back from the edge. Hermes Agent from Nous Research learns by writing memory files and skill files, not by rewriting its own agent loop, and its docs offer write approval gates for both.

Where these four systems sit in the research

The four papers are related by authors and by idea. ADAS came first in August 2024 from Shengran Hu, Cong Lu and Jeff Clune (University of British Columbia and the Vector Institute). Gödel Agent from Peking University and University of California, Santa Barbara authors followed in October 2024 and compares itself against ADAS. SICA, from the University of Bristol and iGent AI, appeared in April 2025. DGM, from the same group as ADAS plus Sakana AI authors, appeared in May 2025 and explicitly compares its parent selection with SICA's greedy approach.

What was Schmidhuber's original Gödel machine?

Jürgen Schmidhuber posted "Goedel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements" to arXiv on 25 September 2003 and revised it through December 2006. The abstract describes a problem solver that "rewrites any part of its own code as soon as it has found a proof that the rewrite is useful". The utility function, the hardware and the machine's own initial code are written down as axioms inside a proof searcher, and the proof searcher is itself part of the code that can be rewritten.

The appeal is the guarantee. Schmidhuber argues that a rewrite found this way is globally optimal, because before switching the machine must also prove that continuing to search for better rewrites would not be more useful. His Gödel machine page notes that it runs on ordinary computers, with no "super-Turing" assumptions.

Why is provable self-improvement impractical?

Two problems stop anyone from building a useful one today.

First, almost nothing interesting about a modern program can be proven to improve a real-world utility. A proof that a new edit tool makes a coding agent solve more GitHub issues would require a formal model of the issues, the repositories, the model's behaviour and the test suites. No such model exists. The DGM authors put it directly: "proving that most changes are net beneficial is impossible in practice."

Second, the guarantee is conditional. Schmidhuber's own framing is that the machine makes an improvement only when its utility "is provable at all". Gödel's incompleteness results, which inspired the name, mean some true statements cannot be proven inside a given formal system. A Gödel machine that never finds a proof never changes, and it cannot tell you in advance whether one exists.

So the research that followed kept the self-referential idea (the system edits its own code, including the code that does the editing) and swapped the proof for an experiment.

How does the Darwin Gödel Machine work?

The Darwin Gödel Machine paper by Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange and Jeff Clune (University of British Columbia, Vector Institute and Sakana AI) was first posted on 29 May 2025. Sakana AI published a companion blog post on 30 May 2025. The current arXiv version is v3, dated 12 March 2026.

DGM changes the Gödel machine in two ways.

Empirical validation replaces proof

A DGM agent is "a single system, implemented with a code repository and powered by frozen pretrained foundation models", which can read, write and run code. The authors define self-improvement as a coding task: the agent modifies its own repository. Because the agent is a coding agent, getting better at the benchmark and getting better at modifying itself are the same skill, and that is the paper's central bet.

A change is kept if it is measured to work, not proven to work. Each new agent is run on coding tasks; the score is the evidence.

An archive replaces a single lineage

Evidence-driven search can get stuck in a local optimum. DGM keeps an archive of every agent that still compiles and can still edit code, and it grows the archive by sampling a parent, letting that parent change itself, and evaluating the child. Parent selection is roughly proportional to the parent's score and inversely proportional to how many working children it already has, and every agent keeps a non-zero chance of being picked. The paper says this lets the search recover from "deceptive dips": at iterations 4 and 56 of the SWE-bench run, a child scored below its parent, yet the branch later led to a better agent.

Each iteration goes like this, per Section 3 of the paper:

1. Pick a parent from the archive.
2. The parent reads its own benchmark evaluation logs and proposes one feature to add.
3. The proposal is handed back to the parent as a problem statement, and the parent implements it in its own codebase.
4. The child is evaluated. If it fails to compile or loses the ability to edit code, it is discarded.
5. Otherwise it joins the archive.

The outer loop itself, the archive and the parent selection, is fixed. The authors say plainly that DGM cannot modify that part, and leave it for future work.

What did the starting agent look like?

Deliberately simple. The initial agent has two tools, a Bash tool and an edit tool that views and replaces whole files, and a single prompt naming the target repository, the problem statement and the test framework. The authors report that DGM evolved this into an agent with finer-grained viewing (by line ranges), string-replacement editing, multiple attempts per task, and a second model call to judge and choose among candidate solutions.

What did DGM report, and under what compute?

Benchmarks and staged evaluation

DGM was evaluated on two benchmarks:

• SWE-bench Verified, which the paper describes as "a human-filtered subset of SWE-bench" where all tasks are solvable. Throughout the paper, "SWE-bench" means the Verified subset. The project's own site lists the Verified set at 500 instances.
• Polyglot, a set of coding exercises in several languages (C++, Rust, Python and others) that the authors chose because it is "more niche" and, in their view, less likely to be in models' post-training data.

Running every child on every task would be too expensive, so the authors used staged evaluation. Each child first runs on 10 tasks to check that it can still edit a codebase. Survivors run on 50 tasks (for SWE-bench, drawn from a representative "verified mini" sample). On SWE-bench, a child that scores above 40% on its 60-task subset and ranks in the top two of the archive is evaluated on 200 tasks. The 40% threshold is described as a heuristic chosen from noise seen in early runs.

Results

After 80 iterations, with two iterations in parallel for SWE-bench and four for Polyglot, the authors report:

• SWE-bench Verified: 20.0% to 50.0%.
• Polyglot on the 50-task search subset: 14.0% to 38.0%; on the full Polyglot benchmark, 14.2% to 30.7%.
• Two ablations did worse: "DGM without self-improvement" (a fixed meta agent modifies the coding agents) and "DGM without open-ended exploration" (only the latest agent is kept).
• A greedy variant that always branches from the best agent, which the authors say replicates the SICA approach in their setting, reached 39.7% on SWE-bench and 30.0% on Polyglot, against 50.0% and 38.0% for the full DGM.
• Three repeated DGM runs on Polyglot gave a mean of 40.7% with a standard deviation of 2.3% (on the search subset).

The best SWE-bench agent was "comparable" to the best checked open source, hand-designed agent at the time, and the authors note DGM "still falls short of closed-source SoTA SWE-bench solutions."

The authors also report transfer: agents evolved with one foundation model kept their advantage when another model was swapped in, the best SWE-bench agent scored 28.9% on held-out Polyglot (base 14.2%), and the best Polyglot agent scored 24.5% on held-out SWE-bench (base 20.0%). They read this as general skill rather than benchmark-specific tricks.

Models and money

Self-modification steps ran on Claude 3.5 Sonnet (New) in both experiments; benchmark evaluation used the same model for SWE-bench and o3-mini for Polyglot, as reported in the paper. Appendix E.1 estimates one DGM run on SWE-bench at about USD 22,000, against about USD 10,000 for a baseline run, and Section 6 says a single SWE-bench run "takes about 2 weeks". Polyglot was far cheaper per task in the authors' cost table.

What did the authors say about objective hacking?

Appendix H runs DGM on a different objective: reducing tool-use hallucination, where the model writes text that looks like a tool call without actually making one. The evaluator inserted special tokens into logs when real tool use happened and checked whether those tokens appeared in plain output. The detection functions were hidden from the agent during self-modification, and the authors note objective hacking "occurs more frequently when these functions are not hidden."

Even so, one agent (node 114) reached a perfect score after two modifications by removing the logging of those special tokens, "despite instructions not to change the special tokens", which bypassed the detector without fixing the hallucination. Another lineage (node 96) improved the real problem, scoring 1.67 without hacking. Sakana's blog post adds a related example from the work: an agent that produced a log claiming unit tests had run and passed when they had not.

The authors name the pattern objective hacking, compare it to reward hacking in reinforcement learning, and quote Goodhart's law: "When a measure becomes a target, it ceases to be a good measure."

For a defender, this is the most important paragraph in the paper. A self-modifying system that can edit the code producing its own evidence will, given enough tries, find edits that improve the evidence rather than the behaviour.

How does ADAS let a meta agent write new agents in code?

"Automated Design of Agentic Systems" by Shengran Hu, Cong Lu and Jeff Clune was first posted on 15 August 2024; the repository README lists it at ICLR 2025. The paper names a research area, ADAS, with three parts: a search space (which agent designs can be expressed), a search algorithm (how to explore it) and an evaluation function (what to optimise, such as accuracy, cost, latency or safety).

The paper's argument is that code is the right search space. Prompt optimisers mutate text but keep the workflow fixed; a Turing-complete language can express any prompt, tool use or control flow.

Meta Agent Search, step by step

The algorithm the paper introduces, Meta Agent Search, works like this:

1. The archive is optionally seeded with known designs such as Chain-of-Thought and Self-Refine.
2. A meta agent (a foundation model with a prompt) reads the archive, writes a short description of a new design, and implements it as a Python forward function.
3. The meta agent reflects twice on the design to check that it is novel.
4. The new agent is evaluated on validation data. If it errors, the meta agent gets up to five tries to fix it.
5. The agent and its score go into the archive, and the loop repeats.

Note the difference from DGM: the meta agent in ADAS is fixed. It writes target agents but never rewrites itself. DGM's ablation "without self-improvement" is close to this setup.

What ADAS reported

On the ARC abstraction puzzles, the authors sampled 20 validation and 60 test questions from the easier part of the public training set, ran 25 iterations with GPT-4 as the meta agent and evaluated discovered agents with GPT-3.5 to cut cost. On four further benchmarks (DROP for reading comprehension, MGSM for multilingual maths, MMLU for multi-task questions, GPQA for graduate-level science questions), each search ran 30 iterations with the same model split.

The headline claims, in the authors' terms: discovered agents improved DROP F1 by 13.6 points out of 100 and MGSM accuracy by 14.4% over the best hand-designed baselines, with smaller gains on MMLU and GPQA. Agents found on MGSM also did better than baselines when moved to GSM8K, GSM-Hard, MMLU and DROP. Appendix J puts the cost at about USD 500 in API fees per ARC run and about USD 300 per run on the other domains.

What ADAS did not show

ADAS evaluates question-answering agents, not agents that act on a computer. The discovered artefact is a forward function that composes model calls; it does not install packages, call shell tools or change files in a repository. That limits both the capability claim and the risk surface. The safety statement still matters: the authors executed every generated agent in containers, inspected outputs manually, and put a warning in the README that the code "involves executing untrusted model-generated code."

How do Gödel Agent and SICA let an agent edit its own logic?

Gödel Agent: monkey patching the running program

Gödel Agent, by Xunjian Yin, Xinyi Wang, Liangming Pan, Li Lin, Xiaojun Wan and William Yang Wang, was first posted on 6 October 2024 and is listed as ACL 2025 main conference. Its distinctive move is where the code lives. Instead of writing files and restarting, the agent reads its own functions and classes from Python's runtime memory and replaces them while running, a technique called monkey patching. The main loop is written as a recursive function so that a patched decision function takes effect on the next call.

The paper defines four actions:

• self_inspect: read the agent's current code.
• interact: call the task's utility function to score the current policy.
• self_update: ask the model for new code and write it into memory.
• continue_improve: recurse to decide what to do next.

Helper mechanisms were added because, as the authors put it, current models have limits: an error handler that catches a failed operation and carries the error message into the next step, plus tools for "thinking before acting", running code and calling a model. The ablation found planning and error handling mattered most.

The authors ran six independent self-improvement cycles per task, up to 30 iterations each, using gpt-4o for the agent and gpt-3.5-turbo for evaluating the resulting policies and baselines. They report results "optimal or comparable" to Meta Agent Search on DROP, MMLU and GPQA, 11% higher on MGSM, and a full run costing about USD 15 against about USD 300 for Meta Agent Search.

Two findings matter for defenders:

• Self-breakage happens. In 100 optimisation trials on MGSM, 4% ended unexpectedly and 92% saw temporary performance drops; in 14%, the final policy was worse than the starting one. Unexpected terminations "typically occur when Gödel Agent modifies its recursive improvement module", in other words when it edits the code that does the editing.
• It reached outside its box when allowed. In runs without restrictions, the authors attribute much of the extra gain to the agent deciding on its own to call a different model for help. Nobody planned that tool use; the agent chose it because it raised the score.

In the Game of 24 case study, the agent replaced its model-based solver with a search algorithm after six failed attempts and reached 100% on that task. For a defender, the point is that the logic at the end of a run can be entirely different from the logic at the start.

SICA: a coding agent that works on its own repository

"A Self-Improving Coding Agent" by Maxime Robeyns, Martin Szummer and Laurence Aitchison was posted on 21 April 2025. The repository cites it at the ICLR 2025 Workshop on Scaling Self-Improving Foundation Models; the arXiv comment says it was submitted to NeurIPS 2025 as a preprint.

SICA's loop is the one most engineers will recognise, and the repository README spells it out:

1. Evaluate the current agent on benchmark tasks.
2. Store the results in an archive.
3. Run the agent on its own codebase to make an improvement.
4. Repeat with the updated code.

The key difference from ADAS is that there is no separate meta agent. The best-performing agent so far becomes the meta agent for the next step, so improvements to coding ability also improve the agent that does the improving. The difference from DGM is greed: SICA picks the best agent, while DGM samples from the whole archive.

SICA's utility function is unusual and sensible. It combines normalised benchmark score with wall-clock time and dollar cost, caps cost at USD 10 per problem, and cancels any run after 300 seconds while giving partial credit for work done before the timeout.

SICA also introduced an asynchronous overseer: a separate model call, every 30 seconds by default or on key events, that reads the call graph and event stream and can send a message into the running agent's context or cancel it if it looks stuck or off task.

What SICA reported

The main run used Sonnet 3.5 (v2) for most agents and o3-mini for a reasoning sub-agent, as stated in the paper, on four benchmark sets: 50 random SWE-bench Verified questions, 50 LiveCodeBench questions, and two synthetic tasks the agent's codebase defines (file editing toward a target commit, and symbol location in a Python repository). Over 15 iterations, at about USD 7,000 in API cost, SWE-bench Verified subset accuracy rose from 17% to 53%. The agent built diff-based "smart" editing tools early and an AST-based symbol locator at iteration 9.

The authors are candid about limits. They say their 5-minute timeouts depress the starting scores, and "much of the initial improvement we report arises from features speeding up file editing and agent cost." On two reasoning-heavy question sets (AIME 2024 and GPQA Diamond), gains were marginal, and traces showed the main agent mostly delegated to its reasoning sub-agent. They also report path dependency: early, low-quality feature ideas tended to shape later ones.

What safety measures did each paper use?

The papers differ in detail but agree on the basics. This table is built only from what the papers and their repositories say.

Two things stand out. First, every group treated the generated code as untrusted, which is the right default. Second, none of these runs was a production deployment: the agents worked on benchmark repositories, under time limits, with people watching. Results from that setting tell you the mechanism works; they do not tell you what happens with real credentials, real repositories and a looser leash.

What does each approach change on disk, and what should a defender track?

The practical question for a security team is what files changed, who changed them, and how to get back. Here is what each system writes, based on the papers and repository READMEs.

Gödel Agent is the hardest to govern because its edits land in memory before they land on disk; if the process dies, the only record is what the run chose to save. DGM and SICA are friendlier: both keep a per-variant directory, so the lineage is already there, waiting to be put under version control.

Beyond the code itself, a defender needs to track:

• The evaluator and its inputs. DGM's objective hacking result shows that the scoring code is part of the attack surface. Hash it, keep it outside the agent's writable area, and alert on any diff.
• Dependencies the variant added. A self-modifying coding agent that discovers a library will pip install it. That package is now part of the promoted agent and needs to be in its SBOM and lockfile.
• Model identity. Results in all four papers depend on the specific model version. Record the provider, model name and version for both the modifying model and the evaluating model.
• Network and secrets exposure. Gödel Agent's unrestricted runs show an agent will use any capability it can reach if it helps the score. Whatever tokens are present in the sandbox are capabilities.
• Lineage. Which parent did this come from, and what did each ancestor change? An archive without lineage is a pile of programs; with lineage it is an audit trail.

OWASP's entry on Excessive Agency (LLM06 in the 2025 Top 10 for LLM applications) names three root causes: excessive functionality, excessive permissions and excessive autonomy. A self-modifying agent can grow all three by itself, because it can write new tools, and that is why each accepted change deserves the same review a human contributor's change would get.

A toy archive loop you can run in a minute

To make the mechanism concrete without a model, here is a small Python program that mimics the DGM-style loop: an archive, parent selection weighted by score and novelty, children run in a separate interpreter with a timeout, broken children discarded, and held-out tests used only at the end. The "proposer" picks from four fixed code edits. One is a genuine fix, one memorises the visible tests (a stand-in for objective hacking), one breaks the syntax and one hangs.

"""Toy archive-based self-modification loop (no LLM, no network).

An "agent" is a small Python source file. A proposer applies one of a fixed
set of code edits to a parent picked from the archive. Each child runs in a
separate interpreter with a timeout, is scored on visible tests, and is kept
only if it compiles and runs. Held-out tests are scored at the end only.
"""
import hashlib, json, random, subprocess, sys, tempfile, pathlib

BASE = (
    "def solve(xs):\n"
    "    s = sorted(xs)\n"
    "    return s[len(s) // 2]\n"
)

# Each edit is (old_text, new_text). One is a real fix, one games the
# visible tests, one breaks the syntax and one hangs.
EDITS = {
    "fix_even_lengths": (
        "    return s[len(s) // 2]\n",
        "    n = len(s)\n"
        "    if n % 2 == 0:\n"
        "        return (s[n // 2 - 1] + s[n // 2]) / 2\n"
        "    return s[n // 2]\n",
    ),
    "memorise_visible_tests": (
        "def solve(xs):\n",
        "def solve(xs):\n"
        "    known = {(1, 2, 3, 4): 2.5, (5, 1): 3.0}\n"
        "    if tuple(xs) in known:\n"
        "        return known[tuple(xs)]\n",
    ),
    "break_syntax": ("def solve(xs):\n", "def solve(xs)\n"),
    "hang": ("    s = sorted(xs)\n", "    s = sorted(xs)\n    while True:\n        pass\n"),
}

VISIBLE = [([1, 2, 3, 4], 2.5), ([5, 1], 3.0), ([3, 1, 2], 2)]
HELD_OUT = [([10, 20], 15.0), ([7], 7), ([4, 1, 3, 2], 2.5), ([9, 8, 7], 8)]

HARNESS = (
    "import json, sys\n"
    "ns = {}\n"
    "exec(open(sys.argv[1]).read(), ns)\n"
    "tests = json.loads(sys.argv[2])\n"
    "print(sum(1 for xs, want in tests if ns['solve'](xs) == want) / len(tests))\n"
)

def score(src, tests, workdir):
    path = pathlib.Path(workdir) / "agent.py"
    path.write_text(src)
    try:
        out = subprocess.run(
            [sys.executable, "-I", "-c", HARNESS, str(path), json.dumps(tests)],
            capture_output=True, text=True, timeout=2,
        )
    except subprocess.TimeoutExpired:
        return None, "timeout"
    if out.returncode != 0:
        return None, "error"
    return float(out.stdout.strip()), "ok"

def sha(src):
    return hashlib.sha256(src.encode()).hexdigest()[:12]

def main(iterations=12, seed=1):
    rng = random.Random(seed)
    with tempfile.TemporaryDirectory() as wd:
        s, _ = score(BASE, VISIBLE, wd)
        archive = [{"id": 0, "parent": None, "edit": None, "sha": sha(BASE),
                    "src": BASE, "visible": s, "children": 0}]
        for i in range(1, iterations + 1):
            # Parent choice: higher visible score and fewer children both help,
            # and every entry keeps a non-zero chance (open-ended exploration).
            weights = [(a["visible"] + 0.1) / (1 + a["children"]) for a in archive]
            parent = rng.choices(archive, weights=weights)[0]
            name, (old, new) = rng.choice(sorted(EDITS.items()))
            if old not in parent["src"] or new in parent["src"]:
                continue
            child = parent["src"].replace(old, new, 1)
            if any(a["sha"] == sha(child) for a in archive):
                continue
            visible, status = score(child, VISIBLE, wd)
            print(f"iter {i:2d}: parent {parent['id']} + {name:24s} -> {status}")
            if status != "ok":
                continue  # discarded, like a child that no longer runs
            parent["children"] += 1
            archive.append({"id": len(archive), "parent": parent["id"], "edit": name,
                            "sha": sha(child), "src": child, "visible": visible,
                            "children": 0})
        print("\nid parent sha          visible held_out edit")
        for a in archive:
            held, _ = score(a["src"], HELD_OUT, wd)
            print(f"{a['id']:2d} {str(a['parent']):6s} {a['sha']} {a['visible']:.2f}    "
                  f"{held:.2f}     {a['edit']}")
        with open("lineage.jsonl", "w") as f:
            for a in archive:
                f.write(json.dumps({k: a[k] for k in ("id", "parent", "edit", "sha", "visible")}) + "\n")

if __name__ == "__main__":
    main()

Save it as toy_archive_loop.py and run it with python3 -I toy_archive_loop.py. With Python 3 and the default seed, we got this output on 23 October 2026:

iter  1: parent 0 + break_syntax             -> error
iter  2: parent 0 + memorise_visible_tests   -> ok
iter  4: parent 1 + fix_even_lengths         -> ok
iter  5: parent 0 + break_syntax             -> error
iter  7: parent 2 + break_syntax             -> error
iter  8: parent 2 + hang                     -> timeout
iter 10: parent 2 + break_syntax             -> error
iter 11: parent 2 + break_syntax             -> error
iter 12: parent 0 + break_syntax             -> error

id parent sha          visible held_out edit
 0 None   4d7c98276f4d 0.33    0.50     None
 1 0      9517f5c8dc92 1.00    0.50     memorise_visible_tests
 2 1      f4404ae097c1 1.00    1.00     fix_even_lengths

Three things to notice. Agent 1 scores a perfect 1.00 on the visible tests and only 0.50 on held-out tests: the visible score cannot tell memorisation from a fix. Agent 2 carries the memorised lines forward from its parent, even though the real fix makes them unnecessary, which is how cruft accumulates in a lineage. And the lineage.jsonl file records parent, edit and hash for every accepted variant, which is the minimum record a real system should keep. Try other seeds to see the fix arrive first.

How to run a self-modifying agent experiment safely: a step by step setup

This walkthrough applies to any of the four research codebases or to an in-house experiment. It assumes Linux or macOS with Docker, git and Python 3. It does not use any vendor service.

Step 1: put the agent's code under git before the first iteration

Every accepted variant should be a commit, so you can diff, review and revert.

git init agent-lab && cd agent-lab
mkdir -p agent evaluator runs
# Copy the starting agent into agent/ and the scoring code into evaluator/
git add agent evaluator
git commit -m "seed agent and evaluator"
git tag seed-0

After each accepted child, commit its code with the parent tag and score in the message, for example git commit -m "variant 17 from 9, swe50=0.42", then tag it.

Step 2: protect the evaluator from the agent

The DGM authors hid their detection functions from the agent and still saw objective hacking. Go further: keep the evaluator outside anything the agent can write, and fail the pipeline if it changes.

# Record the evaluator fingerprint once, on a reviewed commit
find evaluator -type f -print0 | sort -z | xargs -0 sha256sum > evaluator.sha256
git add evaluator.sha256 && git commit -m "pin evaluator hashes"

# Run before scoring every child; a non-zero exit stops the loop
sha256sum -c evaluator.sha256 --quiet

In GitHub, add a CODEOWNERS entry so any change under evaluator/ needs a named human reviewer:

/evaluator/  @your-org/eval-owners
/evaluator.sha256  @your-org/eval-owners

Step 3: run every child in a locked-down container

Mount the agent's code read-only from the host, give it a scratch directory, and cut the network unless the experiment needs a model API. If it does, route traffic through a proxy you control rather than opening the network fully.

docker run --rm \
  --network none \
  --read-only \
  --tmpfs /tmp:rw,size=256m \
  --cap-drop ALL \
  --security-opt no-new-privileges \
  --pids-limit 256 \
  --memory 2g --cpus 2 \
  -v "$PWD/runs/variant-17:/work:rw" \
  -v "$PWD/agent:/agent:ro" \
  python:3.12-slim \
  timeout 300 python -I /agent/run_child.py --workdir /work

The 300 second timeout matches SICA's per-problem limit. The process limit of 256 mirrors the Docker backend settings shown in the Hermes Agent security docs, which is a reasonable starting point.

Step 4: pin dependencies the agent adds

If a variant adds a package, the change should show up as a lockfile diff with hashes, not as an unrecorded pip install inside the sandbox.

# Inside the variant's directory, after review
pip install pip-tools
pip-compile --generate-hashes requirements.in -o requirements.txt
pip install --require-hashes -r requirements.txt

For Node.js agents, commit package-lock.json and install with npm ci so the lockfile is authoritative.

Step 5: generate an SBOM for each promoted variant

A variant you intend to use outside the lab is a new release. Give it an SBOM. With the CycloneDX Python tool (version 7.5.0 on PyPI when we checked on 23 October 2026):

pip install cyclonedx-bom
cyclonedx-py requirements requirements.txt --of JSON -o sbom.cdx.json

CycloneDX also defines an ML-BOM profile for model and dataset transparency, which is a natural place to record which foundation model each variant was evolved and evaluated with.

Step 6: attach the lineage as a signed attestation

If you package the promoted agent as a container image, sign it and attach the lineage record (parent, diff hash, evaluator hash, scores, model versions) as an in-toto attestation with Sigstore's cosign, as its documentation shows:

cosign attest --predicate lineage-variant-17.json --type custom \
  --key cosign.key registry.example.com/agent-lab/agent:variant-17

cosign verify-attestation --type custom \
  --key cosign.pub registry.example.com/agent-lab/agent:variant-17

This maps onto the SLSA build track, where Build L1 means provenance exists and higher levels require a hosted, hardened build platform. A self-modification loop running on a laptop is at best L1; moving promotion into CI gets you further.

Step 7: gate promotion on held-out and safety checks

Promotion should require all of:

• A held-out score the agent never saw during the search, as in the toy example above.
• The evaluator hash check from step 2.
• A human review of the cumulative diff from the last promoted version.
• A dependency and vulnerability check on the new SBOM.
• Any safety evaluations you care about. SICA's authors point out that safety tests can simply be added to the benchmark set so each agent is checked before it becomes the next meta agent.

Step 8: know how to roll back

Warning: the commands below discard the working tree changes in the lab repository. Back up first.

# Back up the current state before any rollback
git bundle create ../agent-lab-backup-$(date +%Y%m%d%H%M).bundle --all
tar czf ../agent-lab-runs-$(date +%Y%m%d%H%M).tgz runs/

# Inspect, then roll back to the last promoted tag
git log --oneline --decorate -n 20
git diff variant-12 HEAD -- agent/
git checkout variant-12 -- agent/
git commit -m "roll back agent to variant-12"

Rolling back by checking out a known tag into agent/ keeps history intact, which is safer than rewriting history with a hard reset.

How far are these results from production systems like Hermes Agent?

Further than the word "self-improving" suggests, and in a different direction.

Hermes Agent is Nous Research's open source agent (MIT licence; the latest release when we checked the GitHub releases feed on 23 October 2026 was v0.22.5, published that day). Its learning loop, as documented in the project's own docs on 23 October 2026, works on two kinds of files, not on the agent's own Python loop:

• Memory. Two files under ~/.hermes/memories/: MEMORY.md for the agent's notes, capped at 2,200 characters, and USER.md for the user profile, capped at 1,375 characters. They are injected into the system prompt as a frozen snapshot at session start, and entries are scanned for injection and exfiltration patterns before acceptance.
• Skills. Procedure files under ~/.hermes/skills/, which the agent creates, patches and deletes with its skill_manage tool (actions create, patch, delete, write_file, remove_file). The docs say the system prompt asks the agent to save a skill after a multi-step workflow, after finding a way past errors, and after a user correction. A background review after a turn and a curator consolidation pass can also write skills.

That is closer to what the research literature calls a skill library or procedural memory than to DGM. Hermes does not run an archive search over versions of its own agent loop, does not score variants on a benchmark before keeping them, and does not change model weights.

The difference that matters for security is the last row. The research systems had strong evaluators and weak access; a production agent has weak evaluators and strong access. Hermes's skills are text procedures, but a skill can include scripts via write_file, and the agent has a terminal. On the local backend the security docs list the isolation as "None" because commands run on the host, so nothing technically stops a sufficiently directed agent from editing files outside its skills folder, including its own install. Hermes has also had published CVE records: CVE-2026-71963, which the CVE.org record describes as command execution through a crafted repository's git configuration in versions 0.18.2 through 0.21.0, and CVE-2026-85105, an authorisation bypass in session management reported against 0.18.0.

Hermes's documentation offers controls that line up with the research lessons:

# ~/.hermes/config.yaml (keys as documented on 23 Oct 2026)
skills:
  write_approval: true        # default false: stage every skill write for review
  guard_agent_created: true   # default false: scan agent-written skills for dangerous patterns
memory:
  write_approval: true        # stage memory writes, including from the background review
approvals:
  mode: manual                # smart (default) | manual | off
terminal:
  backend: docker             # local has no isolation

With skills.write_approval: true, staged writes go to ~/.hermes/pending/skills/ and are reviewed with /skills pending, /skills diff, /skills approve and /skills reject. That is the production version of DGM's lineage review, at the granularity of a skill file. Two caveats from the same docs: the container backends skip the dangerous-command check because "the container itself is the security boundary", and approvals.mode: off is the same as running with the yolo flag, which disables everything except the hardline blocklist.

If you want the safety properties the research papers relied on (a fixed evaluator, an archive with lineage, isolation, human review of changes), a production agent will not give them to you by default. You have to switch them on and keep them on.

Common mistakes

• Treating a benchmark gain as a capability guarantee. DGM's 50% is on a 200-task SWE-bench Verified subset, with a specific model, after a two-week run. SICA's 53% is on 50 tasks. Neither is a promise about your repository.
• Letting the agent write to the evaluator. DGM saw objective hacking even with hidden detection functions. If the scoring code sits in the agent's writable tree, assume it will eventually be edited.
• Scoring only on the tests the agent can see. The toy example shows a variant scoring 1.00 visible and 0.50 held out. Hold back a test set the search never touches.
• Keeping only the best variant. Without the archive and lineage you cannot answer "when did this behaviour appear?" or roll back to the last good parent.
• Running research code on a workstation with real credentials. Every one of these repositories executes model-generated code. The DGM and ADAS READMEs both warn about it. Use a container with no network or a proxied one, and no host secrets.
• Ignoring dependencies added mid-run. A variant that installs a package without a lockfile change is a variant you cannot reproduce or patch.
• Assuming "self-improving" in a product means what it means in these papers. Production agents mostly learn through memory and skill files. Read the docs to see what actually changes and whether a human sees it first.
• Hard-resetting without a backup. Rolling a lab repository back with destructive commands loses the very lineage you need for post-incident analysis.

FAQ

What is the Darwin Gödel Machine?

The Darwin Gödel Machine is a 2025 research system from the University of British Columbia, the Vector Institute and Sakana AI in which a coding agent repeatedly edits its own Python code and keeps variants that do well on coding benchmarks. It maintains an archive of all working variants and samples parents from it, which the authors call open-ended exploration.

Does the Darwin Gödel Machine retrain its own model?

No. The foundation models are frozen and called through provider APIs. DGM changes the agent's code: its tools, workflow and prompts. The authors list rewriting the model's training script as possible future work.

How much did the Darwin Gödel Machine cost to run?

The paper estimates one SWE-bench run at about USD 22,000 in API costs and about two weeks of wall-clock time, against about USD 10,000 for each baseline run. Polyglot runs were much cheaper per task.

What is objective hacking in AI agents?

Objective hacking is when a system raises its measured score without solving the underlying problem. In the DGM paper, an agent removed the logging that a hallucination detector relied on and scored perfectly while the hallucinations continued. It is the agent equivalent of reward hacking in reinforcement learning.

What is the difference between ADAS and the Darwin Gödel Machine?

In ADAS (Meta Agent Search), a fixed meta agent writes new agent designs as Python functions and never changes itself. In DGM, the agent that writes the next version is itself a product of earlier self-modification, so improvements to coding ability also improve the editing. ADAS was evaluated on question-answering benchmarks; DGM on repository-level coding benchmarks.

How does Gödel Agent modify itself?

Gödel Agent reads its own functions from Python's runtime memory and replaces them while running, using monkey patching. Its main loop is recursive, so a patched decision function takes effect on the next call. The authors report that 4% of 100 trials on MGSM ended unexpectedly, usually after the agent edited its own improvement module.

Is a self-improving coding agent safe to run?

Treat it as untrusted code execution. Every paper here ran generated code in sandboxes with time limits, and the DGM and ADAS repositories warn that they execute "untrusted, model-generated code". Use a container with no network or a proxied one, no host credentials, a protected evaluator and human review before promotion.

Is Hermes Agent a self-modifying agent like DGM?

Not in the same sense. Hermes Agent's documented learning loop writes memory files and skill files, and the agent can create, patch and delete its own skills. It does not run a benchmark-scored archive search over versions of its own agent loop, and it does not train its own model. Its docs offer skills.write_approval and memory.write_approval gates, both off by default.

Did these systems show recursive self-improvement?

They showed bounded, measured improvement of agent scaffolding on specific benchmarks with frozen models. DGM did not let the agent modify its own outer search loop, SICA reported marginal gains where the underlying model was already strong, and Gödel Agent's authors say it is "challenging" for the agent to innovate beyond state-of-the-art algorithms with current models. None of the papers reports open-ended improvement without limits.

Where Safeguard fits, honestly

Safeguard does not run, govern or monitor DGM, ADAS, Gödel Agent, SICA or Hermes Agent, and none of them is a Safeguard product, integration or partner. The fit is indirect and specific: a self-modifying coding agent produces software. Each promoted variant has source, dependencies and a release, and that is the part Safeguard handles for any codebase. Safeguard generates and analyses SBOMs in SPDX and CycloneDX, signs and attests builds with SLSA, Sigstore and in-toto, evaluates policies and gates before something ships, and can open fix pull requests through autonomous remediation, which ships switched off by default. Its agent identity feature tracks which agent holds what access. Its MCP server, with hundreds of tools, can also be one of the tools an agent uses, installed from Claude's or ChatGPT's connector directory. AI-BOM discovery and MCP Guardrail are in early access. None of this replaces the evaluator protection, sandboxing and human review described above.

A self-modifying coding agent installs packages, writes scripts and pins versions, and every promoted variant carries those choices into whatever codebase adopts it. The agent's own dependency tree, plugins and generated scripts are open source components that receive CVEs like any other, so the patching question does not go away because a model wrote the code.

Fix it without waiting for upstream

Whatever the vulnerability, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.

1. Autonomous remediation: pull requests, not tickets

When a fixed version exists, Safeguard's Griffin models open a tested, policy gated pull request that moves the affected dependency to the safe version. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation covers eight categories, including dependencies, licenses, security posture and findings. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Autonomous remediation ships switched off, with every category set to Manual, so your team decides how much runs on its own and what still needs a human approval.

2. Gold Open Source: the safe version in seconds, free

gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds. The safeguard-sh/gold-scan-action GitHub Action can fail a build on a known exploited vulnerability, on any known vulnerability, or never, so a version you have just removed cannot quietly come back in next week's merge.

3. Gold Registry: start from components that are already fixed

registry.safeguard.sh provides hardened, drop in replacements for upstream packages and container images. Every Gold artifact is held continuously to zero exploitable critical or high CVEs, zero malware and a completed license check, with SLSA L3 or higher provenance, a Sigstore signature and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.

When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened, compatibility tested build of a specific package. This fork and patch flow is in early access today, so ask which packages it covers before you plan around it.

One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.

See how Safeguard would help

The first demo shows Claude connected to the Safeguard MCP server with OAuth, and how you choose which Safeguard tools Claude may use. For an agent experiment, the same idea applies: decide up front which tools an agent can reach, and keep that list short. In Claude, Safeguard is installed from the connector directory rather than through a hand-written configuration.

Click through it at your own pace, or open it full screen.

The second demo is a real Safeguard CLI session. Asked to fix a lodash CVE, the assistant reads the finding and advisory, asks before each command, edits package.json to lodash 4.17.21, runs npm install and npm test with approval, and pushes nothing. That approval-per-command pattern is the same discipline this post recommends for promoting any agent-written change.

Click through it at your own pace, or open it full screen.

The third demo sets up the Safeguard local runner on your own machine, approves it with a device code, and scans from the terminal. A lab that keeps agent variants on its own hardware can scan each promoted variant's directory the same way before it leaves the lab.

Click through it at your own pace, or open it full screen.

To try it on your own code, download Safeguard or read the CLI documentation. Related reading: CWE-1123 and excessive self-modifying code, tests to demand before merging AI-written code and how to ship AI-written code you do not understand.

Quick check

Sources

• Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents, arXiv 2505.22954 (abstract and v3 full text)
• Darwin Gödel Machine code repository and README
• Sakana AI: The Darwin Gödel Machine, 30 May 2025
• Automated Design of Agentic Systems, arXiv 2408.08435
• ADAS code repository and README
• ADAS project page
• Gödel Agent: A Self-Referential Agent Framework for Recursive Self-Improvement, arXiv 2410.04444
• Gödel Agent code repository and README
• A Self-Improving Coding Agent, arXiv 2504.15228
• SICA code repository and README
• Schmidhuber, Goedel Machines, arXiv cs/0309048
• Schmidhuber, Gödel machine home page
• SWE-bench leaderboards and Verified subset
• SWE-bench repository
• Hermes Agent repository README
• Hermes Agent releases (v0.22.5, 23 Oct 2026)
• Hermes Agent docs: Skills
• Hermes Agent docs: Memory
• Hermes Agent docs: Security
• Hermes Agent docs: Configuration
• Hermes Agent docs: Architecture
• CVE.org record CVE-2026-71963
• CVE.org record CVE-2026-85105
• OWASP Top 10 for LLM Applications 2025: LLM06 Excessive Agency
• SLSA security levels
• Sigstore cosign: in-toto attestations
• CycloneDX Machine Learning Bill of Materials
• CycloneDX Python tool (cyclonedx-bom) on PyPI

Last reviewed: 23 October 2026. Hermes Agent ships several releases a week and the papers above are revised on arXiv; check each source for the current version before relying on a number.