An AI model trained on personal data isn't automatically anonymous under the GDPR, and the burden of showing that it is sits with the company that built it. That's the core of Opinion 28/2024 from the European Data Protection Board (EDPB), adopted on 17 December 2024. The EDPB's draft Guidelines 02/2026 on Anonymisation, adopted for consultation on 7 July 2026, push the same reasoning further. The consultation closes on 30 October 2026 at 23:59 CET.
This post is for engineering, ML and privacy leads at companies that train or fine-tune models on data that includes people: support tickets, user content, scraped web pages. It covers the two-part test, the evidence the Board expects to see, how legitimate interest works for training, what happens if the training data was unlawful, and a script that checks a model's documentation record for gaps. It explains published guidance. It isn't legal advice.
The short answer
Under the EDPB's Opinion 28/2024, an AI model trained with personal data is anonymous only if two likelihoods are both insignificant: that personal data about people in the training set can be extracted directly from the model, including probabilistically, and that such data can be obtained from queries, intentionally or not. Both are judged against "all the means reasonably likely to be used" by the controller or anyone else. Supervisory authorities decide case by case, and they look first at the controller's documentation. If the model isn't anonymous, the GDPR applies to it.
Why the question matters
If a model is anonymous, the GDPR stops applying to the model itself. Then the rules on legal basis, access, erasure and objection don't attach to the weights, though they still apply to the training data and to any personal data you process when you deploy it. If it isn't anonymous, the model is personal data in its own right, and every company that downloads, hosts or fine-tunes it is processing that data.
The Board's starting point is blunt: the opinion says AI models trained with personal data "cannot, in all cases, be considered anonymous." So "we trained it, therefore the weights are just numbers" isn't an argument the Board accepts.
The two-part test
Both parts must hold.
- Direct extraction. How likely is it that someone with access to the model itself, not just an API, could pull out personal data about the people whose data trained it? This includes probabilistic extraction, where the output isn't guaranteed but is likely enough to matter.
- Extraction through queries. How likely is it that prompts, deliberate or accidental, produce that personal data?
Who could try matters. Paragraph 46 of the opinion separates a publicly available model, open to "an unknown number of people with an unknown range of methods", from an internal model only staff can reach. The same model can pass for one and fail for the other, so the testing a regulator expects scales with exposure.
What the Board wants to see
The opinion gives a list it calls "non-prescriptive and non-exhaustive", and says no single item settles the question. In practice it reads like an evidence checklist.
| Area | What supervisors may look at |
|---|---|
| Selection of sources | Selection criteria, whether sources fit the purpose, whether inappropriate sources were excluded |
| Data preparation | Whether anonymous or pseudonymized data was considered, and if not, why; minimization; filtering out irrelevant personal data before training |
| Training method | Regularization to reduce overfitting and, which the opinion marks as crucial, privacy-preserving techniques such as differential privacy |
| Output measures | Controls that lower the chance of getting training data back from queries, while noting these don't stop direct extraction |
| Testing | Structured testing against attribute and membership inference, exfiltration, regurgitation of training data, model inversion and reconstruction attacks |
| Documentation | DPIA or the decision not to do one, DPO advice, threat model, per-source measures with source URLs, test reports and results, and what was told to deployers |
Two details in the documentation list are easy to miss. Paragraph 58(c) asks for the measures taken for each training source, "including relevant source URLs", and that covers measures already taken by third-party dataset providers. Paragraph 58(e) asks for the ratio between the amount of training data and the number of parameters, with an analysis of its effect, alongside re-identification metrics and test reports that say who tested, when, how and to what extent.
The opinion also limits what a passing test proves. Paragraph 55 says successful testing against widely known, state-of-the-art attacks "can only be evidence for the resistance to those attacks." If you only ran a regurgitation test, you have evidence about regurgitation and nothing else.
Without the documentation, the Board says, a supervisor that can't confirm effective anonymization measures may find that the controller failed its accountability duty under Article 5(2).
What the 2026 anonymization guidelines add
Guidelines 02/2026 are general, not AI-specific, but they give the test a structure. Anonymity is judged from each relevant party's perspective, and the guidelines draw repeatedly on the Court of Justice's judgment in EDPS v SRB (C-413/23 P). Data passes if it meets three criteria:
- No Record Isolation: no unique combination of attribute values that relates to one person.
- No Linkage: no record that can be linked to the same person in another dataset.
- No Inference: no meaningful inference about a person can be drawn from the data.
The third criterion is the one that reaches models. Paragraph 82 says inferences can be drawn from data that represents correlations rather than the original records, "in particular" for AI models and synthetic data, by prompting the model with extra information to get new information about a specific person. The guidelines also describe membership inference, where someone learns that a person's data was in the training set, as a possible first step to other inferences or a disclosure in itself.
These guidelines are version 1.0, adopted for public consultation. Feedback is open until 30 October 2026 at 23:59 CET, and the final text may change.
Training on legitimate interest
Most companies training on data they didn't collect for that purpose will look at legitimate interest under Article 6(1)(f). The opinion recalls the three-step test.
- A legitimate interest that is lawful, clearly and precisely articulated, and real and present rather than speculative. The opinion's examples include building a conversational agent to assist users, and improving threat detection.
- Necessity: the processing actually serves that interest, and there's no less intrusive way. Supervisors are told to pay particular attention to the amount of personal data, in light of minimization.
- Balancing: the interest isn't overridden by the rights of the people involved. Their reasonable expectations count, including whether the data was public, where it came from, and whether people even know it's online.
When the balance looks unfavorable, the opinion lists mitigations, and says they must go beyond what the GDPR already requires. For training they include replacing names and email addresses with fake values when the substance doesn't depend on them, leaving a reasonable gap between collecting a dataset and using it, offering an unconditional opt-out before processing, and letting people submit claims that the model regurgitates their data. For web scraping, the examples include excluding sensitive sources and respecting robots.txt, ai.txt or other recognized ways sites object to scraping for AI training. For deployment, they include output filters and post-training techniques that try to remove or suppress personal data.
If the training data was unlawful
The opinion walks through three scenarios.
- Same company, personal data still in the model. Whether the unlawful training taints deployment is decided case by case, depending on whether the two phases have separate purposes.
- Another company deploys the model, personal data still in it. The deployer is expected to have assessed, as part of its own accountability, whether the model was developed lawfully. That includes the data's source and whether a regulator or court has found an infringement. The depth of that check should match the risk of the deployment.
- The model was anonymized after unlawful training. If you can show that operating the model doesn't involve personal data, the GDPR doesn't apply to that operation, and the earlier unlawfulness doesn't carry over. Personal data you collect while deploying it is covered by the GDPR as normal.
Scenario 2 is the one that reaches companies building on third-party models. "We use a well-known model" doesn't replace your own assessment, and the opinion expects you to be able to show you did one.
Check a model's documentation record
This script reads a JSON record of the evidence you hold for a model and lists what's missing against paragraphs 55 and 58. It checks paperwork, not the model, so a clean result means your file is complete, not that your model is anonymous. Standard library only:
#!/usr/bin/env python3
"""Check an AI model's documentation record against the evidence the EDPB
lists in Opinion 28/2024 (paragraphs 46, 55 and 58) for a claim that the
model is anonymous. It finds gaps in the record; it can't tell you whether
the model is anonymous.
Usage: python3 anonymity_record_check.py model_record.json
"""
import json
import sys
# Paragraph 58, items a to f.
DOCS = {
"dpia": "58(a) DPIA, or the decision that one wasn't needed",
"dpo_advice": "58(b) advice from the DPO, where one was or should have been appointed",
"design_measures": "58(c) design measures, threat model and risk assessment",
"lifecycle_measures": "58(d) measures across the lifecycle that limit or verify personal data in the model",
"resistance": "58(e) evidence of resistance to re-identification and the test reports",
"deployer_docs": "58(f) documentation given to deployers or data subjects, incl. residual risks",
}
# Paragraph 55, the attack types named "as at the date of this Opinion".
TESTS = {
"membership_inference": "attribute and membership inference",
"exfiltration": "exfiltration",
"regurgitation": "regurgitation of training data",
"model_inversion": "model inversion",
"reconstruction": "reconstruction",
}
# Paragraph 58(e)(i) to (iv).
RESISTANCE = {
"data_to_parameter_ratio": "58(e)(i) ratio of training data to parameters, with analysis",
"reidentification_metrics": "58(e)(ii) re-identification likelihood metrics",
"test_reports": "58(e)(iii) who tested, when, how and to what extent",
"test_results": "58(e)(iv) test results",
}
def present(v):
return v not in (None, "", [], {}, False)
def check(rec):
gaps = []
docs = rec.get("documentation", {})
for key, label in DOCS.items():
if not present(docs.get(key)):
gaps.append(f"missing {label}")
res = docs.get("resistance") or {}
for key, label in RESISTANCE.items():
if not present(res.get(key)):
gaps.append(f"missing {label}")
tests = rec.get("tests", {})
for key, label in TESTS.items():
t = tests.get(key)
if not present(t):
gaps.append(f"untested {label} (para 55)")
elif not all(present(t.get(f)) for f in ("date", "tester", "result")):
gaps.append(f"partial {label}: record date, tester and result")
sources = rec.get("training_sources", [])
if not sources:
gaps.append("missing 58(c) list of training sources")
for s in sources:
name = s.get("name") or s.get("url") or "unnamed source"
if not present(s.get("url")):
gaps.append(f"missing 58(c) source URL for {name!r}")
if not present(s.get("measures")):
gaps.append(f"missing 58(c) measures taken for {name!r} (yours or the dataset provider's)")
if rec.get("access") == "public" and len([k for k in TESTS if present(tests.get(k))]) < len(TESTS):
gaps.append("note public model: para 46 says testing may need to cover more attackers and methods")
return gaps
def main():
for path in sys.argv[1:]:
with open(path) as f:
rec = json.load(f)
gaps = check(rec)
print(f"{path}: {rec.get('model', '?')} ({rec.get('access', 'access not stated')})")
for g in gaps or ["no gaps against paragraphs 55 and 58"]:
print(" " + g)
if __name__ == "__main__":
main()
We ran it against two made-up records, an internal fine-tuned support model with a full file and a public chat model with a thin one:
fx/support-bot-v3.json: support-bot-v3 (fine-tuned) (internal)
no gaps against paragraphs 55 and 58
fx/public-chat-model.json: public-chat-7b (public)
missing 58(b) advice from the DPO, where one was or should have been appointed
missing 58(d) measures across the lifecycle that limit or verify personal data in the model
missing 58(f) documentation given to deployers or data subjects, incl. residual risks
missing 58(e)(i) ratio of training data to parameters, with analysis
missing 58(e)(ii) re-identification likelihood metrics
missing 58(e)(iii) who tested, when, how and to what extent
untested exfiltration (para 55)
partial regurgitation of training data: record date, tester and result
untested model inversion (para 55)
untested reconstruction (para 55)
missing 58(c) measures taken for 'web crawl 2026-03' (yours or the dataset provider's)
missing 58(c) source URL for 'licensed forum dump'
note public model: para 46 says testing may need to cover more attackers and methods
Keep the record next to the model in version control and update it whenever the training data, the base model or the test suite changes. It's also what you'd hand a customer's privacy team when they ask how you trained.
Collecting that evidence by hand is the slow part. Here's Safeguard's interactive walkthrough of collecting compliance evidence automatically:
What to do
- Decide what you're claiming. If you say a model is anonymous, you need the testing and documentation to back it. If you can't, treat it as personal data and plan for access, erasure and objection requests.
- List every training source with its URL and the measures applied to it, yours or the provider's.
- Test against more than one attack. Cover membership inference, regurgitation, model inversion and reconstruction, record who ran each test and when, and test harder if the model is public.
- Write down the legitimate interest assessment, including the balancing test. The opinion says supervisors may consider whether you published it.
- Assess third-party models you deploy. Record where they came from and whether any finding of infringement has been made about their development.
- Read the draft guidelines and comment by 30 October if the No Inference criterion affects how you'd evidence anonymity.
Related reading: our post on GDPR Article 22 and AI agents covers automated decisions, and GDPR for software developers covers the basics.
The fair view
The Board's approach is demanding, and some of it is hard to meet with today's tools. Proving a negative, that nobody can reasonably get personal data out of a model, depends on an attack catalog that keeps growing, and the opinion itself says passing known attacks only proves resistance to those attacks. Smaller companies will find the documentation list heavy.
It's also more workable than it first looks. Nothing in the opinion forbids training on personal data, it says legitimate interest can work, and its third scenario gives a clear route for a properly anonymized model. Most of what it asks for is what a careful ML team should already keep: where the data came from, what was removed, how the model was tested. The gap for most companies isn't the work. It's that nobody wrote it down.
Check yourself
Your model passes a regurgitation test. Can you call it anonymous?
Not on that basis alone. The opinion asks about both direct extraction and extraction through queries, and says a passing test is only evidence of resistance to the attack tested. You'd also need evidence on membership and attribute inference, model inversion, reconstruction and exfiltration, plus the documentation.
Does it matter whether the model is public or internal?
Yes. The means reasonably likely to be used differ. A public model is open to an unknown number of people with unknown methods, so supervisors may expect more testing and stronger resistance than for a model only employees can reach.
You deploy a third-party model that turns out to have been trained unlawfully. Are you automatically in breach?
Not automatically. If personal data remains in the model, supervisors look at whether you assessed its lawful development before deploying it, including the data's source and any findings of infringement. If the model was effectively anonymized, the GDPR doesn't apply to operating it, though it still applies to personal data you process in deployment.
