Name: Safeguard
Brand: Safeguard
Availability: PreOrder

Machine learning is the part of artificial intelligence where a program learns its behaviour from data instead of having every rule written by hand. You pick a model with adjustable parameters, define a loss that scores how wrong it is, and let an optimiser such as gradient descent tune the parameters until the error on held-out data stops improving. The result is a trained artifact that makes predictions, and like any artifact in your software supply chain it can be poisoned, evaded, copied or made to leak the data it learned from.

Key takeaways

Four learning setups cover most of production: supervised (labelled examples), unsupervised (structure without labels), self-supervised (labels made from the data itself) and reinforcement learning (reward from an environment).
Training is optimisation. We work one gradient descent step by hand below, including a learning rate that makes things worse.
Accuracy alone misleads. A worked confusion matrix shows a 94 percent accurate classifier whose precision drops to about 15 percent when its target becomes rarer.
Most production ML failures are pipeline failures: leakage, train and serve skew, untracked data versions and drift.
Attacks now have shared names. In the MITRE ATLAS 2026.10 data (modified 28 October 2026), poisoning is AML.T0020, evasion AML.T0015, membership inference AML.T0137.000 and model replication AML.T0005. NIST AI 100-2 E2025 (March 2025) gives each a NISTAML ID. The OWASP Machine Learning Security Top 10 is still a draft.
A model file is code you load. Python's documentation warns that unpickling can execute arbitrary code, so the weight format is a security decision.

This is part 2 of a four-part series on AI foundations.

What is artificial intelligence? History, types, and how AI systems are built and governed

What is machine learning? Supervised, unsupervised and reinforcement learning, models, MLOps and attacks (this post)

What is deep learning? Neural networks, transformers, diffusion, scaling and model security (publishes 10 Nov)

What is superintelligence? AGI vs ASI, timelines, alignment research and governance (publishes 11 Nov)

What machine learning actually is

Arthur Samuel's paper "Some Studies in Machine Learning Using the Game of Checkers" (IBM Journal of Research and Development, July 1959) is the usual starting point for the term. His program got better at checkers by playing, not by being given better rules.

A working definition has four parts:

Data. Examples described by features (numbers or categories the model can read) and, in supervised learning, a label (the answer to predict).
A model family. A function shape with adjustable parameters. A straight line has two. A random forest has thousands of split thresholds. A large neural network has billions of weights.
A loss function. One number for how wrong the parameters are: mean squared error for quantities, log loss for classes.
An optimiser. A procedure that changes parameters to lower the loss. Usually gradient descent; trees are grown greedily instead.

The learned parameters plus the code that applies them form the model artifact. The MLOps and security sections below are about managing that artifact as carefully as a compiled binary. Deep learning, the subject of part 3, is the branch of ML built on many-layered neural networks (our neural networks explainer is a short primer); everything here applies to it too.

This post is about how ML works, not about ML inside security products. For that angle, see why machine learning matters in cybersecurity, ML in SIEM, how EDR uses machine learning and where it can be fooled and combining static, dynamic and ML analysis.

The four ways a model learns

Setup	Training signal	Typical tasks	Classic examples	Main evaluation
Supervised	Labels for each example	Classification, regression, ranking	Logistic regression, random forests, gradient boosting, SVMs	Precision, recall, F1, ROC AUC, error in units
Unsupervised	No labels; the model finds structure	Clustering, dimensionality reduction, anomaly detection	k-means, PCA, isolation forests	Cluster quality, analyst review
Self-supervised	Labels manufactured from the data (hide part, predict it)	Pretraining representations for text, images, audio	Next-token prediction, contrastive learning such as SimCLR	Performance after fine-tuning
Reinforcement	A reward from an environment after actions	Control, games, preference tuning of language models	Q-learning, policy gradients, RLHF	Cumulative reward, task success

Supervised learning learns a mapping from inputs to known answers: a category for classification (phishing or not), a number for regression (time to failure). Most security ML in products is supervised, from spam filters to triage models trained on past analyst decisions. The weak point is the labels: someone produced them, and anyone who can influence them can influence the model.

Unsupervised learning finds structure with no labels. In k-means, you pick k centres, assign each point to the nearest, move each centre to the mean of its points, and repeat. Stuart Lloyd's description of this iteration was published in IEEE Transactions on Information Theory in March 1982, which is why it is often called Lloyd's algorithm. In security, unsupervised methods drive anomaly detection, with the catch that unusual is not the same as bad, and a patient adversary can shift what counts as normal.

Self-supervised learning makes the data label itself: hide a word and predict it, or train a model to recognise two augmented crops of one photo as a match, the contrastive approach of SimCLR (Chen, Kornblith, Norouzi and Hinton, arXiv, February 2020). Large language models are pretrained this way. The security consequence is that training sets are often scraped at a scale where provenance is hard to prove.

Reinforcement learning trains an agent to act in an environment to maximise reward over time. Sutton and Barto's "Reinforcement Learning: An Introduction" (second edition, MIT Press, 2018, free from the authors' site) is the standard text, built on value functions, the exploration and exploitation trade-off, and temporal difference learning. Its most visible modern use is preference tuning: Ouyang et al. (arXiv, March 2022) collected human rankings of model outputs and fine-tuned the language model with reinforcement learning from human feedback. If a reward can be gamed, the agent finds the gap; if the environment can be tampered with, the policy learns the tampered lesson. ATLAS 2026.10 lists that case as AML.T0145 Poison AI Agent Learning Environment.

Features, loss and gradient descent, worked by hand

One training step on paper makes everything else clearer.

Data. Three points: (1, 2), (2, 4), (3, 6). The true rule is y = 2x, but the model does not know that.

Model. prediction = w × x + b, starting at w = 0 and b = 0.

Loss and gradients for n = 3 points, using mean squared error:

L     = (1/n) * sum((w*x + b - y)^2)
dL/dw = (2/n) * sum((w*x + b - y) * x)
dL/db = (2/n) * sum(w*x + b - y)

Residuals at w = 0, b = 0. Every prediction is 0, so prediction minus target is -2, -4 and -6.
Loss. (4 + 16 + 36) / 3 = 56 / 3, about 18.67.
Gradients. dL/dw = (2/3) × ((-2)(1) + (-4)(2) + (-6)(3)) = (2/3) × (-28), about -18.67. dL/db = (2/3) × (-12) = -8. Both are negative, so raising w and b lowers the loss.
Update with learning rate η = 0.05. w = 0 - 0.05 × (-18.67) = 0.933. b = 0 - 0.05 × (-8) = 0.4.
Check. Predictions become 1.333, 2.267 and 3.2, residuals -0.667, -1.733 and -2.8, and the loss is (0.444 + 3.004 + 7.84) / 3, about 3.76. One step cut it by about 80 percent.

Same step with η = 0.2. The gradients are identical, so w becomes 3.733 and b becomes 1.6. Predictions jump to 5.333, 9.067 and 12.8 and the loss rises to about 27.67. The step overshot the valley floor and landed higher on the other side. Repeat that and training diverges.

That is why the learning rate is the first hyperparameter everyone tunes, and why a sudden spike in a training curve usually means a step was too big. Real training applies the same rule to millions of parameters, estimates gradients from small random batches (stochastic gradient descent) and uses optimisers such as momentum or Adam. Part 3 covers those and backpropagation.

Features carry most of the work. Real data needs scaling so no feature dominates the gradient, category encoding, missing value handling and derived ratios or counts. In classic ML, good features usually matter more than the algorithm. Every feature is also an input someone might control, so the security question for each column is: who can write to it?

Overfitting, regularisation, bias and variance

A model that memorises its training data can score perfectly on it and fail on anything new. That is overfitting; a model too simple to capture the pattern underfits.

Bias is error from wrong assumptions: a straight line fitted to a curve stays wrong however much data you add. Variance is sensitivity to the particular sample: an unpruned decision tree changes shape every time you resample. Expected error on new data splits into bias squared, plus variance, plus irreducible noise. More flexibility lowers bias and raises variance, and the balance point is found on data the model never trained on.

Technique	What it does	Effect	Source
L2 (ridge)	Adds λ times the sum of squared weights to the loss	Shrinks all weights toward zero	Hoerl and Kennard, Technometrics, February 1970
L1 (lasso)	Adds λ times the sum of absolute weights	Pushes some weights to exactly zero, selecting features	Tibshirani, JRSS Series B, 1996
Early stopping	Stops when validation loss stops improving	Limits how far parameters travel	Needs a validation set
Dropout	Randomly zeroes units while training a neural network	Prevents co-adapted units	Srivastava et al., JMLR volume 15, 2014
More or better data	Reduces variance directly	Changes what is learned	Often the most effective fix

With L2, each weight's gradient gains an extra 2λw, so every update also pulls the weight toward zero. Overfitting also has a privacy cost: a model that behaves differently on records it trained on gives a membership inference attacker something to measure, as the attack section shows.

The classic models you will meet

Tabular business and security data is still dominated by these models. They train fast on a CPU and are easier to explain to an auditor than a deep network.

Linear and logistic regression. Linear regression is the worked example above. Logistic regression passes the weighted sum through the sigmoid, 1 / (1 + e to the power of minus z), to get a probability, and trains on log loss. It is the baseline: if a complex model cannot beat a well-built logistic regression on held-out data, it is not earning its keep.

Decision trees. A sequence of yes or no questions about features ending in a prediction, each split chosen greedily to make the children purer (Gini impurity or entropy). No scaling needed, and badly prone to overfitting on their own.

Random forests. Leo Breiman's "Random Forests" (Machine Learning, volume 45, issue 1, October 2001) trains many trees on bootstrap samples, each considering a random subset of features at every split, and averages their votes. Averaging loosely correlated, high-variance trees cuts variance sharply.

Gradient boosting and XGBoost. Jerome Friedman's "Greedy function approximation: A gradient boosting machine" (The Annals of Statistics, volume 29, issue 5, October 2001) adds trees one at a time, each fitted to the negative gradient of the loss: gradient descent in function space. "XGBoost: A Scalable Tree Boosting System" (Chen and Guestrin, arXiv, March 2016) added a sparsity-aware algorithm, a weighted quantile sketch for approximate tree learning and systems work on cache access, compression and sharding. Boosted trees remain the usual first choice for tabular data.

Support vector machines. Cortes and Vapnik's "Support-vector networks" (Machine Learning, volume 20, issue 3, September 1995) finds the boundary with the widest margin to the nearest training points, with a soft margin for overlap and kernels for curved boundaries. SVMs were also the target of an early, widely cited poisoning paper (see the attack section).

If your problem looks like	Start with	Watch out for
Tabular data, thousands to millions of rows	Gradient boosted trees, with logistic regression as baseline	Leakage through target-derived features
A simple, explainable score for regulators	Logistic regression with L1 or L2	Non-linear effects you did not engineer in
Many noisy features, little tuning time	Random forest	Memory footprint at serving time
Small, clean data with a clear margin	SVM with a suitable kernel	Very large datasets
No labels, want groups	k-means, then inspect clusters	Choosing k; clusters that mean nothing
Images, audio, free text or huge data	Deep learning (part 3)	Compute cost and a larger attack surface

Evaluating a model without fooling yourself

Splits and cross-validation

Train on the training set, tune and choose models on the validation set, and touch the test set once at the end. The scikit-learn documentation (stable version 1.9.1 as of November 2026) says test data "should never be used to make choices about the model". For logs, transactions or anything time-ordered, split by time, not at random.

k-fold cross-validation steadies the estimate on limited data: split the training data into k folds, train on k minus 1, validate on the last, rotate and average. With 10,000 rows and k = 5, each round trains on 8,000 and validates on 2,000, and the spread of the five scores shows how much the estimate wobbles. Keep preprocessing inside the folds with a pipeline:

from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score

model = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000))
scores = cross_val_score(model, X_train, y_train, cv=5, scoring="f1")
print(scores.mean(), scores.std())

The scaler is refitted on each training split. Scaling the whole dataset before cross-validating is one of the leakage patterns scikit-learn's "Common pitfalls" page warns about.

Tuning hyperparameters without leaking

Hyperparameters (regularisation strength, tree depth, learning rate) are chosen on validation folds, never on the test set. Put the search around the whole pipeline so every candidate is scored on folds its preprocessing never saw:

from sklearn.model_selection import GridSearchCV

search = GridSearchCV(model, {"logisticregression__C": [0.01, 0.1, 1, 10]}, cv=5, scoring="f1")
search.fit(X_train, y_train)
print(search.best_params_, search.score(X_test, y_test))

In scikit-learn, C is the inverse of regularisation strength, so smaller values mean a stronger L2 penalty. Score the test set once, after the search is finished.

A confusion matrix, worked by hand

A phishing classifier is tested on 1,000 emails, 100 of them phishing:

	Predicted phishing	Predicted legitimate	Total
Actually phishing	80 (true positives)	20 (false negatives)	100
Actually legitimate	40 (false positives)	860 (true negatives)	900

Accuracy = (80 + 860) / 1,000 = 0.94
Precision = TP / (TP + FP) = 80 / 120 = 0.667: two in three flags were real.
Recall = TP / (TP + FN) = 80 / 100 = 0.80: it caught four in five.
F1 = 2 × 0.667 × 0.8 / (0.667 + 0.8) = 0.727
False positive rate = 40 / 900 = 0.044

A model that flags nothing scores 0.90 accuracy here, so accuracy mostly rewards the majority class.

Now deploy the same model, same recall and false positive rate, on 100,000 emails where only 1 percent are phishing. It catches 800 and wrongly flags 0.044 × 99,000 = 4,400 legitimate emails. Precision falls to 800 / 5,200, about 0.154, so roughly five in six alerts are false. The model did not change; the base rate did. Evaluate detectors at the prevalence you expect in production. Carlini et al. ("Membership Inference Attacks From First Principles", arXiv, December 2021) made the same argument for attacks, which they say should be judged by true positive rate at very low false positive rates.

from sklearn.metrics import confusion_matrix, classification_report

print(confusion_matrix(y_test, y_pred))
print(classification_report(y_test, y_pred, digits=3))

ROC AUC, calibration and leakage

ROC AUC summarises ranking quality across every threshold: 0.5 is random, 1.0 means every positive outranks every negative. On heavily imbalanced data the precision and recall curve says more about the alert volume analysts will face.

Calibration means a score of 0.8 is right about 80 percent of the time, which is how the scikit-learn calibration guide defines a well calibrated classifier. Guo et al. ("On Calibration of Modern Neural Networks", arXiv, June 2017) found modern neural networks poorly calibrated and temperature scaling surprisingly effective. Calibrate, for example with CalibratedClassifierCV, before building block, warn or ask thresholds on a probability.

Data leakage, in scikit-learn's words, "results in overly optimistic performance estimates". The usual forms are features derived from the label, preprocessing fitted before the split, duplicates across train and test, random splits on time series, and the same user or host on both sides. Leakage is the most common reason a model that shone in a notebook disappoints in production.

From notebook to production: the ML pipeline and MLOps

"Hidden Technical Debt in Machine Learning Systems" (Sculley et al., NIPS 2015) argued that model code is a small part of a real ML system, and that ML adds its own debt: boundary erosion, entanglement, hidden feedback loops, undeclared consumers, data dependencies, configuration issues and changes in the external world. Every item still describes production ML in 2026.

Stage	What to version or record	Typical failure
Data collection	Source, licence, snapshot hash, date	Unknown provenance; poisoned or mislabelled data
Data preparation	Transformation code and its dependency versions	Train and serve skew
Training	Code commit, data hash, hyperparameters, seeds, library versions	Models nobody can rebuild
Evaluation	Metrics per slice, test set version	Leakage; testing at the wrong base rate
Registry	Model version, lineage, approver, intended use	Shadow models outside the registry
Packaging and serving	Serialisation format, file hash, image digest	Unsafe deserialisation; exposed endpoints
Monitoring	Drift statistics, thresholds, retraining triggers	Silent drift; feedback loops
Retraining	New data snapshot and the reason for it	Learning from attacker-supplied or self-influenced data

Write down what each model is for. "Model Cards for Model Reporting" (Mitchell et al., arXiv, October 2018) proposed short documents released with a model that state its intended uses and its performance characteristics, so it is not used where it is not suited. A model registry is the system of record: which version is live, what data and code produced it, who approved it and for what use. Without one, nobody can answer "which model made this decision last March?", the first question from an incident responder or auditor. Datasets need the same versioning; a model version means little without its data version.

Drift comes in two kinds. Data drift is a shift in inputs, such as a new market or a new email client. Concept drift is a shift in what the inputs mean for the label, such as phishing kits changing their wording. Monitor input and score distributions against the training baseline, and real precision and recall once ground truth arrives. A slow, deliberate shift by an adversary looks exactly like ordinary drift.

Feedback loops are Sculley et al.'s hidden debt in action. A fraud model blocks transactions, so they never get outcome labels, so the next model trains on a biased sample. Any path from the model's outputs back into training is a path an attacker can steer.

Serialisation is a security decision

scikit-learn models are usually saved with pickle or joblib, which builds on it. Python's documentation is blunt: "The pickle module is not secure. Only unpickle data you trust." Unpickling can run arbitrary code by design. scikit-learn's model persistence page says pickle and joblib files should only be loaded from a trusted and verified source and lists alternatives, including skops.io, which it describes as more secure than pickle based formats, and ONNX for serving (it also notes an ONNX graph can represent arbitrary computation, so serve only models you trust). Treat any outside model file as untrusted code until it is scanned and its hash and origin are verified. Our explainer on the Python pickle security model covers the mechanism; part 3 covers deep learning formats such as safetensors and GGUF.

The machine learning attack surface

ML adds three new things to attack: the training data, the trained model and the inference interface. The attacks are over a decade old; what is new is shared vocabulary. For a wider tour of attacks on AI systems, see hacking AI.

Three reference frameworks, status as of November 2026

NIST AI 100-2 E2025, "Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations", published March 2025 (Vassilev, Oprea, Fordyce, Anderson, Davies and Hamin). NIST uploaded a corrected PDF on 1 April 2025 and a 3 June 2025 note flags a further fix for a future update. It follows the E2023 edition of January 2024. It has a predictive AI and a generative AI taxonomy and gives each class an ID: availability violations NISTAML.01, integrity violations NISTAML.02, privacy compromises NISTAML.03 and supply chain attacks NISTAML.05, with misuse (NISTAML.04) on the generative side.

MITRE ATLAS, a knowledge base of adversary tactics and techniques against AI systems, modelled on MITRE ATT&CK. The authoritative data is in the mitre-atlas/atlas-data repository; the latest release is 2026.10 (format 6.1.0, modified 28 October 2026), and it moved several IDs. In the older 5.6.0 data, membership inference was AML.T0024.000 and extraction AML.T0024.002. In 2026.10, AML.T0024 Exfiltration via AI Inference API is the transfer channel, membership inference is AML.T0137.000, behaviour replication is AML.T0005, and recovering parameters or architecture is AML.T0144 Recover AI Model Internals. Update threat models and detections that cite the old IDs. See our guide to using the MITRE ATLAS matrix.

OWASP Machine Learning Security Top 10, a lab project led by Shain Singh, Sagar Bhure and Ben Kereopa-Yorke. As of November 2026 its page says "The current version of this work is in draft and is being modified frequently." Its items carry 2023 labels: ML01 Input Manipulation, ML02 Data Poisoning, ML03 Model Inversion, ML04 Membership Inference, ML05 Model Theft, ML06 AI Supply Chain Attacks, ML07 Transfer Learning Attack, ML08 Model Skewing, ML09 Output Integrity Attack and ML10 Model Poisoning. Cite it as a draft.

Attack	Goal	MITRE ATLAS 2026.10	OWASP ML (draft)	NIST AI 100-2 E2025	Key paper
Data poisoning	Change what the model learns	AML.T0020 Training Data Poisoning; AML.T0115.000 Publish Poisoned AI Artifacts: Datasets	ML02; ML10 for tampered models	NISTAML.013 Data Poisoning; NISTAML.024 Targeted Poisoning; NISTAML.023 Backdoor Poisoning	Biggio, Nelson and Laskov, ICML 2012
Evasion	Misjudge a chosen input at inference	AML.T0015 Evade AI Model via AML.T0043 Craft Adversarial Data	ML01	NISTAML.022 Evasion; NISTAML.025 Black-box Evasion	Biggio et al., 2013; Szegedy et al., 2013
Extraction	Copy behaviour or recover internals	AML.T0005 Replicate AI Model Behavior (AML.T0005.001 from model outputs); AML.T0144 Recover AI Model Internals	ML05	NISTAML.031 Model Extraction	Tramèr et al., USENIX Security 2016
Membership inference	Tell whether a record was trained on	AML.T0137.000 Infer Membership	ML04	NISTAML.033 Membership Inference	Shokri et al., IEEE S&P 2017
Model inversion	Reconstruct sensitive attributes or examples	AML.T0137.002 Invert AI Model	ML03	NISTAML.032 Reconstruction	Surveyed in NIST AI 100-2 E2025

The vocabularies slice the same ground differently. NIST groups by objective, so data poisoning sits under both availability and integrity, and extraction counts as a privacy compromise because it leaks weights or architecture. The EU AI Act puts the list into law for high-risk systems: Article 15 asks for technical solutions to address "data poisoning", "model poisoning", "adversarial examples or model evasion" and "confidentiality attacks or model flaws", per the European Commission's AI Act Service Desk text.

Each attack in brief, with the defence

These posts go deep on each attack; here is the short version, without attack code.

Data poisoning. Crafted or mislabelled training data changes what the model learns: broadly worse, wrong on one target, or triggered by a hidden pattern. Biggio, Nelson and Laskov (ICML 2012) showed crafted points raising an SVM's test error and named the root cause: learning algorithms assume well-behaved data, which "does not generally hold in security-sensitive settings". Jagielski et al. (arXiv, April 2018) extended attacks and countermeasures to regression. Defend with dataset provenance and hashes, write controls on label stores, schema, duplicate, outlier and per-source quota checks, a trusted evaluation set every new model must match, and reviewed retraining. Deep dives: training data poisoning and tampered model weights.
Evasion. An input altered at inference is misjudged by a sound model. Biggio and colleagues presented gradient-based evasion testing of common classifiers in 2013 (posted to arXiv in August 2017), Szegedy et al. showed imperceptible perturbations flipping deep networks in December 2013, and Goodfellow, Shlens and Szegedy (December 2014) blamed the models' "linear nature". Defend with layered controls so no single model decides, minimal outputs and rate limits, and perturbation testing as a release metric. See adversarial AI attacks explained and types and defences.
Extraction. Queries through the normal interface are used to copy the model or recover its parameters. Tramèr et al. (USENIX Security 2016) extracted logistic regression, neural network and decision tree models "with near-perfect fidelity" from the BigML and Amazon Machine Learning services, and found that hiding confidence values did not close the gap. Defend with authenticated callers, per-key quotas, minimal outputs and monitoring for queries that sweep the input space. See model extraction attacks.
Membership inference. Model outputs reveal whether a record was trained on, which is the leak when the dataset itself is sensitive. Shokri et al. (IEEE S&P 2017) demonstrated it against models trained by commercial ML-as-a-service providers, and Carlini et al. reported a Likelihood Ratio Attack ten times more powerful at low false positive rates. Defend with less overfitting, differential privacy for sensitive data (Abadi et al., arXiv, July 2016, trained deep networks under a modest privacy budget), data minimisation and output limits. See membership inference attacks.

The supply chain around the model

Many ML incidents need no clever maths. ATLAS AML.T0010 Supply Chain Compromise and OWASP ML06 cover the ordinary route: a compromised package in the training environment, a tampered pretrained model, a swapped dataset or a model file that runs code when loaded. A notebook importing scikit-learn and pandas pulls in a deep tree of transitive packages, each running with access to your data and credentials. Our guide to securing models, notebooks and AI SDKs, the explainer on AI-BOMs and SBOM standards for AI and ML models cover the inventory side.

A dated timeline of machine learning and its security

Date	Event	Source
July 1959	Samuel's checkers learning paper	IBM Journal of Research and Development, volume 3
February 1970	Ridge regression (Hoerl and Kennard)	Technometrics, volume 12
March 1982	Lloyd's k-means iteration published	IEEE Transactions on Information Theory, volume 28
September 1995	Support-vector networks (Cortes and Vapnik)	Machine Learning 20(3)
October 2001	Random forests (Breiman); gradient boosting machine (Friedman)	Machine Learning 45(1); Annals of Statistics 29(5)
June 2012	Poisoning attacks against SVMs	ICML 2012, arXiv 1206.6389
2013	Gradient-based evasion attacks (Biggio et al.); adversarial examples in deep networks (Szegedy et al., December)	arXiv 1708.06131; arXiv 1312.6199
2015	"Hidden Technical Debt in Machine Learning Systems"	NIPS 2015
March 2016	XGBoost	arXiv 1603.02754
2016	Stealing models via prediction APIs	USENIX Security 2016
2017	Membership inference attacks	IEEE S&P 2017
March 2022	Instruction following with human feedback	arXiv 2203.02155
January 2024	NIST AI 100-2 E2023	NIST CSRC
March 2025	NIST AI 100-2 E2025	NIST CSRC
28 October 2026	MITRE ATLAS data release 2026.10	mitre-atlas/atlas-data

A practical checklist for ML teams

Inventory every model, with owner, training data source, serialisation format and serving location.
Pin and scan the ML dependency tree like application code.
Version data like code: no training run without a recorded data hash.
Register and approve every production model.
Load only trusted model files, preferring formats that cannot execute code.
Evaluate at production base rates and per slice.
Harden the inference interface with authentication, quotas, minimal outputs and logging.
Monitor drift and outcomes, and gate retraining on review.
Keep the evidence (model cards, evaluations, approvals) for EU AI Act, NIST AI RMF or ISO/IEC 42001 audits.

Common mistakes

Reporting accuracy on imbalanced data. Report precision, recall and F1 at the real base rate.
Tuning on the test set. Every look that changes a decision turns test data into training data.
Fitting preprocessing before splitting. Scalers and encoders belong inside the cross-validation pipeline.
Random splits on time series. Split by time.
Shipping a model without its data version. If you cannot rebuild it, you cannot investigate it.
Treating model files as data. A pickle file is a program.
Trusting uncalibrated scores as probabilities.
One model as the whole control. Then evading it is the whole attack.
Citing retired ATLAS IDs. Check against the 2026.10 data.

FAQ

What is machine learning in simple terms?

Software that learns patterns from examples instead of following hand-written rules. You supply data, a measure of error and a way to reduce it, and get a trained model that predicts on new data.

What is the difference between AI, machine learning and deep learning?

AI is the broad goal of machines doing tasks that need intelligence (part 1). Machine learning is the subset that learns from data. Deep learning is the subset of ML using many-layered neural networks (part 3, 10 November).

What are the main types of machine learning?

Supervised, unsupervised, self-supervised and reinforcement learning. Modern systems often chain them, for example self-supervised pretraining, supervised fine-tuning, then reinforcement learning from human feedback.

How does gradient descent work?

It computes the gradient of the loss for each parameter and moves each one a small step downhill. In our example a learning rate of 0.05 cut the loss from 18.67 to 3.76 in one step, while 0.2 overshot to 27.67.

What is the difference between precision and recall?

Precision is the share of flagged items that were truly positive; recall is the share of true positives that were flagged. F1 is their harmonic mean.

What is overfitting and how do you prevent it?

Learning the noise in training data so performance on new data drops. Use held-out validation, cross-validation, L1 or L2 regularisation, early stopping, dropout, simpler models and more data.

What is MLOps?

The engineering practice of moving models from experiment to production and keeping them healthy: versioned data and code, reproducible training, registries, evaluation, deployment, drift monitoring and controlled retraining.

What is data poisoning in machine learning?

Manipulating training data so the model behaves as an attacker wants. MITRE ATLAS lists it as AML.T0020 and the OWASP ML Top 10 draft as ML02.

What is a membership inference attack?

Deciding whether a specific record was in a model's training set from its outputs. ATLAS 2026.10 lists it as AML.T0137.000, NIST AI 100-2 E2025 as NISTAML.033.

Is the OWASP Machine Learning Security Top 10 final?

No. As of November 2026 OWASP describes it as a draft lab project, with 2023 item labels.

Where Safeguard fits

Safeguard does not train your models, judge whether your training data has been tampered with, or stop extraction and membership inference against an API you expose. Those are design and data controls, covered above. It covers the supply chain and runtime around your ML systems, where many of the failures in this post begin.

Know what you have. AI-BOM discovery inventories AI assets across code, cloud and connected SaaS: AI SDK imports such as transformers, model weight files, notebooks, prompt templates and MCP server configuration in repositories, and managed inference endpoints such as SageMaker, Bedrock and Vertex AI through cloud connectors. Per the AI-BOM and model security docs, an AI-BOM records model identity, weight provenance and hash, fine-tune lineage, declared dataset references and library dependencies, emitted as CycloneDX ML-BOM and the SPDX 3.0 AI profile, with integrations for Hugging Face, MLflow and the major cloud model registries.
Scan model files before you load them. AI-SPM statically analyses model artifacts without loading them. In pickle-based files it flags hidden code-execution instructions, mapped to CWE-502 and CWE-94. It unpacks PyTorch archives, confirms safetensors files are tensor-only, validates GGUF containers and inspects ONNX graphs for custom operators and external data paths that escape the model directory. Findings land in the unified findings model.
Guard models at runtime. The AI Gateway sits between an application and its model provider and checks prompts, responses and tool calls for direct and indirect prompt injection, jailbreak patterns and PII or secret egress. Monitor mode is the default, enforce mode can redact or block, and it fails open with an alert so the app never breaks. The AI Gateway post covers the policies.
Scan the dependencies. Safeguard's software composition analysis covers the Python packages in training and serving environments like any application code. The free Gold Open Source directory also lists open-weight models sourced from the Hugging Face Hub.
Keep governance evidence. The EU AI Act, NIST AI RMF and ISO/IEC 42001 are in Safeguard's framework list. Safeguard automates evidence collection and checks against them; it does not certify anyone.

ML systems are built from open source packages, pretrained components and container images, and every retraining run pulls them again. When one of those dependencies carries a known vulnerability, the same remediation path you use for application code applies.

Fix it without waiting for upstream

Whatever the vulnerability, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.

1. Autonomous remediation: pull requests, not tickets

When a fixed version exists, Griffin, Safeguard's security model, opens a policy gated pull request that moves the affected dependency to the safe version, and your own CI runs your tests on it before anyone merges. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation categories include dependencies, licenses, security posture and findings, and each one is set separately. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Auto-merge is a separate switch and stays off by default, so your team decides what merges on its own and what still needs a human approval.

2. Gold Open Source: the safe version in seconds, free

gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds.

3. Gold Registry: start from components that are already fixed

Safeguard Gold rebuilds patched versions of open-source packages and images from source, signs each build, attaches an SBOM, SLSA provenance and VEX, and serves them from registry.safeguard.sh to your organization. Every Gold artifact is held continuously to zero exploitable critical or high CVEs and a completed license check, with signed build provenance (signed with Safeguard's key and verifiable with cosign, SLSA Build Level 3 as the design target) and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.

When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened build of a specific package. Ask which packages it covers before you plan around it.

Your situation	What to use
A fixed version exists	Autonomous remediation opens the upgrade pull request; Gold Open Source confirms the safe version
You need to know today whether you are affected	Gold Open Source lookup and "Am I affected?" checker, free and without login
You want vulnerable versions to stop shipping	Safeguard policies and gates in CI
Upstream has no fix yet, or will not ship one	A custom Gold request; fork and patch
You want to start clean on new projects	Pull hardened components and images from Gold Registry

One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.

See how Safeguard would help

The first demo is a real Safeguard CLI session: install, ask it to fix a vulnerable dependency in a repository, approve each step that changes something, then scan. A training repository with a pinned scikit-learn, pandas or numpy version is handled the same way: find the vulnerable package, move it to a fixed version with your approval, and let your own tests confirm the model still trains.

Click through it at your own pace, or open it full screen.

The second demo sets up the Safeguard local runner on your own machine, approves it with its device code, and runs a scan from the terminal. That suits ML teams whose notebooks and training data cannot leave a workstation or a private build host.

Click through it at your own pace, or open it full screen.

To try it yourself, start at safeguard.sh/download and the CLI docs.

Quick check

A linear model starts at w = 0, b = 0 on the points (1, 2), (2, 4), (3, 6). With learning rate 0.05, what are w and b after one step?

w = 0.933 and b = 0.4. The gradients are about -18.67 and -8, and each parameter moves by minus 0.05 times its gradient. The loss falls from about 18.67 to about 3.76.

A classifier has 80 true positives, 40 false positives and 20 false negatives. What are precision, recall and F1?

Precision 80 / 120 = 0.667, recall 80 / 100 = 0.80, F1 about 0.727.

Why did the same phishing model's precision fall to about 0.154?

The base rate fell to 1 percent. A 0.044 false positive rate on 99,000 legitimate emails gives 4,400 false alarms against 800 true detections.

In MITRE ATLAS 2026.10, which ID covers membership inference?

AML.T0137.000 Infer Membership. In the older 5.x data it was AML.T0024.000.

Why is loading a pickle-based model file from an unknown source dangerous?

Unpickling can execute arbitrary code by design. Python's documentation says to only unpickle data you trust, and scikit-learn repeats the warning for model files.

Sources

Last reviewed: 2 November 2026.