Facial Recognition Explained: How Face Matching Works and the Law

Facial recognition is software that finds a face in an image, turns it into a list of numbers called a template, and compares it with stored templates to get a similarity score; above a threshold, it reports a match. Verification (1:1) checks a claimed identity, such as phone unlock. Identification (1:N) searches a database for whoever looks most similar.

The privacy stakes depend on where the template lives and who decided to collect it. A phone that keeps your face data in a hardware enclave and never uploads it is a very different system from a camera at a shop door that searches every visitor against a watchlist. Illinois' Biometric Information Privacy Act, the EU General Data Protection Regulation and the EU AI Act all draw lines between those cases, and the US Federal Trade Commission has said that misusing biometric information can be an unfair practice.

This post sits beside our explainers on automated license plate readers, browser fingerprinting and photo metadata; this one covers tracking by your face, which you cannot change or leave at home. Legal summaries are for orientation, not legal advice. We do not name system vendors, and nothing here explains how to defeat a face recognition system.

Key takeaways

• The pipeline is detect, align, extract a template, compare. The score is "vendor-defined" (NIST) and a threshold turns it into match or no match.
• A false match links two different people; a false non match misses the same person. Moving the threshold trades one for the other.
• In a 1:N search every extra person in the gallery is another chance for a false match, so a threshold safe for phone unlock can flood a large database search with wrong candidates.
• NISTIR 8280 (December 2019) tested 189 algorithms from 99 developers on 18.27 million images and found false positive rates varying "by factors of 10 to beyond 100 times" across demographic groups in many algorithms.
• Apple puts the odds of a random person unlocking Face ID below 1 in 1,000,000 and says the data never leaves the device. Android Class 3 biometrics must reach a false acceptance rate of 1 in 50,000.
• BIPA requires a written release and a public retention schedule and lets individuals sue. GDPR treats biometric data used to identify someone as special category data. The EU AI Act has banned untargeted facial image scraping since 2 February 2025.
• Most failures are organisational: treating a score as identity, no retention limit, and no bias testing evidence.

How does facial recognition work?

Every face recognition system, from a phone to a border gate, runs some version of the same pipeline. NIST's demographic study gives the clearest plain language description: face recognition algorithms "include a face detector followed by a feature extraction algorithm that converts one or more images of a person into a vector of values that relate to the identity of the person." The extractor is usually a neural network trained on labelled photos, and in operation it acts as a generic extractor "of identity-related information from photos of persons they have usually never seen before."

Stage 1: face detection

Detection asks whether there is a face in the frame and where. It outputs a bounding box and a confidence value. On a fixed camera it runs on every video frame, so one entrance camera can produce thousands of face crops per hour even if nobody is identified.

Detection is not recognition. A system that only counts or blurs faces runs a detector and stops, and the law cares about that difference: GDPR and the EU AI Act turn on whether data is processed to identify a person.

Stage 2: alignment and quality checks

Alignment finds landmarks such as the eye centres, nose tip and mouth corners, and rotates and scales the crop so those points land in standard positions. Many systems also score quality here (blur, pose, lighting, occlusion) and reject poor crops, because they drive errors later.

Image quality is one of the strongest drivers of accuracy. NISTIR 8280 found that false negatives with high quality mugshots were highest for Asian and American Indian individuals, but with lower quality border crossing photos they were generally higher for people born in Africa and the Caribbean. NIST attributed the difference to image quality: the mugshots followed a standardised capture setup, the border photos did not.

Stage 3: feature extraction (the template)

The aligned crop goes into a deep neural network that outputs a fixed length vector, commonly a few hundred numbers. This is the template, also called an embedding or feature vector. Two photos of the same person should produce vectors that point in nearly the same direction; photos of different people should produce vectors that point in different directions.

A template is not a photo, but it is still personal data. It is built to stay stable for one person across years and cameras, which is what makes a leaked template harmful: you can reset a password, not your face.

Stage 4: comparison, score and threshold

Comparison computes a similarity score between two templates. A common choice is cosine similarity, the cosine of the angle between the two vectors, where 1 means identical direction. NIST is careful to say the score is "vendor-defined," so a 0.6 from one product means nothing on another product's scale.

A score at or above the threshold is a match. NIST calls same person comparisons "mate" scores and different person comparisons "nonmate" scores. The two distributions overlap, so wherever you put the threshold some nonmates land above it and some mates below it.

The open source OpenCV library's face recognition tutorial makes this concrete. It publishes thresholds for one recognition model, SFace, on several public benchmarks. On the LFW benchmark it lists 99.60 percent accuracy at a cosine similarity threshold of 0.363 (or a normalised L2 distance of 1.128). On the harder CFP-FP benchmark, with large pose changes, the same model scores 94.80 percent and its listed cosine threshold drops to 0.212. Same model, different photos, different operating point.

A minimal illustration of the comparison step, with made up four number vectors (real templates have hundreds):

import numpy as np

def cosine(a, b):
    a, b = np.asarray(a, float), np.asarray(b, float)
    return float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b)))

enrolled   = [0.12, 0.85, -0.33, 0.40]   # template stored at enrolment
same_later = [0.10, 0.80, -0.30, 0.47]   # same person, new photo
someone    = [0.70, 0.10,  0.55, -0.20]  # a different person

THRESHOLD = 0.90  # illustrative; real thresholds are model specific

for name, probe in [("same person", same_later), ("different person", someone)]:
    s = cosine(enrolled, probe)
    print(f"{name}: score={s:.3f} match={s >= THRESHOLD}")

It prints a score near 0.995 for the same person (a match) and a negative score for the different person (no match). The system never "knows" who anyone is; it only knows whether a number crossed a line someone chose.

Presentation attack detection (liveness)

A camera can be shown a photo, a screen or a mask. Presentation attack detection, often called liveness, tries to tell a live face from an artefact. Apple says Face ID "always requires depth information, which isn't found in print or 2D digital photographs" and uses anti spoofing neural networks. NIST's digital identity guideline SP 800-63B requires presentation attack detection for facial recognition used in authentication, and says testing should show an impostor attack presentation accept rate below 0.07. Android measures the same thing as a spoof acceptance rate (SAR), a metric introduced in Android 9.

What is the difference between verification (1:1) and identification (1:N)?

Verification answers "is this the person they claim to be?" One probe image is compared with one enrolled template. Unlocking a phone, matching a traveller's face to the photo on their passport, and checking an employee badge photo at a turnstile are all 1:1.

Identification answers "who is this?" with no claim attached. The probe is compared with every template in a gallery and the system returns the top candidates or all candidates above a threshold. NISTIR 8280 gives examples ranging from building access with no badge to checking a list of gamblers banned from a casino.

Why 1:N multiplies false matches: a worked example

Suppose an algorithm has a false match rate (FMR) of 1 in 100,000 at a given threshold. That is a respectable figure for one comparison.

Example 1: phone unlock. A thief tries to unlock your phone with their own face. One comparison. The chance of a false match is 0.00001, or 0.001 percent. Apple adds a cap of five failed attempts before the passcode is required, so even repeated tries stay tiny.

Example 2: a retail watchlist of 10,000 people. A shopper who is on no list walks past the camera. The system compares their template with 10,000 enrolled templates. The probability that at least one comparison falsely crosses the threshold is 1 minus (1 minus 0.00001) to the power 10,000, which is about 1 minus 0.905, or roughly 9.5 percent. Nearly one innocent shopper in ten triggers a candidate alert.

Example 3: a gallery of 10 million. Run the same arithmetic with N equal to 10,000,000. The expected number of false candidates per search is N times FMR, which is 100. Every single search of someone not in the database returns around a hundred wrong names above threshold.

The model assumes independent comparisons, but the direction holds. Large 1:N systems therefore raise the threshold, return a short candidate list for trained human review, or both, and NIST reports 1:N accuracy as a false positive identification rate (FPIR) rather than the 1:1 FMR. "The system found a match" is never, on its own, an identification.

How is facial recognition accuracy measured?

Accuracy is two numbers, never one. "99 percent accurate" means little without the error type, threshold and image set.

NIST's SP 800-63B, the US federal digital identity guideline, turns these into hard requirements for biometrics used in authentication. The biometric system "SHALL operate with an FMR of one in 10000 or better for all demographic groups," with sex and skin tone considered where they affect performance; it "SHOULD demonstrate" an FNMR below 5 percent; performance shall be tested to ISO/IEC 19795-1; and the system "SHALL be configured with a fixed threshold" because "it is not feasible to change the threshold for each demographic." It also allows biometrics only as part of multi factor authentication with a physical authenticator.

What is NIST FRTE (formerly FRVT)?

NIST ran its ongoing algorithm tests for years as the Face Recognition Vendor Test (FRVT). In NIST's words, "what was formerly known as FRVT has been rebranded and split into FRTE (Face Recognition Technology Evaluation) and FATE (Face Analysis Technology Evaluation)." Identity tasks such as 1:1 verification and 1:N identification run under FRTE; tasks that analyse images without identifying anyone, such as morph detection, image quality, presentation attack detection and age estimation, run under FATE.

Developers submit algorithms, NIST runs them on its own operational image sets (visa, border crossing and kiosk photos among them) and publishes error rates at fixed operating points. FRTE "runs continuously" and participants may submit "as often as every four calendar months." The FRTE 1:1 leaderboard initially ranks algorithms by false non match rate at a false match rate of 1 in a million on its combined visa and border dataset. NIST also notes that multiple faces appear in about 3 percent of border images and 7 percent of kiosk images, the kind of detail that breaks naive deployments.

FRTE results describe algorithms, not deployed systems. NIST says so directly in NISTIR 8280: it is "incumbent upon the system owner to know their algorithm," and it will usually be informative to "measure accuracy of the operational algorithm on the operational image data."

What did NIST IR 8280 find about demographic differentials?

NISTIR 8280, "Face Recognition Vendor Test (FRVT) Part 3: Demographic Effects," was published in December 2019 by Patrick Grother, Mei Ngan and Kayee Hanaoka. It is worth reading what it says rather than headlines about it.

What NIST tested. 189 mostly commercial algorithms from 99 developers, run over four US government datasets: domestic mugshots, immigration benefit application photos, visa photos and border crossing photos. In total, 18.27 million images of 8.49 million people. Demographic groups were defined by sex, age, and race (for mugshots) or country of birth (for the other sets), restricted to 24 countries in 7 regions.

Main result. "False positive differentials are much larger than those related to false negatives and exist broadly, across many, but not all, algorithms tested." False positive rates "often vary by factors of 10 to beyond 100 times" across demographics. False negatives "tend to be more algorithm-specific, and vary often by factors below 3."

False positives in detail.

• On high quality application photos, false positive rates were highest in West and East African and East Asian people and lowest in Eastern European individuals, with a factor of 100 between countries in many algorithms.
• With a number of algorithms developed in China, the East Asian effect was reversed, with low false positive rates on East Asian faces. NIST's point: training data and development choices shape who the algorithm errs on.
• With domestic mugshots, false positives were highest in American Indians, with elevated rates in African American and Asian populations.
• False positives were higher in women than men across algorithms and datasets, an effect smaller than race.
• False positives were elevated in the elderly and in children, smallest in middle aged adults.

False negatives in detail. With mugshots, higher in Asian and American Indian individuals; with lower quality border crossing photos, higher in people born in Africa and the Caribbean, especially older people.

The exception that matters for buyers. NIST found that "some developers supplied highly accurate identification algorithms for which false positive differentials are undetectable." Differentials belong to specific algorithms and conditions, so due diligence must cover the exact algorithm version you will run.

A reporting lesson. Since most systems run at a fixed threshold, NIST says it is "necessary to report both false negative and false positive rates for each demographic group at that threshold." It also noted that much public discussion of bias cited studies of gender classification algorithms, a different kind of software.

Why the consequences differ by use. In verification, NIST notes, false negatives mostly cause inconvenience and a retry. In identification uses such as visa fraud checks or surveillance, "a false positive match to another individual could lead to a false accusation, detention or deportation."

Where is facial recognition used, and where are templates stored?

The useful question is not "is face recognition used here?" but "who holds the template, who chose to enrol, and what is it compared against?"

Phone and laptop unlock

On modern phones the template is created and stored on the device. Apple says the TrueDepth camera projects "thousands of invisible dots to create a depth map" plus an infrared image, which the Secure Enclave protected part of the neural engine turns into a mathematical representation. Face ID data "doesn't leave your device, is never available to iOS or iPadOS, apps, or Apple, and is never backed up to iCloud or anywhere else." Apple's Platform Security guide adds that the Secure Enclave "processes, encrypts, and stores" the template data and does the matching.

Apple puts the odds of a random person unlocking the device below 1 in 1,000,000 for one enrolled appearance, higher for twins, siblings and children under 13, and requires the passcode after five failed matches, a restart or 48 hours without unlock.

Android sets requirements by class in its biometric security documentation. A Class 3 (formerly Strong) biometric must meet a false acceptance rate of 1 in 50,000 and a false rejection rate of 10 percent, keep its spoof acceptance rate within set limits (0 to 7 percent across presentation attack instrument species), and run a secure pipeline in which kernel or platform compromise cannot read raw biometric data or inject synthetic data. Class 3 biometrics can be exposed to apps through the BiometricPrompt or FIDO2 APIs.

Apple says Face ID data is never available to apps; an app asks the system to authenticate you and gets a result. Our post on crypto bound versus callback only biometric login on Android explains why that key binding matters.

Airports and borders

The US Transportation Security Administration says it is testing 1:1 facial matching that takes a picture of the passenger and compares it to the photo on their ID, using its Credential Authentication Technology (CAT-2), and says passengers "may choose not to have a photo taken" and ask for a standard ID check instead. US Customs and Border Protection describes its own facial comparison programme on its biometrics pages. Read each agency's privacy notice for retention details.

Workplaces and access control

Access control and time clocks match against a gallery of employees (1:N) or the badge holder's enrolled template (1:1). Templates sit on an on premises server or a vendor's cloud. Illinois' BIPA anticipates this setting: its definition of a written release includes, in employment, a release executed by an employee as a condition of employment.

Retail and venues

Retail identification compares shoppers' faces with a watchlist of people the operator has flagged. Shoppers have not enrolled and may not know a comparison happened, which is the kind of "surreptitious and unexpected collection" the FTC lists among possibly unfair practices. The European Commission's own summary of the AI Act gives a retail example of a high risk use: an AI system used "to retroactively identify a shoplifter."

Policing and investigations

Police can search a still from a camera against a gallery of custody or licence photos and get a ranked candidate list for an examiner to review. The European Data Protection Board's Guidelines 05/2022 (final version dated 17 May 2023) cover law enforcement use in the EU, where real time remote identification in public spaces is prohibited outside narrow, authorised cases.

Online face search and photo apps

Photo apps group pictures by person, on the device or in the provider's cloud depending on the app. Public face search services go further: they collect images from the open web into a 1:N gallery of people who never enrolled. The EU AI Act now bans building such databases by untargeted scraping; recital 43 explains that the practice "adds to the feeling of mass surveillance" and can lead to gross violations of fundamental rights.

On device face unlock versus cloud face search: what is different for privacy?

The same mathematics can be private or invasive depending on four design choices.

Who enrols. With phone unlock you enrol yourself and can delete the template. With cloud face search, the gallery is built from photos of people who were never asked.

Where the template lives. A template in a hardware enclave on your device cannot be searched by anyone else. One in a cloud database can be searched, copied, subpoenaed, breached or reused for a new purpose.

What it is compared against. Phone unlock compares one live face with one stored template. Cloud search compares one uploaded face with millions or billions of templates, so the false match arithmetic above applies in full.

What happens after a match. A phone match unlocks a key you already own. A cloud match produces names and links about a person who may not know they were searched, and a false match attaches the wrong identity to a stranger.

The law tracks these differences: the EU AI Act keeps pure verification out of its remote biometric identification category, and saves its prohibitions and high risk rules for identification done at a distance without the person's active involvement.

What do privacy laws say about facial recognition?

What follows summarises primary legal texts and regulators' own statements. Laws change and courts interpret them; consult a qualified lawyer for advice about a specific deployment.

Illinois Biometric Information Privacy Act (BIPA)

The Illinois Biometric Information Privacy Act, 740 ILCS 14, enacted in 2008, is the US state law with the most direct consequences for face recognition, because it lets individuals sue.

• What it covers. A "biometric identifier" includes a "scan of hand or face geometry." "Biometric information" is information based on a biometric identifier used to identify an individual. Photographs themselves are excluded from the definition of biometric identifier, but the face geometry derived from them is not.
• Written policy and retention (Section 15(a)). A private entity in possession of biometric data must publish a written policy with a retention schedule and guidelines for permanent destruction, destroying the data when the initial purpose has been satisfied or within 3 years of the individual's last interaction with the entity, whichever comes first.
• Notice and written release (Section 15(b)). Before collecting, the entity must tell the person in writing that biometric data is being collected, the specific purpose and the length of time it will be stored, and receive a written release.
• No selling or profiting (Section 15(c)). No private entity may sell, lease, trade or otherwise profit from a person's biometric data.
• Disclosure limits and security (Sections 15(d) and 15(e)). Disclosure needs consent or another listed basis, and the data must be protected with the reasonable standard of care in the entity's industry, at least as protective as for other confidential information.
• Private right of action (Section 20). A person aggrieved by a violation may recover liquidated damages of 1,000 dollars per negligent violation or 5,000 dollars per intentional or reckless violation, or actual damages if greater, plus reasonable attorneys' fees and costs.
• 2024 amendment. Public Act 103-0769 amended Section 20 so that repeated collection of the same biometric from the same person by the same method counts as a single violation, and added electronic signatures as a valid written release.

Washington and other US states

Washington's biometric identifiers law, chapter 19.375 RCW (2017), takes a different route. A person may not enroll a biometric identifier in a database for a commercial purpose without first providing notice, obtaining consent or providing a mechanism to prevent later use for a commercial purpose. Its definition of "enroll" is notable: capturing a biometric identifier and converting it "into a reference template that cannot be reconstructed into the original output image." There is no private right of action; RCW 19.375.030 says the chapter "may be enforced solely by the attorney general" under the state consumer protection act. Several other states have biometric provisions in their own statutes or comprehensive privacy laws; check each statute's current text rather than relying on summaries.

FTC policy statement on biometric information

On 18 May 2023 the Federal Trade Commission issued a policy statement on biometric information under Section 5 of the FTC Act. In deciding whether a use is unfair, the FTC said it will consider factors including failing to assess foreseeable harms before collection, failing to address known or foreseeable risks, "surreptitious and unexpected collection or use of biometric information," failing to evaluate third parties such as vendors, failing to train staff, and failing to monitor the technology after deployment. It is enforcement policy, not a new rule, but it tells US businesses what the regulator expects to see documented.

GDPR: biometric data as special category data

Under the EU General Data Protection Regulation (Regulation 2016/679), and its UK equivalent:

• Definition (Article 4(14)). "Biometric data" means personal data resulting from specific technical processing relating to the physical, physiological or behavioural characteristics of a person, which allow or confirm the unique identification of that person, such as facial images or fingerprint data.
• Photos are not automatically biometric data (Recital 51). Processing of photographs is not systematically special category processing; photos are covered by the biometric definition only when processed through specific technical means that allow unique identification.
• Prohibition with exceptions (Article 9). Processing biometric data "for the purpose of uniquely identifying a natural person" is prohibited unless an Article 9(2) condition applies, such as explicit consent, employment law obligations, vital interests, or substantial public interest on the basis of law.
• Impact assessment (Article 35). A data protection impact assessment is required where processing is likely to result in high risk, including large scale processing of special categories and systematic monitoring of publicly accessible areas on a large scale.
• Automated decisions (Article 22). People have the right not to be subject to a decision based solely on automated processing that produces legal or similarly significant effects, with limited exceptions.

The UK Information Commissioner's Office's biometric recognition guidance applies the same concepts in practical terms, explaining when biometric data is special category data and how to protect templates. The ICO notes that the guidance is under review after the Data (Use and Access) Act.

EU AI Act: prohibitions and high risk rules

The EU Artificial Intelligence Act (Regulation 2024/1689) entered into force on 1 August 2024. Per the European Commission, prohibited practices and AI literacy duties applied from 2 February 2025, obligations for general purpose AI models from 2 August 2025, and the Act became generally applicable on 2 August 2026, with exceptions covered below. For face recognition the key provisions are:

• Prohibited: untargeted scraping (Article 5(1)(e)). AI systems that create or expand facial recognition databases through untargeted scraping of facial images from the internet or CCTV footage.
• Prohibited: emotion recognition (Article 5(1)(f)) in workplaces and education institutions, except for medical or safety reasons.
• Prohibited: biometric categorisation (Article 5(1)(g)) that deduces sensitive characteristics such as race, political opinions, religious beliefs or sexual orientation.
• Prohibited with exceptions: real time remote biometric identification in publicly accessible spaces for law enforcement (Article 5(1)(h)). Allowed only where strictly necessary for listed objectives: targeted search for specific victims of abduction, trafficking or sexual exploitation and missing persons; prevention of a specific, substantial and imminent threat to life or a genuine and present or foreseeable terrorist threat; and locating or identifying suspects of offences listed in Annex II that carry a maximum sentence of at least four years. Each use needs prior authorisation by a judicial or independent administrative authority; in a duly justified urgency use may start first, but authorisation must be requested within 24 hours and, if refused, use stops and the data is deleted.
• High risk: remote biometric identification. Remote biometric identification systems are listed as high risk in Annex III. The Act explains that this excludes biometric verification whose sole purpose is to confirm a person is who they claim to be, such as "unlocking a device or having security access to premises." High risk systems carry duties including risk management, logging, human oversight and accuracy.
• Definitions to know. Article 3 defines "biometric identification" as recognising features to establish identity by comparing a person's biometric data with data of individuals stored in a database, and "biometric verification" as "automated, one-to-one verification, including authentication." Those two definitions map exactly onto the 1:N and 1:1 modes described earlier.

When do the high risk rules apply? This date has moved. The Act as published set Annex III high risk rules to apply from 2 August 2026. The European Commission's AI Act page now says the rules for high risk use cases in the sensitive areas of Annex III, which include remote biometric identification, "have been extended to 2 December 2027" following political agreement on its "AI Omnibus" simplification proposal, and that rules for high risk AI embedded in regulated products under Annex I have a transition period until 2 August 2028. The prohibitions in Article 5 are not affected: they have applied since 2 February 2025.

Comparison: four face recognition deployments

Step by step: what individuals can do

None of these steps can stop a camera from photographing you in public. They limit where your face data is stored and how easy it is to find.

1. Review face data on your phone

On an iPhone or iPad Pro, open Settings, then Face ID & Passcode, and turn off individual uses or reset Face ID. Apple recommends a passcode if you are concerned about lookalike siblings or children.

On Android, look under Settings for the security section and the face unlock entry, and remove the face model if you do not use it; menu names vary by manufacturer.

2. Review face grouping in photo apps

Check whether your photo app groups faces on the device or in the cloud. Google Photos, for example, has a "Face Groups" setting; Google's help page says that turning it off deletes your face groups, the face models used to create them and the face labels you created. For other apps, check whether switching grouping off deletes face models or only hides them.

3. Limit public photos and strip metadata

Public face search tools work only with photos they can find. Review which profile and tagged photos are public, ask friends before tagging, and remove location metadata before posting. Our photo metadata guide covers this in depth. A quick check with the open source exiftool:

# Show location tags in a photo before you post it
exiftool -gps:all -a -G1 photo.jpg

# Write a copy with all metadata removed (keeps the original as photo.jpg_original)
exiftool -all= photo.jpg

4. Use your rights where you have them

• In the EU and UK, you can submit a subject access request under GDPR Article 15 to any organisation you believe holds your biometric data, and request erasure under Article 17 where the conditions apply.
• In Illinois, a private entity holding your biometric data must have a public retention and destruction policy; ask for it.
• At US airport checkpoints, TSA says facial comparison "is voluntary": tell the officer you do not wish to participate and they will use the standard ID check.

Step by step: a data protection impact assessment checklist for organisations

Run this before you sign a contract. GDPR Article 35(3) requires a DPIA for large scale processing of special category data and for systematic monitoring of publicly accessible areas on a large scale, which covers most face recognition deployments; elsewhere it is still the cheapest way to avoid an FTC or BIPA problem.

1. Purpose and necessity. State the problem in one sentence. If a badge, PIN or human check would solve it, stop.
2. Mode. Prefer verification (1:1); never use 1:N if 1:1 meets the need.
3. Lawful basis. Document the GDPR Article 9(2) condition, the BIPA notice and written release, and other state requirements.
4. Algorithm evidence. Ask the vendor for the exact algorithm name and version, its NIST FRTE results, and FMR and FNMR at the threshold you will use, broken down by demographic group. NISTIR 8280 says the system owner must "know their algorithm."
5. Operational testing. Test on your own cameras, lighting and population before go live. Record FMR and FNMR per group at your threshold.
6. Threshold and human review. Fix and document the threshold; require trained human review of every match before action.
7. Data minimisation. Discard non matching probes immediately.
8. Retention schedule. Set and publish retention periods for templates, probes, alerts and logs, with automated deletion. BIPA's ceiling is three years from the last interaction; shorter is better.
9. Security. Encrypt templates at rest, separate them from identity records, restrict access by role, and log every search.
10. Watchlist governance. Define who can add a person to a watchlist, on what evidence, for how long, and how they are removed.
11. Transparency and redress. Tell people before collection, explain how to contest a match, and give a non biometric alternative.
12. Monitoring. Review error rates, complaints and demographic outcomes on a schedule, and set a trigger for switching the system off.
13. Vendor and software review. Require the vendor to disclose the software components and models in the system, how they are patched, and where data is processed.

A retention rule is only real if something enforces it. An illustrative policy and nightly purge for a self hosted system on PostgreSQL:

# retention-policy.yaml (illustrative)
biometric_retention:
  non_matching_probes: 0d        # never persisted
  match_alert_images: 30d
  reviewed_false_matches: 7d
  enrolled_templates:
    delete_after_last_interaction: 365d
  search_audit_log: 730d         # keep the log, not the faces

-- Nightly purge, run by a scheduled job with a dedicated role
DELETE FROM match_alerts
 WHERE created_at < now() - interval '30 days';

DELETE FROM enrolled_templates
 WHERE last_interaction_at < now() - interval '365 days';

-- Prove it ran: write a row the auditor can see
INSERT INTO retention_runs(run_at, table_name, rows_deleted)
VALUES (now(), 'match_alerts', 0); -- replace 0 with the DELETE row count in your job

Common mistakes

Treating a match score as identity. A score above threshold means two templates are similar, nothing more. NISTIR 8280 warns that in identification uses a false positive "could lead to a false accusation, detention or deportation." Every 1:N match needs independent corroboration before anyone acts.

Using a 1:1 threshold for a 1:N search. The larger the gallery, the more false candidates a fixed threshold returns; set thresholds and review for the gallery size.

No retention limit. Templates, probes and alert images pile up by default. BIPA requires a public retention schedule and destruction rule; GDPR requires storage limitation.

Buying a system without bias testing evidence. NIST found differentials of 10 to 100 times in many algorithms and undetectable differentials in some. Without demographic error rates for the exact algorithm version at your threshold, you do not know which kind you bought.

Relying on signage as consent. A sign at a shop door may satisfy a notice duty in some places but is not a BIPA written release and is not GDPR explicit consent.

Forgetting the software around the model. A face recognition system is also a web console, a database, camera firmware, an API and dozens of open source libraries. An unpatched admin console exposes every template.

FAQ

How does facial recognition work in simple terms?

Software finds a face, straightens it, turns it into a list of numbers (a template) and compares that list with stored templates. If the similarity score passes a chosen threshold, it reports a match.

What is the difference between facial recognition and face detection?

Detection only finds faces in an image. Recognition compares a face with stored templates to verify or identify a person.

Is Face ID facial recognition?

Yes, it is 1:1 verification. Apple says the template is stored and matched in the Secure Enclave and never leaves the device.

How accurate is facial recognition?

It depends on the algorithm, the threshold, image quality and the population. Ask for both false match and false non match rates at a stated threshold, per demographic group.

What did NIST find about facial recognition bias?

NISTIR 8280 found false positive rates varying by 10 to over 100 times across demographic groups in many algorithms, higher for women, the elderly and children, and undetectable differentials in some of the most accurate algorithms.

Can I opt out of facial recognition at the airport?

At US TSA checkpoints, yes: TSA calls facial comparison "voluntary" and offers a standard ID check instead. Other operators and countries differ.

Does a photo count as biometric data under GDPR?

Not automatically. Recital 51 says a photo is biometric data only when processed by specific technical means that allow unique identification, such as building a template.

How long can a company keep my face data in Illinois?

BIPA requires destruction when the initial purpose is satisfied or within 3 years of your last interaction, whichever comes first, under a published retention schedule.

Where Safeguard fits

Safeguard does not evaluate face recognition algorithms, test for bias, analyse cameras or give legal advice. Its fit is indirect: a face recognition deployment is also an ordinary software estate of vision libraries, inference runtimes, a web console, an API and a database, much of it open source packages and container images. Safeguard can build a software bill of materials for those repositories and images, match components against vulnerability databases, show which of your projects contain a given component version, and open fix pull requests when a fixed version exists. That helps with item 13 of the checklist above. It does not tell you whether the system is lawful or fair.

Biometric systems run on open source vision libraries, inference runtimes and model loaders, and those dependencies get vulnerabilities like any other code. When a template database sits behind a console built from hundreds of packages, how quickly you can move to a fixed version is part of how well you protect the faces in it.

Fix it without waiting for upstream

Whatever the vulnerability, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.

1. Autonomous remediation: pull requests, not tickets

When a fixed version exists, Safeguard's Griffin models open a tested, policy gated pull request that moves the affected dependency to the safe version. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation categories include dependencies, licenses, security posture and findings, and each one is set separately. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Autonomous remediation ships switched off, with every category set to Manual, so your team decides how much runs on its own and what still needs a human approval.

2. Gold Open Source: the safe version in seconds, free

gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds.

3. Gold Registry: start from components that are already fixed

Gold Registry provides hardened, drop in replacements for upstream packages and container images. Every Gold artifact is held continuously to zero exploitable critical or high CVEs and a completed license check, with signed build provenance (signed with Safeguard's key and verifiable with cosign, SLSA Build Level 3 as the design target) and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.

When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened, compatibility tested build of a specific package. Ask which packages it covers before you plan around it.

One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.

See how Safeguard would help

The first demo is a real Safeguard CLI session: the assistant fixes a lodash CVE, asking before any command that changes something, edits package.json to lodash 4.17.21, runs npm install and npm test with approval, and pushes nothing. The same flow applies when a biometric system's admin console carries an outdated library.

Click through it at your own pace, or open it full screen.

The second demo sets up the local runner on your own machine, approves it with a device code and scans from the terminal. It suits a team that wants a component inventory of a self hosted biometric system before a DPIA, without sending source code anywhere.

Click through it at your own pace, or open it full screen.

The third demo connects Claude to the Safeguard MCP server with OAuth and chooses which tools Claude can use, so you can ask which projects carry a given vision library version.

Click through it at your own pace, or open it full screen.

To try it on your own projects, download Safeguard and follow the CLI documentation.

Quick check

Sources

• NIST: Face Technology Evaluations, FRTE and FATE
• NIST: FRTE 1:1 verification results and notes
• NIST IR 8280: FRVT Part 3, Demographic Effects (2019)
• NIST SP 800-63B
• Apple Support: About Face ID advanced technology
• Apple Platform Security: Biometric security
• Android: Measuring biometric unlock security
• OpenCV: face detection and recognition tutorial
• TSA: Biometrics technology
• CBP: Biometrics
• Illinois BIPA, 740 ILCS 14
• Illinois Public Act 103-0769
• Washington chapter 19.375 RCW
• FTC: biometric information policy statement announcement (18 May 2023)
• GDPR, Regulation (EU) 2016/679
• EU AI Act, Regulation (EU) 2024/1689
• European Commission: AI Act timeline
• EDPB Guidelines 05/2022 on facial recognition in law enforcement
• ICO: Biometric recognition guidance
• Google Photos Help: Face groups

Last reviewed: 25 October 2026. Biometric privacy laws, court rulings and NIST results change; check the linked statutes, regulations and reports for the current text before relying on them. This is not legal advice.