NIST AI RMF: how do you keep monitoring after the assessment?

After an AI risk review, monitoring means watching the system in production, recording incidents and errors, reviewing risks on a schedule and retesting when the model, data or use changes. NIST's AI Risk Management Framework names these duties in MEASURE 2.4 and MANAGE 4.1 and MANAGE 4.3.

What does the framework actually say?

The AI RMF Core has four functions: Govern, Map, Measure and Manage. The subcategories that matter after launch are short and specific.

MEASURE 2.4 says the functionality and behavior of the AI system and its components are monitored when in production. MEASURE 1.2 asks that the appropriateness of metrics and the effectiveness of existing controls are regularly assessed and updated. MANAGE 4.1 calls for post-deployment monitoring plans, including ways to capture input from users, appeal and override, decommissioning, incident response, recovery and change management. MANAGE 4.3 covers tracking, responding to and recovering from incidents and errors.

There is no certificate attached to the framework. It is voluntary guidance, so what counts as enough is your own risk tolerance, written down.

What should you actually watch?

Start from the risks you recorded in the Map function. Monitoring a risk you never named is guessing.

For each recorded risk, pick one signal you can read on a schedule. A few examples of the kind of signal that works:

• Output quality: a sampled review of answers against a rubric, since MEASURE 2.3 asks that performance be measured for conditions like the real deployment.
• Drift: whether inputs in production still resemble the test data.
• Safety and security: blocked or flagged requests, tool calls the system made, and any access outside expected scope. MEASURE 2.7 asks that security and resilience be evaluated and documented.
• Privacy: whether outputs expose personal data, tied to MEASURE 2.10.
• User reports: complaints, overrides and appeals, which MANAGE 4.1 expects you to capture.

Write a threshold for each signal and the person who acts when it is crossed.

How do you handle third parties and pre-trained models?

MANAGE 3.1 asks that risks from third-party resources are regularly monitored, and MANAGE 3.2 says pre-trained models used for development are monitored as part of regular maintenance. If a vendor changes the model behind your feature, your assessment is out of date the same day. Ask for change notices and re-run your key tests when you receive one.

When should you stop the system?

MANAGE 2.4 says mechanisms must exist, with assigned responsibilities, to supersede, disengage or deactivate AI systems that show performance or outcomes inconsistent with intended use. Decide before launch who can switch the feature off and how fast.

What to do

1. List the risks from your Map step and choose one signal for each.
2. Set thresholds and name an owner for every signal.
3. Log incidents and errors in one place, and set rules for who is told, per MANAGE 4.3.
4. Add a retest trigger: model change, data change, new tool access or new user group.
5. Review metrics and controls on a fixed schedule and record the result.
6. Document who may disable the system, and test that switch.
7. Keep dated records of all of the above. They are your evidence if someone asks later.

Quick check

Sources

• NIST AI RMF Core (Measure and Manage functions), fetched 18 October 2026
• NIST AI Risk Management Framework overview, fetched 18 October 2026

Last reviewed: 18 October 2026.