How to run a two week CTEM pilot and judge it on evidence

To trial a CTEM approach in two weeks, pick one narrow scope, define measures before you start, and test the full loop: discover, prioritize, validate and fix. Judge it on whether it changed what you fix first and how fast you fixed it, not on how many findings it lists.

What is a CTEM trial really testing?

CTEM, continuous threat exposure management, is a cycle rather than a tool. A trial therefore tests whether a tool or service helps you run the cycle on a real slice of your environment. NIST SP 800-137 describes continuous monitoring as giving visibility into assets, awareness of threats and vulnerabilities, and insight into whether controls work. Use those three as your lens.

Before day one

Choose one scope: for example, one product's internet facing services plus its repositories. Narrow beats broad, because you must verify results by hand.

Write measures and thresholds:

1. Coverage: how many of your known assets in scope appear?
2. Unknowns: how many real, previously unknown items appear?
3. Noise: how many of the first fifty findings are false or duplicates?
4. Reordering: how many of your current top ten items drop, and how many new ones enter, with a reason you accept?
5. Time: how long from finding to ticket with an owner?
6. Fix: how many findings get fixed within the trial?

Capture a baseline: your current top ten list with your own reasons. This is the control you compare against.

Set the decision owner and date.

Days 1 to 3: Connect and scope

Connect sources: cloud accounts, repositories, scanners, ticketing. Record the effort. Confirm that the tool probes only what you own.

Days 4 to 6: Discovery and coverage

Compare findings to your known list. Check unknown items by hand. CISA's BOD 23-01 gives a federal benchmark of automated discovery every 7 days and vulnerability enumeration started every 14 days, so note how frequently the tool refreshes and whether changes you make appear.

Days 7 to 9: Prioritization

Look at how the tool ranks. Does it use exposure, known exploitation and asset value, or just severity scores? Check against public references. CISA's Known Exploited Vulnerabilities catalog lists flaws seen exploited in the wild. FIRST's EPSS estimates exploitation probability over 30 days. Ask whether the tool uses signals like these and shows its reasoning.

Compare the new top ten with your baseline. For each change, decide whether you agree. A good outcome is not agreement with you every time. It is a clear reason you can check.

Days 10 to 11: Validation

This is the step that separates exposure management from a long scan list. Pick three high ranked findings and test whether they are real and reachable. For a dependency, is the vulnerable function called? For a host, does a path from the internet exist? Record how the tool supports this and how much you did by hand.

Days 12 to 13: Mobilization

Send five findings to owners through your ticket system. Measure how many owners could act without asking you questions. Track at least one fix to completion and note the time.

Day 14: Decide

Score each measure against the threshold, add evidence, and decide: adopt, extend the trial, or stop. Record costs learned using your own counts and written quotes. Capture lessons even if you stop, since your baseline and measures are reusable.

What will a two week trial not show?

Long term noise levels, support quality, and how the product copes as you add scope. Add reference calls and a staged rollout.

Where Safeguard fits

Safeguard provides reachability analysis for validating dependency findings, policies and gates at build, CI and release, and autonomous remediation through fix pull requests, off by default with categories starting on Manual. It is not a CTEM platform, and CTEM tooling is outside what we offer today.

What if the pilot results are mixed?

Mixed results are common. A tool might find unknown assets well and rank poorly, or rank well and integrate badly. Write both lists down and decide which weakness you can live with or work around. Ask the vendor for a plan with dates for the gaps you care about, and keep the measures so a second tool can be tested on the same scope. A pilot that ends in a clear "no, and here is what we need" is still a successful pilot.

What to do

1. Pick one narrow scope and write six measures with thresholds.
2. Record a baseline top ten with your reasons.
3. Test coverage, noise and refresh frequency.
4. Compare prioritization with the baseline and public references.
5. Validate three findings by hand and track one fix.
6. Score, decide and keep the measures for next time.

Quick check

Sources

1. NIST SP 800-137, Information Security Continuous Monitoring
2. CISA BOD 23-01
3. CISA Known Exploited Vulnerabilities Catalog
4. FIRST, Exploit Prediction Scoring System

Last reviewed: 18 October 2026.