To trial threat intelligence in two weeks, measure whether it changes what your team does. Define three questions your analysts need answered, replay a past incident against the feed, and count the items that were relevant, timely and actionable. A big feed that changes nothing has failed the test.
What are you buying?
NIST defines cyber threat intelligence as cyber threat information that has been aggregated, transformed, analyzed, interpreted, or enriched to provide the necessary context for decision-making. The last words matter. If a feed gives you raw indicators without context for a decision, it is data, not intelligence, and your team will have to supply the context.
Before day one
Write three intelligence requirements, the questions that matter to your business. Examples: which actors target companies like ours, which vulnerabilities are being exploited in software we run, and which of our brands or domains are being impersonated. Anything the trial cannot help with these three is out of scope.
List your consumers. A SOC analyst wants indicators to match against logs. A vulnerability manager wants exploitation status. An executive wants a short brief. A trial that serves only one of them can still pass, but know which one it is.
Decide pass criteria now: for example, at least a stated share of items relevant to your sector and stack, delivery within a time you set, and at least two cases where the intelligence changed a priority or a detection.
Days 1 to 4: relevance sampling
Take one hundred items from the first days and mark each as relevant, partly relevant or noise. Relevance depends on you: your industry, your software, your regions. Check how items are described. Does the source say how it knows, and how confident it is? Reports that cannot distinguish observed fact from judgment are hard to use.
Check formats too. NIST's guide to threat information sharing lists indicators of compromise, tactics, techniques and procedures, suggested actions and incident findings as kinds of threat information. Structured formats such as STIX exist so that tools can exchange this material. Confirm the product exports in a format your tooling accepts.
Days 5 to 8: replay
Pick a past incident or near miss your team already understands. Ask: if this feed had existed then, what would it have told us, and when? Compare the timing of any relevant item to your actual detection date. This replay is the nearest thing to ground truth you will get in two weeks.
Run the same replay for one widely known vulnerability. Did the source report exploitation, and how long after public reports?
Days 9 to 12: wire one workflow
Connect a single use. Load a small set of indicators into detection and watch for matches, or feed exploitation status into your patch queue. Count what happens. How many matches, how many false, how many led to a ticket? Indicators age quickly, so ask whether expiry is handled.
Days 13 and 14: decide
Total the results against your criteria. Add analyst hours spent and the effort to integrate. Ask about licensing limits on sharing items with partners or customers, because some terms restrict that.
A short result sheet should say: relevant share, median delay on the replay, decisions changed, hours spent, and a clear yes or no.
Common trial mistakes
Judging by volume is the first. Trialing during a quiet fortnight is another, so replay history to compensate. A third is letting the vendor choose the sample. Pull your own.
Free and open sources deserve a place in the comparison. Public exploitation catalogs and open scoring data cost nothing, and a paid product should beat them on context for your environment, not just on count.
Where Safeguard fits
Threat intelligence feeds are not a Safeguard product, and we do not replace one. We focus on the application and software supply chain side. Exploitation context for dependencies can feed prioritization there, but your intelligence provider stays separate.
What to do
- Write three intelligence requirements and name the consumers.
- Set pass criteria before the trial begins.
- Sample and grade one hundred items for relevance.
- Replay a past incident and compare timing.
- Wire one workflow and count outcomes.
- Record hours, licensing limits and data formats.
- Report decisions changed, not items delivered.
Quick check
What is the difference between threat data and threat intelligence?
Data is raw observations such as an IP address. Intelligence adds analysis and context so someone can make a decision.
Why replay a past incident?
It gives you a known outcome to test against, so you can see whether the feed would have helped and how early.
Sources
- NIST glossary: cyber threat intelligence
- NIST SP 800-150, Guide to Cyber Threat Information Sharing
- Introduction to STIX
- CISA Known Exploited Vulnerabilities
Last reviewed: 18 October 2026.
