Compare SIEM tools by testing how they collect logs, normalize them, correlate across sources and retain data, using your own log sources. Dashboards and rule libraries look alike in demos. The differences show up in coverage of your systems, data volume handling, timestamp accuracy and the work needed to keep rules current.
What is a SIEM?
NIST's log management guide (SP 800-92) describes security information and event management software as centralized logging software that has one or more log servers performing analysis and one or more database servers storing the logs. It adds that SEM and SIM are older names for SIEM-like products. The guide is from 2006, so treat its product details as background, but its method for judging log tooling still applies.
1. How does it collect logs?
NIST describes two approaches. In agentless collection, the SIEM server pulls logs from hosts or hosts push to it, with the server authenticating to each host. In agent-based collection, a program installed on the host filters, aggregates and normalizes logs before sending them on.
NIST names the tradeoffs. Agentless saves you from installing and maintaining agents everywhere, but it lacks host-level filtering and aggregation, which can mean much more network traffic and longer analysis time, and the server may need credentials for each host. Sometimes only one method works, because a given host may have no remote collection option.
Test: list your twenty most important log sources and ask the vendor to show the collection method for each, in your environment.
2. Does it understand your log formats?
NIST says SIEM products usually support several dozen log source types, and for each non-generic type they know how to categorize the important fields, for example which field holds the source address. That improves normalization and correlation over software with less granular understanding. For generic formats such as syslog, it does not.
Test: feed an application log of your own, one that is not on the vendor's supported list. Measure how long it takes to extract the user, source address and result into normalized fields.
3. Are timestamps trustworthy?
SP 800-92 warns that each host stamps entries from its own clock, and an inaccurate clock makes multi-host analysis difficult. Its example: timestamps showing event A 45 seconds before event B, when A really happened two minutes after B. It recommends synchronizing clocks to a common time source.
Test: ask how the tool handles skewed or missing time zones, and whether it keeps both event time and ingest time.
4. What are the retention and protection options?
NIST notes that organizations might need to adjust storage to meet data retention requirements, and that log data availability can be lost without it. Ask about hot and archive tiers, cost per gigabyte-day and how long a restore from archive takes. Also ask how logs are protected from alteration, because attackers delete logs.
5. How does correlation get built and maintained?
A rule that never fires or always fires is worse than none. Ask who writes rules, how they are tested and what happens when a log format changes. NIST's SI-4 control adds a point commonly forgotten: adjust monitoring levels when risk changes, and obtain legal opinion on monitoring activities.
6. What is the cost model at your volume?
Collect a week of real log volume, in events per second and gigabytes per day, and have each candidate price that, including growth. NIST says prioritize log management according to the perceived reduction of risk, which is an argument for not ingesting everything.
7. What is the operating burden?
Count the steps to onboard a source, tune a noisy rule and answer an auditor's question about a past event. Staff time often outweighs license cost.
A constructed example
A 200-person company evaluates two products over two weeks with the same 15 sources. One ingests firewall and directory logs in an hour but needs a custom parser for its internal application, which takes two days. The other parses the application log in minutes by learning the format, but its archive restore takes most of a day. The company has a seven-year retention rule for finance systems, so restore time decides the choice.
How to check or apply this
- Write your log sources, volumes and retention periods before talking to vendors.
- Run a proof of concept against live data, not sample data.
- Score each criterion above on evidence, and record it.
- Check clock sync across sources before you blame the tool for strange timelines.
- Test incident questions: "who logged in from this address between these times?"
- Ask for an exit plan: how you export your data and rules.
Safeguard is not a SIEM and does not collect logs. Its relevance is upstream, finding vulnerable dependencies so fewer alerts start from known weaknesses.
Quick check
Agent-based or agentless log collection?
Agent-based gives filtering and normalization at the host but means installing and maintaining software. Agentless is simpler to deploy but can move more data and needs credentials to each host, according to NIST SP 800-92.
Why does clock accuracy matter in a SIEM?
Correlation depends on event order. If host clocks differ, an attack sequence can appear in the wrong order or not match at all.
Sources
- NIST SP 800-92, Guide to Computer Security Log Management
- NIST SP 800-53 Rev. 5, SI-4 System Monitoring (control text)
Last reviewed: 18 October 2026.
