How Do You Know Your SIEM, EDR, and Firewall Are Configured to Catch a Real Attack?

Most security teams can answer "what tools do we have deployed" in under a minute. The harder question — "what do those tools actually detect under realistic attack conditions" — takes significantly longer, and for most organizations, the honest answer is "we're not entirely sure."

 

That uncertainty isn't a criticism. Managing a modern security stack is genuinely complex work, and keeping track of what every component detects across a continuously changing environment isn't something any team can do from memory or from a configuration file. What it does mean is that the gap between "this tool is deployed" and "this tool is detecting what it should" is real, present in almost every environment, and only visible if you actively look for it.

 

Here we will go layer by layer through the main security systems organizations deploy — SIEM, EDR, firewall, and identity controls — describing specifically what effective testing looks like for each one and what findings typically emerge when that testing is done properly.

 

 

SIEM: Testing the Detection Logic, Not the Dashboard

A SIEM dashboard shows log ingestion rates, rule counts, and alert volumes. None of these metrics tell you whether the right attack behaviors are actually producing alerts. Testing a SIEM properly requires going a level deeper — into the specific rules, the log sources feeding them, and the conditions under which they fire.

 

What actually gets tested:

Log source coverage is the foundation. Before a rule can fire, the relevant log data needs to be reaching the SIEM in the first place. Testing this means verifying — not assuming — that every log source a detection rule depends on is actually feeding data, in the right format, at the right volume, without compression or coalescing that drops events before correlation runs. The most common finding here is a log source that appeared healthy on the collection side but was silently dropping events due to volume throttling at the forwarder — a configuration setting nobody changed deliberately, but that changed the SIEM's effective coverage significantly.

 

Rule coverage mapping tests which MITRE ATT&CK techniques have active, functioning detection coverage in the SIEM. The goal isn't a percentage — it's understanding which techniques in the most likely attack paths for this specific environment are and aren't covered, and whether the rules covering them are actually firing against real technique execution rather than just syntactically valid in the rule editor.

 

Correlation logic validation tests whether multi-event rules — the ones that detect attacker behavior by correlating several events over time, rather than firing on a single log entry — are actually working end-to-end. These rules are the most likely to break silently when event formats change, when time windows need adjustment, or when the events they correlate arrive out of sequence. They're also the rules that detect the most sophisticated attack behavior. A SIEM with healthy single-event rules and broken correlation rules has a significant detection gap in exactly the techniques that matter most.

 

Alert fatigue analysis maps the relationship between rule sensitivity and operational trust. Rules that generate excessive false positives are frequently muted, deprioritized, or deleted — which closes the false positive problem and opens a detection gap that nobody formally documented. Testing this means not just running simulations, but auditing the history of which rules have been modified or suppressed in response to noise, and whether those modifications created coverage gaps.

 

Common findings: Rules that fire in test environments but not in production due to differences in log format. Detection coverage for initial access and execution techniques that's significantly stronger than coverage for lateral movement and persistence — because most SIEM tuning focuses on known external threats rather than the techniques attackers use after they're already inside.

 

 

EDR: Testing Policies, Not Just Presence

An EDR agent on every endpoint is a deployment metric, not a detection metric. What matters is whether the policy applied to those agents is actually configured to detect the techniques an attacker would use — and whether exclusions, privilege limitations, or coverage gaps have created paths through the endpoint that the EDR won't see.

 

What actually gets tested:

Exclusion mapping is consistently the highest-yield part of EDR testing. Every exclusion in an EDR policy is a path an attacker can potentially use. Testing exclusions means actively executing known-bad behavior within excluded paths or processes — not to confirm the exclusion is working (it is), but to confirm whether that exclusion creates an exploitable gap. Temporary file directories, backup application processes, and remote management tool paths are the most commonly exploited exclusion categories, because they're also the most commonly excluded for legitimate noise reasons.

 

Privilege verification checks whether EDR agents are running with the system-level privileges they need to intercept kernel-level behaviors. An EDR agent that deployed without full system privileges will miss memory injection, credential dumping, and rootkit behaviors that operate at the kernel level — exactly the behaviors that matter most in a sophisticated attack. This isn't detectable from the console health dashboard; it requires verifying agent privileges on each endpoint class, particularly newly provisioned systems and cloud-hosted instances where deployment automation sometimes cuts corners.

 

Behavioral detection testing runs specific MITRE ATT&CK techniques — credential dumping via LSASS, process injection, living-off-the-land binary abuse, persistence mechanism installation — against live endpoints and measures whether the EDR generates the expected detection. The distinction between this and a penetration test is that the goal isn't to find an exploitable vulnerability; it's to confirm whether the detection that should fire when a technique is executed actually fires.

 

Coverage gap analysis maps endpoint types against EDR deployment — specifically identifying legacy systems, IoT devices, specialized industrial systems, and contractor-managed endpoints that either can't run the EDR agent or are excluded from policy for operational reasons. These coverage gaps represent paths an attacker can move through without the EDR seeing them.

 

Common findings: Exclusions added for legitimate tools (remote desktop software, backup agents, developer utilities) that create exploitable paths. EDR agents on cloud instances running without kernel access due to container privilege restrictions. Behavioral detection configured for one operating system version and not updated when a major OS upgrade rolled out.

 

 

Firewall: Testing What's Actually Blocked, Not What Should Be

Firewall rules accumulate over years. Rules get added for specific projects, temporary exceptions get made permanent, and the relationship between the rule base and the current network architecture gradually diverges. Testing a firewall means testing the actual rule base against the actual traffic patterns of an attacker — not reviewing the documentation of what the rules were intended to do.

 

What actually gets tested:

Egress filtering validation tests whether outbound connections that shouldn't be permitted are actually blocked. Command-and-control traffic over common ports (80, 443, 53), data exfiltration to external cloud storage services, and protocol tunneling through permitted traffic types (DNS tunneling, HTTPS beaconing) are all technique categories where firewall rules are supposed to provide a layer of containment. Testing this means attempting these connections from inside the network and confirming the outcome — permitted or blocked — against what the policy says it should be.

 

Lateral movement containment testing validates whether network segmentation rules are actually preventing east-west movement between network segments. Network segmentation is frequently described in architecture diagrams but imperfectly implemented in rule bases — VLANs that should be isolated share firewall rules that permit more communication than the architecture intends. Testing lateral movement attempts between segments surfaces the gaps between the architecture diagram and the actual rule behavior.

 

Rule base audit identifies rules that should no longer exist — temporary exceptions that became permanent, rules for systems that were decommissioned, and overly permissive any-to-any rules that predate the current network architecture. These aren't configuration errors in the traditional sense; they're the natural entropy of a rule base that's been maintained by multiple people over multiple years without a formal rationalization process.

 

Firewall logging verification confirms that blocked and permitted connections are actually being logged at the level of granularity the SIEM needs to run detection rules. Firewalls configured to log at a summary level rather than per-connection level produce logs that look healthy but don't provide the data needed for threat detection correlation.

 

Common findings: Egress filtering rules that block known-malicious IP ranges but permit outbound HTTPS to any destination, leaving C2-over-HTTPS techniques undetected. Lateral movement between network segments that should be isolated because a firewall rule for a discontinued system still permits cross-segment access on management ports.

 

 

Identity Controls: The Layer Most Testing Misses

Identity is where most modern attacks operate — stolen credentials, privilege escalation, service account abuse — and it's the layer that security control testing most consistently underweights. Testing identity controls means validating that authentication policies, privilege boundaries, and activity monitoring work the way they're described.

 

What actually gets tested:

Privilege escalation path testing maps whether a standard user account can reach elevated privileges through legitimate system features — misconfigured service accounts with excessive permissions, Kerberoastable service principal names, or misconfigured delegation settings that permit privilege escalation through legitimate Kerberos operations. These paths don't require exploiting a vulnerability; they require using Active Directory features the way an attacker would use them.

 

MFA coverage verification confirms that multi-factor authentication is actually enforced for the systems and accounts it's supposed to protect. MFA policies that appear global frequently have legacy application exceptions, service account exclusions, or network location bypasses that create paths where MFA isn't required — and that an attacker who has obtained credentials can use to authenticate without triggering MFA.

 

Identity monitoring gap analysis tests whether identity-based attack behaviors — password spraying, credential stuffing, Kerberoasting, pass-the-hash — actually produce alerts in the monitoring stack. The Active Directory event logs that should feed these detections are sometimes missing from SIEM coverage entirely, or are configured to only log at a verbosity level that misses the specific events identity attacks generate.

 

Common findings: Service accounts with domain admin privileges that haven't been used in months — meaning they're unmonitored and their credentials could be compromised without anyone noticing. Authentication events from legacy protocols (NTLM, basic auth) that bypass modern identity controls because the legacy protocol exclusion was added years ago and never revisited.

 

 

Putting the Picture Together

The value of testing each layer individually is in understanding where the coverage gaps are. The value of testing them together is in understanding whether an attacker could move from initial access through the environment — using each gap in sequence — without the stack seeing the chain of behavior as a connected attack.

 

That's the assessment that moves security control testing from a compliance exercise into a genuine security capability: understanding not just that individual controls have gaps, but whether those gaps combine into paths an attacker could actually use, and whether the integrated detection across the stack would surface the behavior before it became consequential.

 

The tools are there. The question is what they actually see.

---

Comments

---

No Comments Found.