AI Red Team Report: What a Real Deliverable Looks Like

27/08/2026 Comments

Most security reports follow a familiar shape. Executive summary at the front, technical findings in the middle, remediation steps at the back. A CVSS score on each finding, a severity label, and a handover meeting where someone walks the team through what it all means.

An AI red team report looks different. Not because the discipline is less rigorous — if anything, it demands more — but because the attack surface is fundamentally different, and a report built for deterministic software doesn't translate cleanly to a system that generates a different output every time someone interacts with it.

Understanding what a real AI red team report actually contains, and why it's structured the way it is, matters — both for organizations commissioning one for the first time, and for those trying to evaluate whether what they've received is genuinely useful or just familiar-looking paperwork wearing a new label.

Section One: Scope and Test Configuration

Before a single finding appears, a credible AI red team report documents exactly what was tested, how, and under what conditions. This section isn't administrative preamble — it's the foundation on which every finding's validity rests.

The system under test needs to be described with precision: the model or models involved, the version tested, the system prompt in use, the tools and integrations the agent had access to, the retrieval pipeline if one exists, and any guardrails or filters operating between the model and the user. A finding against last month's deployment may not apply to this week's — in a world where organizations update prompts, switch models, and add integrations regularly, version specificity matters far more than it does in traditional application testing.

The scope section also defines what was deliberately left out of scope, and why. Testing a model in isolation produces different findings than testing the full stack — the application layer, the agent tools, the MCP-connected systems, the inference API — and a report that conflates the two is worse than one that's honest about its boundaries.

Section Two: Threat Model and Framework Mapping

This is the section that separates a serious engagement from a tooling run. Before findings appear, a credible report lays out the adversarial threat model: who might reasonably attack this system, what they're likely after, and what capabilities they'd need to get there.

Framework alignment lives here too. MITRE ATLAS — which as of its latest update covers 16 tactics, 84 techniques, 32 mitigations, and 42 documented case studies specifically for AI and ML systems — provides the taxonomy most mature engagements use to structure this section. Tactics represent what an adversary is trying to achieve (initial access, reconnaissance, exfiltration). Techniques describe the specific methods used to get there (prompt injection, model inversion, ML supply chain compromise). Mapping the engagement's attack surface to ATLAS gives the entire report a structure that's comparable across assessments, trackable over time, and legible to a security team that already speaks ATT&CK's language.

Depending on the organization and deployment context, OWASP's Top 10 for LLM Applications and the NIST AI Risk Management Framework typically appear here too — less as a checklist to tick off, and more as a cross-reference layer that places each finding in a recognized risk taxonomy. For organizations operating in European markets, EU AI Act alignment matters in this section too: full compliance requirements for high-risk AI systems landed on August 2, 2026, and penalties for non-compliance run up to €35 million or 7% of global annual turnover, whichever is higher. A well-structured AI red team report maps its findings to that regulatory framework directly, not as an afterthought.

Section Three: Findings

This is the core of the report — and how findings are documented says more about the quality of the engagement than almost anything else.

Each finding in a credible AI red team report contains several distinct components.

* A precise description of what was observed, not just what was attempted. There's a meaningful difference between "a prompt injection technique was tested" and "a crafted system-level instruction embedded in a retrieved document caused the agent to disclose its system prompt and attempt to execute an unauthorized file operation." The second version is a finding. The first is a test log.

* Reproducibility evidence. AI systems are probabilistic — the same prompt doesn't reliably produce the same output. A finding without reproducibility evidence (the exact input used, the temperature and sampling settings, the number of attempts required, and the rate at which the behavior appeared) is difficult to verify, difficult to fix, and difficult to confirm as resolved after remediation. Good reports include the verbatim attack payload, the model's verbatim response, and honest documentation of how consistently the behavior appeared.

* MITRE ATLAS technique mapping. Each finding should be traceable to one or more ATLAS techniques — connecting the observed behavior to a documented, recognized attack category rather than leaving it as a standalone observation with no external reference point.

* Severity classification. Unlike traditional CVSS scoring, AI findings don't always map cleanly to confidentiality, integrity, and availability impact in the standard way. Mature AI red team reports use a hybrid severity model that accounts for AI-specific factors: the reliability of the exploit (does it work 30% of the time or 90%?), the level of access or manipulation required, the scope of potential harm, and whether exploitation requires insider knowledge of the system prompt or succeeds against a standard user interaction.

* A clear distinction between findings that were verified as exploitable and findings that are theoretical. An unverified finding — something that looks like it might be exploitable but couldn't be confirmed during the engagement — belongs in the report, but separately, with honest language about what was and wasn't demonstrated. Conflating theoretical risk with confirmed exploitation produces remediation backlogs nobody trusts.

Section Four: Attack Chains

This section is frequently absent from lower-quality engagements, and its absence is one of the clearest signs that what's been delivered is a tooling run rather than a genuine adversarial assessment.

Individual findings matter. What matters more — and what a real attacker would actually pursue — is whether multiple lower-severity findings can be chained together into something significantly more damaging than any single one of them suggests.

A prompt injection finding that exposes the system prompt, combined with a misconfigured tool integration that allows the agent to make outbound HTTP requests, combined with insufficient logging, doesn't produce three medium-severity findings. It produces a path by which an external attacker can manipulate an internal agent into exfiltrating data to an external endpoint with no record of the interaction. The severity of a chain is determined by the destination, not the individual steps.

Documenting attack chains properly requires understanding the system well enough to reason about how its components interact under adversarial conditions — the kind of contextual judgment that automated tooling doesn't perform, and the reason human-led testing remains essential even as the tooling improves.

Section Five: Remediation Roadmap

A finding without a remediation path is a problem report, not a security deliverable. Every finding in a credible AI red team report comes with a specific, actionable recommendation — not "improve input validation" but "implement a deterministic content filter at the retrieval layer that rejects document chunks containing instruction-format text patterns before they reach the model context."

The remediation roadmap organizes all findings into a prioritized sequence: what to fix first, what to address in the near term, and what represents a longer-term architectural decision. Priority is determined by a combination of severity, exploitability, and remediation cost — a critical finding that can be resolved in a single afternoon should sit higher on the immediate list than a medium finding requiring a six-month architecture change, even if both matter.

Remediation guidance for AI systems also needs to account for AI-specific complexity that doesn't appear in traditional reports. Some fixes require retraining or fine-tuning. Some require changes to the prompt or system configuration that may affect model performance elsewhere. Some require tool integration changes that touch systems outside the security team's control. A report that acknowledges these dependencies — and helps the team sequence remediation around them — is genuinely useful. One that issues generic recommendations without understanding the system being fixed is not.

Section Six: Retest Plan

The final section of a credible AI red team report documents how findings will be verified as resolved — which is a harder problem for AI systems than for traditional applications.

Because AI outputs are probabilistic, confirming that a finding has been remediated requires retesting under the same conditions as the original engagement, with documented evidence that the behavior no longer appears at a meaningful rate. A retest plan defines which findings require formal verification, what the pass/fail criteria look like, and what retesting protocol will be used.

What This Should Tell You About Any Report You've Already Received

Most of the components above aren't exotic requirements. They're the baseline for a deliverable that actually helps an organization understand its exposure and close it. If a report you've received treats AI findings the same way a conventional vulnerability scanner report would, skips the attack chain analysis, provides generic remediation guidance, or maps nothing to a recognized AI threat framework, it's worth asking what exactly the engagement was designed to surface.

At ILLUME Intelligence, every AI red team engagement produces a report built around these sections — scoped precisely, mapped to MITRE ATLAS and relevant compliance frameworks, findings documented with reproducibility evidence and verified exploitability, attack chains identified and prioritized, and remediation guidance specific enough to act on. If you'd like to see what that looks like in practice, or compare it against something you've already received, reach out to Illume to start the conversation.