What Boards Should Ask Before Approving an AI Red Team Budget

Nobody on a board needs to understand prompt injection to approve this budget well.

They need to ask a handful of good questions and know what a good answer sounds like.

 

That matters because the request is arriving at more boards now, and it can be hard to judge. It sits between security and AI, two topics where most directors, quite reasonably, defer to management. BDO's 2026 survey of more than 200 U.S. directors, fielded in June, found that only 36% were very confident their board had the right skills to oversee AI, cybersecurity and data governance; 49% were somewhat confident and 15% were not. That is a U.S. sample, not an Indian one, but the discomfort will be familiar. So here are the questions, with notes on the answers worth listening for.

 

 

"What is the worst case, in plain terms?"

Ask management to describe the single worst thing an attacker could make this AI do, without technical vocabulary.

It could show one customer another customer's records.

It could approve a refund it should not.

It could reveal our internal instructions and credentials.

 

If the answer is vague, the budget request probably is too. A good answer names a specific system, a specific harm, and who would be affected.

 

Some context for scale. IBM's 2026 India findings put the average breach at INR 25.5 crore, and reported that 26% of malicious breaches in India were AI-generated. Globally, as summarised by Kiteworks and by Baker Donelson, IBM found that incidents involving an organisation's own AI models or applications rose from 13% to 21% of breaches, and that among organisations that had such an incident, 92% lacked proper AI access controls at the time. Treat the global figures as secondary-sourced; the India figures come from IBM's own release.

 

 

Do we have a list of our AI systems?

You cannot test what you have not counted. A credible request starts from an inventory: what AI is in use, what data it touches, what it can do, who owns it.

 

Listen for shadow AI too, meaning tools staff adopted without approval. IBM India reports that shadow AI adds an average of INR 1.79 crore to a breach where it is present. If management cannot say how many AI tools are in use, an inventory may need to come before the red team does.

 

 

How is this different from the penetration test we already buy?

Fair question, and a good one to press on. Microsoft's AI Red Team, says it has tested more than 100 generative AI products, makes two points that help here. AI systems amplify existing security risks and introduce new ones, so old weaknesses still matter. And automation helps, but people stay at the centre, because judgement, subject knowledge and context cannot be fully automated.

 

A good answer says that traditional testing still has a job, that AI adds new routes in (such as steering a model through content it reads), and that the engagement will cover the system around the model, not just the model.

 

 

What is in scope, and what is not?

The request should say which system is being tested, which attackers are being simulated, and what is off limits. If it says "our AI," send it back. A narrow, well-chosen first scope beats a broad vague one, and we cover how to write one in our guide to scoping a first AI red team engagement.

 

 

How will we know it worked?

Be wary of any plan whose output is a report and nothing else. Ask for four things:

* findings that your own team can reproduce

* severity ratings that tie back to the worst cases from question one

* a named owner for every fix

* a retest after the fixes, in the same budget

 

Microsoft describes repeated cycles of testing, measurement and mitigation as part of keeping AI systems safe. A single test with no retest tells you what was wrong on one day.

 

 

What are regulators and auditors expecting?

For financial-sector boards, RBI's FREE-AI report says each regulated entity should adopt an AI governance policy approved by its board, and recommends structured red teaming across the AI lifecycle, proportionate to risk. These are the committee's recommendations, and we could not confirm from the report whether they are binding, so ask your compliance head for the current position.

 

Beyond finance, Section 8(5) of the DPDP Act requires reasonable security safeguards for personal data, with penalties of up to INR 250 crore, and the main duties are due from about mid-May 2027. If your AI touches personal data, documented adversarial testing is one way to show reasonable steps. That is our reading, so take legal advice.

 

 

Who decides what gets fixed?

Findings do not fix themselves.

The board should expect one named executive who accepts or rejects each risk, and a date by which the board will hear what changed.

This is the question that separates a purchase from a programme.

 

 

What a sound request looks like

You do not need to judge the price against a benchmark, and we have not found a reliable published one. Judge the shape instead. A sound request names one system and one worst case, comes with an AI inventory, explains what the test adds over existing testing, includes a retest, names an owner, and says how often the testing will repeat. A request missing three or more of those is not ready.

 

 

How ILLUME Intelligence can help

ILLUME Intelligence can sit with management before the board pack goes out, helping turn "we need AI red teaming" into a scoped, board-ready request. Our AI Red Teaming service then carries out the testing, with AI Penetration Testing (AI VAPT) and LLM Security Testing covering the surrounding application layers, and our vCISO service can own the follow-through and the board reporting.

 

 

One last question

If the board approves this and nothing changes six months later, what would that say about the decision?

Ask for the answer to that in writing, before you sign.

---

Comments

---

No Comments Found.