The AI Supply Chain Security Checklist: What to Verify Before You Trust an AI Tool

An AI tool is never just one tool. Behind the login page sit a model, often extra layers fine-tuned on top of it, plugins and connectors, a pile of open-source packages, a data store and a hosting provider. Each of those has its own owner, its own update schedule and its own ways to fail. When you approve the tool, you approve all of it.

 

Use this checklist before anything new goes live. Tick each item only when you have seen evidence, not when someone has said "yes." Any item you cannot tick, and any marked * that gets a "don't know," is a reason to pause.

 

Our view at ILLUME Intelligence is simple: trust in an AI tool should be earned item by item, and recorded.

 

 

1. Know the chain

A. * We have a written list of what the tool is built from: model, adapters, plugins, libraries, data sources and hosting.

B. We know who owns each part, and which parts the vendor sources from others.

C. The vendor can provide a software bill of materials, or an AI-specific equivalent such as an ML BOM.

D. We have recorded the version of the model and each component we are approving.

 

Why it matters: OWASP's guidance on LLM supply chain risk (LLM03:2025) recommends keeping an up-to-date inventory using an SBOM, and evaluating AI BOM and ML SBOM options, starting with OWASP CycloneDX. You cannot defend a chain you have not written down.

 

 

2. The model and its add-ons

A. * We know where the model came from and can show how its source was verified.

B. Downloaded model files come with hashes or signatures, and we checked them.

C. Any fine-tuned layers or adapters have a named, trusted origin.

D. The model is currently maintained, with no deprecation notice.

E. Model files are never loaded from sources nobody has vetted.

 

Why it matters: OWASP states that "currently there are no strong provenance assurances in published models," and warns that a malicious LoRA adapter can compromise the integrity of the base model it is merged with. In February 2024, BleepingComputer reported that JFrog's security team found about 100 models on Hugging Face with some form of malicious functionality. The payloads abused Python's pickle format to run code when a PyTorch model file was loaded, and one opened a reverse shell to an outside host.

 

 

3. Connectors, plugins and agent tools

A. * Every connector and plugin has a verified publisher, not just a plausible name.

B. We have confirmed that each is the official version from the vendor it claims to represent.

C. Each connector has only the permissions its job needs.

D. Updates to connectors are reviewed before they are accepted, not applied automatically.

E. We know which connectors can read email, files or customer records, and why.

 

Why it matters: In September 2025, Koi Security reported a package called postmark-mcp that copied an official email tool for AI agents. BleepingComputer reported that versions through 1.0.15 were clean, and that version 1.0.16 added a line forwarding users' emails to an outside address. It was available for about a week and was downloaded roughly 1,500 times. The tool had behaved normally for fifteen versions first, which is why reviewing each update matters.

 

 

4. Data and contract terms

A. * We know exactly which data the tool sees, and have limited it to what the task requires.

B. The vendor's terms say plainly whether our data is used for training.

C. We know where data is stored and processed, and who else touches it.

D. Retention and deletion terms are written down and acceptable.

E. Breach notification commitments are in the contract, with a timeframe.

F. If the tool handles personal data, we have checked our DPDP position: Section 8(1) keeps a Data Fiduciary responsible even when a processor handles the data.

 

Why it matters: OWASP lists unclear terms and privacy policies from model operators as a supply chain weakness, because sensitive data can end up in training and later be exposed.

 

 

5. Access and blast radius

A. The tool uses single sign-on or multi-factor authentication for every user.

B. Its service accounts and keys are scoped to the minimum and stored in a secrets manager.

C. We can revoke its access in one step, and know who would do it.

D. If it is compromised, it cannot reach our financial systems or full customer database.

 

 

6. Change control

A. We hear about model changes, version bumps and new sub-processors before they take effect, or at least promptly.

B. We have a patching policy for the components we run ourselves, as OWASP recommends.

C. A change to the tool's behaviour triggers a retest of what we care about.

 

7. Evidence before approval

A. * The tool was tested for abuse and data leakage before we approved it, by us or by an independent party.

B. We did not rely only on the vendor's benchmark results.

C. Findings from testing were fixed or formally accepted by a named owner.

 

Why it matters: OWASP advises applying comprehensive AI red teaming and evaluations when selecting a third-party model, and notes that benchmarks alone are insufficient, because models can be fine-tuned to pass them.

 

 

8. If it goes wrong

A. We have an exit plan: how to switch off the tool and recover our data.

B. Our incident procedure covers a vendor-side breach, including who we notify.

C. Logs of the tool's activity are kept long enough to investigate.

 

 

How to read your result

Count the unticked items. A few gaps in sections 5, 6 or 8 are normal for a first review and can be scheduled. A gap in any * item is different: those are the points where one weak link can reach your data directly. Fix those before go-live, or accept the risk in writing with a named owner.

 

Keep the completed checklist with the contract. When the vendor changes something, it is the first thing to reopen.

 

 

How ILLUME Intelligence can help

ILLUME Intelligence's AI Supply Chain Security service reviews the parts of an AI product you do not see: models, adapters, connectors, dependencies and vendor terms.

Our AI Red Teaming and AI Penetration Testing (AI VAPT) services supply the testing evidence in section 7, our DPDP compliance service covers section 4, and our vCISO service can run the approval process for you. A review of one tool you are about to adopt is a practical first step.

---

Comments

---

No Comments Found.