AI security · Supply chain & model audit
You did not train the model. Know what you inherited.
Provenance, integrity and licensing review of every model, dataset, package, prompt and third-party service your AI product depends on, ending in a maintained AI bill of materials and a risk register with owners.
Why audit the supply chain
Most of an AI product is somebody else’s work
A foundation model from a provider, weights from a public hub, a dataset of uncertain licence, forty Python packages in the orchestration layer, a vector database, three tool integrations and a system prompt last edited by someone who has left. Each is a place where integrity, licensing or data handling can fail, and most teams cannot list them all.
Questions the audit answers
- Which models and datasets are in the product, and under what licence?
- Could a model file execute code when loaded, and has it been verified against its published hash?
- What customer data reaches which provider, and can they train on it?
- Who can change a system prompt or a tool definition, and who reviews it?
- If a provider changes or withdraws a model, what breaks and what is the fallback?
Scope
Six layers, one inventory
Models and weights
Where each model came from, who trained it, under which licence, and whether the file you load is the file that was published. Unsafe serialisation formats that execute code on load are flagged and replaced.
Training and fine-tuning data
Provenance, consent basis and licence of every dataset, including the ones a vendor used on your behalf. Poisoning exposure is assessed from how data was collected and who could have written to it.
Packages and inference stack
Vulnerability and integrity review of the Python and system dependencies, model servers, vector databases, orchestration frameworks and agent toolkits, with pinning and signing recommendations.
Third-party AI services
What you send to each provider, what they retain and train on, where it is processed, how keys are managed, and what a model update or an outage does to your product.
Prompts, tools and connectors
System prompts, tool definitions and MCP servers treated as supply chain: who can change them, how changes are reviewed, and whether a third-party tool description can steer your agent.
AI bill of materials
A maintained inventory of every model, dataset, package, prompt and service in the system, with versions, licences and owners, in a format your procurement and compliance teams can consume.
What you receive
An inventory you keep, and a risk register with owners
The audit ends in documents that your engineering, procurement, legal and compliance teams each use for their own purpose, built once from the same evidence.
AI bill of materials with licences, versions, provenance and owners
Ranked risk register per component with remediation and owner
Licence and consent conflicts called out plainly, including what a model trained on restricted data means for your product
Integrity controls: hashes, signatures, safe formats, pinned versions and a review gate for prompt and tool changes
Vendor assessment template and completed assessments for in-scope providers
Mapping to EU AI Act documentation duties, NIST AI RMF and ISO/IEC 42001
Provenance we practise
We hold ourselves to the same standard on the data we supply
Softuvo supplies de-identified medical imaging datasets for diagnostic AI. Every delivery carries its source-agreement reference, permitted-use scope, hashed manifest and per-study acceptance verdict. That is the level of provenance an audit looks for, and it is why we know what to ask a vendor for.
How it runs
Inventory first, then evidence, then controls
Scope
We map the system under test: models, prompts, tools, retrieval sources, data flows, users and the business impact of each failure. Rules of engagement and a test plan are agreed in writing.
Assess
Threat modelling, adversarial testing or audit against the agreed plan, with every finding reproduced and recorded with evidence, severity and the affected component.
Report and fix
A report your engineers can act on and your leadership can read: ranked findings, root causes, concrete remediation, and a control map against the frameworks you answer to.
Re-test and embed
Fixed findings are re-tested. Regression test suites, guardrail evaluations and monitoring recommendations are handed over so the next release does not reopen them.
FAQ
Frequently asked questions
Why is a model file a security risk?
Several common model formats serialise arbitrary code alongside the weights and execute it when the model is loaded. A model downloaded from a public hub can run code on your inference server before it has answered a single prompt. The audit identifies those formats and migrates you to safe ones with integrity checks.
We only use hosted models. Is there still a supply chain?
Yes, and it is mostly contractual and operational: what the provider retains, whether your data can be used for training, where processing happens, how a model version change is announced, and what your fallback is. Plus every open-source package in your orchestration layer, which is where most of the vulnerability findings sit.
Can you tell us whether our vendor’s model was trained on data it was allowed to use?
We can tell you what the vendor can and cannot evidence, which is what matters for your exposure. We review the provenance documentation, consent basis and licence terms they provide, identify the gaps, and tell you what contractual protection to ask for where evidence is missing.
How does this relate to the EU AI Act?
Providers and deployers of high-risk systems carry documentation duties on training data, and general-purpose model providers must publish training-content summaries and respect copyright reservations. The AI bill of materials and provenance register from this audit are the working documents those duties are built on.
How long does an audit take?
A product with a handful of models and providers is usually three to four weeks. Platforms with many fine-tuned models and datasets are phased. The bill of materials is handed over as a living inventory so the next audit is an update rather than a rediscovery.
Send us the list of models and vendors you think you use.
We will come back with the list you actually use, what each one can evidence, and the three things to fix first.
Or email [email protected] · Mohali, India