Free AI Vendor Assessment Checklist: 25 Questions to Ask
Here's a free AI vendor assessment checklist you can copy straight into a document, plus the exact questions to ask, the evidence to request, and how it maps to the EU AI Act, GDPR, NIST and ISO. When you buy or embed a third-party AI tool, you inherit its risk. If it hallucinates, leaks data, or breaches a rule, the buck often stops with you, not the vendor. Under the EU AI Act you can be the deployer of that system, with your own legal duties. Under GDPR you're usually the controller, so the vendor's data handling is your accountability. This checklist is built to drop into a due-diligence pack, a supplier questionnaire, or a procurement gate. Work through the questions, demand the evidence, and score each area before you sign.
What is an AI vendor assessment?
An AI vendor assessment is the due-diligence you run on any third-party AI product, model or feature before you buy it, embed it, or ship it to your users. It's the same idea as a classic supplier security review, pointed at the risks that only show up once AI is in the stack: what model sits underneath, what it was trained on, how it fails, who's liable when it does, and what happens to your data on the way through.
It does three jobs at once. It forces the vendor to show their working instead of waving a marketing deck. It gives you evidence to keep on file for your own regulator, board or enterprise buyer. And it tells you, before money changes hands, whether the tool is safe to put in front of your customers or your own people.
One thing to be clear about. This isn't a one-off gate you clear at purchase and forget. Vendors swap their underlying model, change terms, and ship new features. The assessment is a control you re-run, not a form you file.
Why does third-party AI need due diligence?
Because you don't get to outsource the risk. Three duties land on the buyer, not the seller.
You inherit the vendor's risk. If the tool produces a biased decision, states false information as fact, or leaks a record, the harm reaches your customers under your name. "The vendor's model did it" is not a defence a regulator or a court accepts.
You may be the deployer under the EU AI Act. The Act splits obligations between the provider (who builds the system) and the deployer (who uses it under their own authority). If you deploy a high-risk AI system, Article 26 puts real duties on you: use it per the provider's instructions, assign competent human oversight, monitor it, keep the logs, and tell the provider and authorities if something goes wrong (EU AI Act Article 26). Those duties are yours even though someone else built the model.
You carry GDPR processor obligations. When a third-party AI tool processes personal data on your behalf, you're the controller and the vendor is your processor. GDPR Article 28 says you can only use a processor that gives "sufficient guarantees," and you must have a contract (a DPA) that binds them to documented instructions, confidentiality, security, sub-processor control, and deletion or return of data (GDPR Article 28). No DPA, no lawful basis to hand them the data.
The free AI vendor assessment checklist
Copy the table below. Work through each area with the vendor, record their answer, and demand the evidence in the third column. Don't accept a verbal "yes." If they can't produce the document, treat the answer as "no." Score each area red, amber or green once you've seen the evidence, using the method in the next section.
| # | Question to ask the vendor | Why it matters | Evidence to request |
|---|---|---|---|
| Model and data provenance | |||
| 1 | Which foundation model powers this, and is it yours, open-weight, or a third-party API (OpenAI, Anthropic, Google, etc.)? | You need to know how many parties touch your data and whose model behaviour you're depending on | Named model and version, architecture diagram showing every model call |
| 2 | What data was the model trained or fine-tuned on, and do you have rights to all of it? | Training-data disputes and IP claims flow downstream to you as the user | Training-data provenance statement, IP and copyright indemnity in the contract |
| 3 | Is our data used to train or improve your models? | Your confidential inputs becoming training data is a leak and often a GDPR breach | Written no-training term, enterprise tier confirmation |
| 4 | Does generated output risk reproducing copyrighted or third-party material? | You publish or act on the output, so you carry the infringement risk | Output-screening description, indemnity terms |
| Security | |||
| 5 | How do you defend against the OWASP LLM Top 10, prompt injection and system-prompt leakage especially? | Prompt injection is the top LLM risk; a hijacked agent acts with your permissions | Security testing results mapped to the OWASP LLM Top 10 |
| 6 | How do you prevent data exfiltration through model responses? | An over-permissioned assistant can leak confidential records in plain text | Access-scoping design, output filtering, red-team report |
| 7 | Do you hold SOC 2 Type II or ISO/IEC 27001, and can we see the current report? | Independent audit is the baseline proof of a real security programme | SOC 2 Type II report or ISO 27001 certificate, in date |
| 8 | How is our data encrypted in transit and at rest, and who can access it internally? | Weak encryption or broad internal access turns your data into their liability and yours | Encryption standards, internal access-control policy |
| Data protection | |||
| 9 | Are you our processor, and will you sign a GDPR Article 28 DPA? | Without a compliant DPA you have no lawful route to share personal data | Signed DPA covering the Article 28 terms |
| 10 | Who are your sub-processors, and how are we told when they change? | Their sub-processors touch your data; a silent swap breaks your compliance | Current sub-processor list, change-notification commitment |
| 11 | Where is our data stored and processed, and are there transfers outside the UK or EEA? | International transfers need a valid safeguard or you're processing unlawfully | Data-residency statement, transfer safeguard (SCCs, adequacy) |
| 12 | How do you support data subject access, rectification and erasure requests? | You must answer these in 30 days; you can't if the vendor can't act | Documented DSAR process, deletion route including model-held data |
| EU AI Act | |||
| 13 | Is this system, in our intended use, high-risk under the EU AI Act? | High-risk use triggers heavy duties; you need the classification before you deploy | Written use-case classification against Annex III |
| 14 | Are you the provider and are we the deployer, and what documentation do you give us to meet our duties? | Article 26 deployer duties are yours; you need the provider's instructions to meet them | Instructions for use, technical documentation, provider declaration |
| 15 | Do you provide the logging we need to keep records for the required period? | Deployers must keep automatically generated logs, at least six months for high-risk use | Logging capability and retention confirmation |
| Accuracy and reliability | |||
| 16 | How accurate is the model on our kind of task, and how did you measure it? | A tool that's right in the demo and wrong on your data fails where it counts | Evaluation methodology, benchmark results on representative data |
| 17 | How do you detect and handle hallucinations and model drift? | Confident false output in a live decision is the failure that reaches your customer | Grounding approach, drift monitoring, confidence handling |
| Human oversight and control | |||
| 18 | Can a human review, override or stop an AI decision or action before it takes effect? | Meaningful human control is both an AI Act duty and your last line of defence | Description of the oversight and override controls |
| 19 | What can the AI do on its own, and how is that scoped? | Excessive agency lets an agent act beyond what you intended, with your access | Permission and tool-access model, action allow-lists |
| Transparency | |||
| 20 | How does the system reach a decision, and can you explain a specific output to a customer or regulator? | You have to answer "why did it decide that," and "the model is a black box" won't do | Explainability tooling, decision-logging, audit trail per decision |
| 21 | Do users know when they're interacting with AI, and is AI-generated content labelled? | The AI Act's transparency duties and basic trust both require disclosure | Disclosure and labelling approach in the product |
| Support, SLAs and incidents | |||
| 22 | What are your uptime SLA, support tiers and response times? | An AI tool in a live workflow is a dependency; downtime is your outage | SLA document with credits and response commitments |
| 23 | How do you handle a security incident or data breach, and how fast do you tell us? | GDPR gives you 72 hours to report a breach; you can't if the vendor sits on it | Incident-response and breach-notification policy with timelines |
| Exit and portability | |||
| 24 | If we leave, how do we export our data and configuration, and is our data then deleted? | Lock-in and orphaned data are real costs; deletion on exit is a GDPR duty | Export format, documented offboarding and deletion process |
| 25 | What happens to a workflow that depends on you if you change the model, pricing or terms? | A vendor changing the model underneath you can break a live process without warning | Change-notice terms, version pinning or fallback options |
Adapt the rows to your use case. Add questions for anything specific to your sector, and cut any that genuinely don't apply, noting why so a reviewer can see you considered them.
How do you score an AI vendor assessment?
The questions are the easy part. The scoring is where you make the call.
Score each area red, amber or green. Rate each of the nine areas on the evidence you actually saw, not the promises you heard:
- Green: evidence produced, meets your bar, no material gap
- Amber: partial evidence, or a gap with a credible remediation plan and date
- Red: no evidence, evasive answer, or a gap you can't accept
Weight the areas that carry legal duties. A red on the DPA, on data residency, or on the EU AI Act classification is a different order of problem from a red on support tiers. A legal-duty red is a blocker, not a negotiating point.
Set a decision rule before you start. Agree the threshold up front so the result isn't argued after the fact. A workable rule: any red in data protection, security or the EU AI Act areas blocks the purchase until it's cleared. Two or more ambers across the whole assessment forces a documented risk-acceptance decision.
Name who owns the decision. Procurement runs the process, but the accountable owner is the person who carries the risk if the tool fails: often the CISO for security, the DPO for data protection, and the business owner for the use case. One of them signs the go or no-go, and it's dated and kept on file. Not "the team." A person.
Keep the evidence. The point of the third column is that you can produce a folder, not a memory, when your own auditor or enterprise buyer asks how you vetted the tool. File the SOC 2 report, the DPA and the classification with the decision.
How does it map to the EU AI Act, GDPR, NIST and ISO?
The checklist isn't a framework on its own. It's the working artefact that lets you satisfy several at once. Here's how the areas connect to the four that matter most.
EU AI Act (provider vs deployer). The Act splits duties between the provider who builds a system and the deployer who uses it. If your use of a vendor's tool makes you a deployer of a high-risk system, Article 26 gives you your own obligations: operate it per the provider's instructions, assign competent human oversight, monitor it, keep the automatically generated logs (at least six months for high-risk use), and report serious incidents (EU AI Act Article 26). Questions 13 to 15 exist to get you the classification and the provider documentation you need to meet those duties. Deployer obligations for high-risk systems apply from 2 August 2026 (EU AI Act implementation timeline). Note that as of mid-2026 the timing of some high-risk obligations is under review through the EU's proposed Digital Omnibus, so check the current position before you rely on a date.
GDPR Article 28 (processor obligations). When a vendor processes personal data for you, you're the controller and they're the processor. Article 28 requires you to use only processors giving "sufficient guarantees," and to have a written contract (the DPA) that binds them to act on your documented instructions, keep data confidential, apply Article 32 security, control sub-processors, help you meet data subject requests, delete or return data at the end, and submit to audits (GDPR Article 28). Questions 9 to 12 are built to test exactly this. No compliant DPA means you have no lawful way to hand the vendor personal data.
NIST AI Risk Management Framework. NIST's AI RMF is built around four functions, GOVERN, MAP, MEASURE and MANAGE (NIST AI RMF Core). A vendor assessment is where MAP (understanding the third-party context and its risks) and MEASURE (scoring them on evidence) happen for supplier risk, with GOVERN sitting underneath in the ownership and decision rule. The framework is voluntary and US-published (NIST AI 100-1), but it's the reference most boards now point to for third-party AI risk.
ISO/IEC 42001 and SOC 2. ISO/IEC 42001:2023 is the first international standard for an AI management system, and it can be certified by an accredited body (ISO/IEC 42001:2023). If a vendor holds it, that's meaningful evidence their AI governance is real, not aspirational. For security specifically, a SOC 2 Type II report or an ISO/IEC 27001 certificate (ISO/IEC 27001) is the baseline independent proof behind question 7. And the OWASP Top 10 for LLM Applications (OWASP LLM Top 10, 2025) is the reference to hold the vendor's security answers against in questions 5 and 6.
For where these duties bite by sector, see our AI compliance audit guide and test your position with the EU AI Act readiness tool.
An assessment is a control you re-run, not a gate you clear once
The most common mistake isn't running a weak assessment. It's running a solid one, signing the contract, and never looking again. Vendors don't stand still. The model you assessed gets swapped for a cheaper one. A new sub-processor appears in a jurisdiction you didn't approve. A feature ships that turns a limited-risk use into a high-risk one.
The assessment that earns its keep is the one you reopen when something material changes, and review on a schedule even when it doesn't. New model underneath, re-assess. New sub-processor, re-check residency. Contract renewal, run the whole thing again. That habit is the difference between due diligence and a signature you once collected.
Frequently asked questions
How do you assess an AI vendor?
Work through the areas in the checklist above: model and data provenance, security, data protection, the EU AI Act position, accuracy, human oversight, transparency, support and SLAs, and exit. For each question, ask the vendor, record the answer, and demand the evidence, a SOC 2 report, a signed DPA, a use-case classification, evaluation results. Score each area red, amber or green on what you actually saw, weight the areas that carry legal duties, and set a decision rule before you start so the result isn't argued after the fact. Keep the evidence on file.
Are we liable for a third-party AI tool?
Usually, yes, at least in part. Under GDPR you're the controller for personal data you send a vendor to process, so their handling is your accountability and you need an Article 28 DPA. Under the EU AI Act, if you deploy a high-risk system you carry Article 26 deployer duties even though someone else built it. "The vendor's model did it" doesn't move the risk off you when a customer is harmed or a regulator asks. That's exactly why the due diligence, and the evidence you keep, matters.
What should an AI vendor DPA include?
At minimum the GDPR Article 28 terms: the vendor acts only on your documented instructions, keeps data confidential, applies Article 32 security measures, gets your authorisation before adding sub-processors and tells you when they change, helps you meet data subject requests, deletes or returns the data when the contract ends, and submits to audits. For an AI tool, add explicit terms on whether your data trains their models (it shouldn't, by default), data residency, and the sub-processor list. See our free AI DPIA template for the assessment that sits alongside the DPA.
Is our AI vendor's model high-risk?
It depends on your use, not just the model. The EU AI Act classifies systems by how they're used, and the high-risk cases are listed in Annex III, things like employment decisions, credit scoring, and critical infrastructure. The same model can be minimal-risk in one use and high-risk in another. Get the vendor's written classification for your specific use case, and don't take "it's just a chatbot" as an answer. If you're the deployer of a high-risk use, Article 26 duties are yours.
Do we still need our own assessment if the vendor is certified?
Yes. A SOC 2 report or an ISO 42001 certificate is strong evidence and it saves you time, but it's evidence that feeds your assessment, not a substitute for it. Certification tells you the vendor runs a real programme; it doesn't tell you whether their tool is safe for your use, on your data, in your regulatory context. You still classify the use case, sign the DPA, and decide whether to deploy. The certificate is a green tick in one column, not the whole scorecard.
The bottom line
Most teams buying AI tools don't run a real vendor assessment. They watch a demo, read a one-page security summary, and sign. The gap between that and actual due diligence shows up the moment the tool leaks data, states something false in front of a customer, or turns out to be a high-risk system nobody classified.
Start with the checklist above. Ask every question, demand the evidence, and score each area on what you saw, not what you were told. Weight the legal-duty areas, set your decision rule before you start, and keep the folder. That single discipline puts you ahead of most buyers, and it gives you something concrete to hand a board, an auditor or an enterprise customer's security team.
The harder part is reading the evidence well: spotting a DPA that's missing the teeth, a classification that flatters the vendor, a SOC 2 scope that doesn't cover the product you're buying. That's the work we do. For the fuller picture, read our free AI risk register template and our AI agent risk assessment framework, and if you're vetting a vendor's security posture for a deal, our note on the AI security and compliance data room.