Skip to content
All articles
Audit practice 9 min read· by Lars Zimmermann

Vetting AI service providers: 20 questions before you sign

When you buy an AI service, you buy the provider’s governance with it. 20 questions in four blocks, each with the clause or article that backs it alongside.

In short

A question catalogue does not replace an audit, it prepares one. The 20 questions cover four blocks: who holds which role, where the data comes from, what happens when the model changes, and what you receive in writing. Each question hangs on an Annex A control of ISO/IEC 42001 or on an article of the EU AI Act.

Auf Deutsch lesen: deutsche Fassung

When you buy an AI service, you buy the provider’s governance with it. That sounds like a sentence for a slide, but it has a very tangible side: if the provider changes its model, your result changes. If its training data has a problem, you have it too. And if it puts your name on the system, you can become a provider within the meaning of the EU AI Act without anyone in your company having written a line of code. I ask the following twenty questions before anything is signed. They are deliberately uncomfortable.

What this catalogue is good for, and what it is not

I have written elsewhere that a self-assessment questionnaire is a good filter and poor evidence. That still holds, and this catalogue does not contradict it. It is a filter. Its job is to tell you before the first meeting where you need to look, and to show you after the responses come back whether a meeting is worthwhile at all.

The difference lies in what you do with the answers. Anyone who files them has a collection of claims. Anyone who reads them as an audit plan has an expectation for each point against which something can be held up in the meeting. That is exactly why each question is followed by the clause or article that backs it.

Block 1: who is actually who?

The question of roles comes first because it decides everything else. And it is trickier than it looks, because in German the same word means two different things in two sets of rules. The German edition of ISO/IEC 42001 (DIN EN ISO/IEC 42001:2026-08) titles control A.10.3 “Lieferanten” (suppliers). Older, unofficial translations say “Anbieter” there, and in everyday German almost everyone says Anbieter when they mean their supplier. The German version of the EU AI Act, by contrast, uses “Anbieter” in Article 3(3) for the manufacturer, the provider in the English text. When everyone in a meeting talks about the provider, it is worth asking who is meant.

That is not quibbling over words, it has an expensive consequence. Article 25 of the EU AI Act describes the circumstances under which a distributor, an importer or a deployer is itself considered the provider of a high-risk system, for example if it passes the system on under its own name or changes its intended purpose. In mid-sized companies that happens not in the IT department, but in marketing: a purchased tool is given an in-house name, its own logo, its own interface. With that, the question of roles has been posed anew, and nobody has noticed. Article 26 also sets out what a deployer owes in any case, including human oversight and use in accordance with the instructions for use. Anyone who does not know which of the two roles they are in also does not know which of the two lists of obligations applies to them.

QuestionWhy it mattersReference
Are you the provider, deployer or distributor of the system you supply to us?The role decides the obligations. Anyone who cannot name it has not examined it.Art. 3, Art. 25 and Art. 26 EU AI Act
Do you use a third-party model, and if so, which one in which version?A great many services are an interface in front of someone else’s model. That is fine, but it has to be on the table.ISO/IEC 42001, A.10.3
Does the system appear under your name or under ours?Anyone who places a system on the market under their own name can become a provider themselves. That happens in marketing, not in IT.Art. 25 EU AI Act
Whom do you subcontract to, and in which country are they based?The chain does not end with your contracting partner. Subcontractors belong on the table before anything is signed.ISO/IEC 42001, A.10.3
Who in your organisation is responsible for AI by name, and what may that person decide?A role without a name is not a role. A role without authority is a letterbox.ISO/IEC 42001, A.3.2

Block 2: where does the data come from?

This is the block in which the answers most often become evasive. That is itself a result. A provider that cannot or will not describe the provenance of its training data is telling you that it does not know it itself.

Also ask the data question twice, once for training and once for operation. These are two different data flows with two different risks. Training is about rights, representativeness and bias. Operation is about your own inputs: where do they end up, who sees them, how long do they stay there. Many providers answer the one question at length and the other not at all, and it is usually the second that is closer to home for you.

QuestionWhy it mattersReference
What was the model trained on, and on what basis were you allowed to use this data?Without provenance there is no statement about rights, quality or bias.ISO/IEC 42001, A.7.3 and A.7.5
Do our inputs flow into your training, and how do we switch that off?The answer is rarely in the quotation and almost always in the terms of use. It belongs in the contract.ISO/IEC 42001, A.7.2
Where is the data held, in which country, with which operator?The server location is a partial answer. What matters is who has access and which law they are subject to.ISO/IEC 42001, A.4.3 and A.4.5
How long do you store inputs and outputs, and how do you evidence deletion?Deletion at the end of the contract is the point that is argued about later.Art. 28 GDPR
What data quality do you commit to, and what do you measure it by?Commitments without a metric are declarations of intent.ISO/IEC 42001, A.7.4

Block 3: what happens when the model changes?

This is the block that classic supplier questionnaires do not know, because it was not an issue with conventional software. An AI service can change its behaviour without anyone touching a contract: a new model in the background, a new system prompt, a different version. Your acceptance was then a snapshot.

QuestionWhy it mattersReference
Who in your organisation decides on a model change?If the answer is “the upstream supplier”, you have two levels without control.ISO/IEC 42001, A.6.2
Are we informed before a change, and with how much notice?Without lead time you cannot retest. Without a notice period the commitment is worthless.ISO/IEC 42001, A.6.2
Can it be established afterwards which version answered on a particular day?What you cannot reproduce, you cannot evidence. That is the core of every later clarification.Art. 12 EU AI Act, ISO/IEC 42001 A.6.2
How do you check before a change that the behaviour does not deteriorate?A provider without a regression test shifts its risk to you.ISO/IEC 42001, A.6.2
What happens to our settings and prompts when there is a change?Configurations are your work product. They should survive a change.ISO/IEC 42001, A.9.2

Block 4: what do you receive in writing?

In the end what counts is what you hold in your hands when the friendly contact person moves to another company. This block asks for documents, not for statements.

QuestionWhy it mattersReference
Is there system documentation for users, and what does it say about limits and intended use?Without this document you cannot instruct your own people.ISO/IEC 42001, A.8.2
Do you have an impact assessment for this system, and may we inspect it?Anyone who has never assessed the impacts does not know their own risks.ISO/IEC 42001, A.5.2
Which known limits and failure cases do you tell us about of your own accord?A provider that knows no limits has not measured. Honest limits are a mark of quality.ISO/IEC 42001, A.8.2
What does the data processing agreement say about deletion and about evidence of deletion?The contract is the obligation. Following up is also the obligation.Art. 28 GDPR
Do you grant us an audit right, and what exactly may we audit?Without an agreed audit right, an audit is a request. The regulation expressly provides for this right.Art. 28(3) GDPR

How to read the answers

The catalogue measures content and behaviour. Three patterns are more telling than most individual answers.

  • The speed. Anyone who needs more than two weeks for roles, subcontractors and the deletion concept does not have these documents, but is producing them for you right now.
  • The reference to confidentiality. For training data it is sometimes justified. For the provider’s own role, for subcontractors and for the change procedure it is an excuse.
  • The answer that does not fit your question. If you ask about the change procedure and receive a certificate, that is a finding, not a misunderstanding.

A poor response, by the way, does not automatically mean: hands off. It means: negotiating leverage. Every gap the catalogue uncovers can be written into the contract as long as it has not yet been signed. No information on the change procedure becomes a duty to inform with a notice period. No evidence of deletion becomes a deletion confirmation in text form. No audit right becomes an agreed audit right. After signature the same gap is merely an annoyance that stays with you for three years.

And one more thing that often gets lost in purchasing: these twenty questions are not a vote of no confidence. A provider with clean governance answers them in an hour and is glad that someone is finally asking. The questions do not separate the good from the bad, they separate the prepared from the unprepared.

If you have the answers and need someone to hold them up against reality: that is exactly what a second-party audit is, and that is exactly what I do for you.

View supplier audits

A final note on the order. Do not send the catalogue to all your service providers at once. Start with those whose failure would cost you the most, and work your way down. I have described separately how to form this ranking.

Expect this catalogue to age. Two years ago nobody would have asked about the change procedure for a model, because hardly anyone had bought one. In two years there will be questions that nobody asks today. Take the twenty as a basic framework and, after every supplier conversation, add the one question you were missing in the meeting. A catalogue that grows with practice is worth more after a year than any template you download somewhere.

Share: LinkedIn E-Mail

Frequently asked questions

Does a question catalogue replace the supplier audit?+

No. A completed catalogue is a claim by the provider about itself. It is a good filter before the meeting and poor evidence afterwards. Its job is to give the audit a target.

Why is each question followed by a clause or article?+

So that you can justify the question when someone asks back, and so that you can later measure the answer against something. A question without a yardstick produces an opinion, not a finding.

Which is the most important block when time is short?+

Block 3, the model change. Others also check roles and data. That the behaviour of an AI service can change without a contract amendment is overlooked by most classic supplier questionnaires.

Does the German word “Anbieter” mean the same in ISO 42001 and in the EU AI Act?+

No, and that is a real stumbling block. In everyday German and in older translations of ISO/IEC 42001 the supplier is often called “Anbieter”, while the official German edition titles A.10.3 “Lieferanten” (suppliers). The German version of the EU AI Act uses “Anbieter” in Article 3(3) for the manufacturer, the provider in the English text.

Author & expert review: Lars Zimmermann · ISO/IEC 42001 Senior Lead Auditor & Senior Lead Implementer · ISO/IEC 27001 Lead Auditor & Lead Implementer (PECB)

Last updated: 10 November 2026. Researched and reviewed to the best of our knowledge; not a substitute for individual legal advice.

Questions about your own case?

In a free 15-minute intro call we assess where you stand on ISO 42001, ISO 27001 and the EU AI Act, honestly and without a sales pitch.

Continue reading