Skip to content
All articles
Audit practice 9 min read· by Lars Zimmermann

How do you obtain audit evidence for a language model?

The same result reads differently tomorrow. Why a screenshot proves nothing, which six items of evidence hold up and what Article 12 EU AI Act requires.

In short

With a classic system you audit the result. With a language model you audit the process behind it, because the same result can read differently tomorrow. Robust items of evidence are model version, system prompt state, input window, filters and guardrails, the human approval step and the log. A single screenshot is none of these.

Auf Deutsch lesen: deutsche Fassung

The question comes up by the second AI audit at the latest, and it makes experienced auditors falter. You ask for evidence that the system works correctly. You get a screenshot: an input, a good answer, the date in the corner. Looks neat. Then you ask the same question once more, and the answer reads differently. Not wrong, just different. And it dawns on you that you have just audited nothing.

That is not a fault of the business and not a trick by the provider. It is the nature of the thing. A language model is not deterministic: the same input does not necessarily produce the same output. Anyone who audits this with the means that work for a machine or a booking entry is auditing thin air.

I build AI systems myself and I audit them. So here is the answer I give in the classroom, and the audit objects I ask for in a real audit.

Why the result is not evidence

ISO 19011 defines audit evidence in clause 3.10 as records, statements of fact or other information which are relevant to the audit criteria and verifiable. The whole weight lies on the last word.

Verifiable means: a third party can confirm that the statement is true. With an invoice that works, it is there and does not change. With a machine setting that works, it is in the log. With a single model output it does not work, because it is not repeatable. It is a snapshot from a process you did not see.

A screenshot proves that at some point something was on a screen. It does not prove which model answered, with which system prompt, on which data basis and whether anyone looked at the answer before it was used.

From this follows the shift that makes the whole difference. With a classic system you audit the result. With a language model you audit the process that leads to the result. The process is stable and can be documented, the individual result is not.

Six items of evidence that hold up

These are the things I ask about. They can all be recorded, they are all verifiable and they are all independent of the individual answer.

Item of evidenceWhat you ask forWhere it fails
Model versionWhich model, which version, which provider, in use since whenOnly the product name is given, without a version
System promptThe wording, its state, who last changed itIt sits in a configuration field that nobody versions
Input windowWhich data the model actually sees: documents, database extracts, historyThe answer is that it has access to everything
Filters and guardrailsWhat is checked before and after the model, and what happens when they triggerThey exist, but nobody can say when they last fired
Approval stepWho looks at the output before it is used, and how you can see thatThe oversight is described but leaves no trace
LogWhat is recorded, how long it is retained, who may look into itAvailability is logged, not usage

The most important sentence about this table: every row is documented information, not an observation. You can take it with you, file it and bring it out again in two years. That is exactly what a screenshot cannot do.

The approval step deserves a remark of its own, because it is the same point of contention in almost every audit. Human oversight that exists only in the concept is not oversight. I therefore ask how I can recognise that someone is looking: a tick with a timestamp, a second user name, a processing status, a note in the case file. If there is no trace, there is no evidence, and then the oversight is in the concept and nowhere else.

With the input window, asking further is particularly worthwhile, because the first answer is almost always too generous. “The system has access to our wiki” is not a description of the input window, it is a description of a permission. What matters is what actually goes into the request: how many pieces of text are selected, by what criteria, and what happens if nothing suitable is found? If the system then carries on answering from the model’s general knowledge without making that apparent, you have a statement without a basis, and it looks like any other.

And with the system prompt it is about versioning. It is the operating instruction of the system. With a machine nobody would accept the work instruction sitting in a text field that anyone can change and that keeps no history. With AI applications exactly that is the normal case, and it is the most frequent point at which my chain of evidence breaks.

What the standard and the law require here

ISO/IEC 42001 has three controls that apply exactly here. A.6.2.6 requires the elements for operation and monitoring of the AI system. A.6.2.7 requires the technical documentation to be determined. And A.6.2.8 has the title that sums the matter up: AI system recording of event logs. The organisation must determine in which phases of the life cycle logging takes place, at a minimum during use.

On the legal side there is Article 12 of the EU AI Act. For high-risk systems it requires the automatic recording of events over the lifetime of the system, applicable from 2 December 2027 for systems under Annex III and from 2 August 2028 for systems under Annex I (Regulation (EU) 2026/1744). Paragraph 2 says what for: to identify situations that present a risk or a substantial modification, to facilitate post-market monitoring under Article 72 and to monitor operation under Article 26(5).

For one particular group paragraph 3 then becomes very concrete and names a minimum scope: the period of each use with start and end, the reference database against which a check was made, the input data that led to a match, and the identification of the persons who verified the result.

  • The obligation under Article 12 hits high-risk systems. Anyone who uses a language model for internal text work does not, as a rule, fall under it.
  • The minimum scope mentioned in paragraph 3 applies to an even narrower group, namely biometric systems under Annex III, point 1(a).
  • The list is nevertheless useful for every business, because it shows what the legislator considers the minimum amount of traceability.

I say this so clearly because in consulting conversations the opposite readily happens: Article 12 is cited as if it applied to every chatbot. That is wrong, and it damages the credibility of those who rightly invoke it.

One point that regularly gets lost with the word log: there are two kinds of it, and only one holds up. The operational log records that the service ran, how fast it answered and when it failed. That is availability. For evidence you need the other: a record of usage, that is who made which request when, under what conditions it was answered and what happened to the result. If a business shows me a log and it contains response times in milliseconds, then I am holding the wrong document.

Then there is retention. A log that is overwritten after seven days does not help in an audit that takes place six months later. The question about the retention period therefore belongs in the same place as the question about scope, and both should have the same answer as the other records of the management system.

The change that nobody signs

Here lies the difference from every other system you have ever audited. A machine changes its behaviour when someone rebuilds it. Software changes it when someone installs an update. A bought-in language model can change it without anyone in your organisation doing anything.

A practical case from a supplier audit: the evaluation of incoming enquiries had been running stably for a year. Between two samples the wording of the results changed noticeably. Nobody in the business had changed anything. The provider had swapped the model version behind the same product name. Contractually nothing had happened, technically everything.

For evidence this means two things. First, the model version belongs in every record, not just the product name. Second, there needs to be a place where such changes are noticed at all. In practice that is the management review: a separate agenda item with the current inventory of the AI systems in use and the changes since last time.

Anyone who does not keep this item only notices a change when it has consequences. And then the question of since when the system has been working differently can no longer be answered.

How I actually proceed in an audit

Three moves that make the difference, and none of them needs a tool.

First: the same input twice. I have the same case run through twice, with an interval. If the results differ, that is not a finding, it is the starting position. The finding arises from how the business deals with it. Is there a range that counts as acceptable? Who defined it? How does anyone notice that it has been left?

Second: the log instead of the screenshot. I ask for the extract for a period. An example is something the business brings along, an extract is produced before my eyes. The difference is the same as between a prepared file and reaching into the shelf.

Third: the chain backwards. I take a result that the business actually used and go back: which version produced it, with which system prompt, on which data basis, who approved it. Where the chain breaks is where the finding lies. Mostly it breaks at the system prompt, because nobody versions it.

The selection of the case is important here. I do not have one shown to me, I draw it from the inventory, and deliberately not a simple one: a case in which something was queried or corrected. A smooth case shows that the chain holds on a good day. A contentious one shows whether it holds when it matters, and that is exactly what an audit is about.

Where the chain breaks, incidentally, I do not word the finding as “the system is not traceable”. That would be an assessment of the tool. The finding is that the organisation has not determined in which phases logging takes place, or that it has determined it but not adhered to it. That is verifiable, it has a reference, and it can be remedied. The other sentence only leads to a discussion about technology that nobody wins.

And there is one expectation I take away from businesses, because otherwise they build the wrong thing: the aim is not to make every output reproducible. That would only be possible by taking away from the system the capability that was bought. The aim is to make the conditions reproducible under which it worked.

What this means for bought-in systems

Most businesses do not build a model, they buy a service. Four of the six items of evidence thereby move to the provider: model version, filters, log and part of the input window lie there. They nevertheless remain your responsibility, because the statement to the outside world is made by your organisation, not by the provider.

That is why these points belong in the contract and in the supplier audit, not on a wish list: are model changes announced, and with what notice? Do you get logs, or only an availability commitment? May you audit or have an audit carried out? Who are the subcontractors?

Anyone who does not get these four questions answered does not have an evidence problem but a procurement problem. And that is the better news, because a procurement problem can be solved before signing.

Share: LinkedIn E-Mail

Frequently asked questions

Is a screenshot sufficient as audit evidence for an AI output?+

No. ISO 19011 requires in clause 3.10 that audit evidence is verifiable. A single output of a language model is not repeatable and therefore not verifiable. It only proves that at some point something was on a screen.

Do I have to log every AI output?+

The obligation of automatic recording under Article 12 of the EU AI Act hits high-risk systems. Anyone who uses a language model for internal text work does not, as a rule, fall under it. A record is sensible nevertheless, because without it no statement about operation is robust.

What do I do if the same result reads differently twice?+

That is the starting position, not the finding. The finding arises from how the business deals with it: is there a defined range, who determined it, and how does anyone notice that it has been left?

Which ISO 42001 controls apply to evidence?+

Above all three from Annex A: A.6.2.6 on operation and monitoring of the AI system, A.6.2.7 on technical documentation and A.6.2.8 on the recording of event logs. The latter requires determining in which phases of the life cycle logging takes place.

What changes if the AI system is bought in?+

Four of the six items of evidence then lie with the provider: model version, filters, log and part of the input window. Responsibility for the statement to the outside world remains with you. That is why the announcement of model changes, access to logs, the right to audit and subcontractors belong in the contract.

Author & expert review: Lars Zimmermann · ISO/IEC 42001 Senior Lead Auditor & Senior Lead Implementer · ISO/IEC 27001 Lead Auditor & Lead Implementer (PECB)

Last updated: 10 November 2026. Researched and reviewed to the best of our knowledge; not a substitute for individual legal advice.

Questions about your own case?

In a free 15-minute intro call we assess where you stand on ISO 42001, ISO 27001 and the EU AI Act, honestly and without a sales pitch.

Continue reading