Skip to content
All articles
Audit practice 7 min read· by Lars Zimmermann

The AI does not sign off its own work: reviewers for agents

AI agents report done while parts are missing. How to separate author and reviewer, fix criteria in advance and make silent failures visible.

In short

An AI agent should not sign off its own work, because it checks with the same understanding it built with. An agent workflow becomes reliable with a separate reviewer who never sees the author's working history, criteria fixed before the run, locked test files, counters for every case and a sample checked by a human.

Auf Deutsch lesen: deutsche Fassung

An AI agent works on a task for an hour and reports at the end: done, everything checked, all tests green. Whoever looks finds gaps. One part is missing, another is only sketched, and the tests check something other than the requirement. Guides on AI agents now often say, in substance, that the agent that built something is the worst judge of it.

The sentence is true, and for auditors it is old. Whoever produced a piece of work does not sign it off. That holds in manufacturing, in accounting and in auditing. Here you can read how to build that separation into an agent workflow. And what to do about the quieter failure: the agent that reports success and has done nothing.

Why self-review fails

The first reason is the shared context. The agent checks with the same understanding of the task it built with. If it misread a requirement, it misreads it again during the check. Its tests then confirm its reading. Green tests prove in that case only that the result does what the agent believes is right.

The second reason has been measured. A study from April 2024 describes that language models acting as evaluators rate their own texts higher than those of others, although humans judge both as equally good. The authors link this to the models' ability to recognise their own texts. The limit of this source belongs here too: the study examined models of that time on text tasks. Nobody has measured your agent workflow. I take the direction from it and no number.

What audit practice says about it

ISO/IEC 42001 requires in clause 9.2.2 that auditors are selected and audits conducted in a way that preserves objectivity and impartiality. ISO 19011 lists independence as an audit principle of its own: auditors should, wherever practicable, be independent of the activity they audit.

Both passages refer to the audit of the management system. The standard does not require a reviewer agent anywhere. I transfer the idea because it holds: a sign-off is worth as much as the distance between the one who checks and the one who delivered.

Closer to the agent is control A.6.2.4 from Annex A. It requires verification and validation measures for an AI system to be defined and documented, including criteria for their use. Why the timing of those criteria decides is covered in a separate article.

Why criteria have to be fixed before the run, and why checkable does not yet mean checked:

Automation starts with what can be checked

Four building blocks for the separation

A separate reviewer with a fresh context. The reviewer receives the requirement and the result. It does not receive the author's working history, and not its summary either. Whoever first reads “all done” looks less thoroughly afterwards. That is true of people, and I assume the same effect in models.

Criteria before the run. What “done” means is written down before the agent starts. A freely chosen worked example: the task is to match 40 supplier records against a list. The criterion is then: 40 rows in the result, each with a status and a reference, each deviation listed individually. If one row is missing, the run is not done, whatever the message says.

Lock the test file. The agent may read tests and criteria. It may not change them. That limit belongs in the file permissions. As a request in the task text it stops nothing. An agent that is supposed to turn a red test green and is allowed to change the test has two ways to the goal. One of them is shorter.

Why a rule in the task text is not a barrier, and how to recognise a technical limit:

When the only barrier is a sentence in the system prompt

A counter-test belongs to it. Deliberately build in an error and see whether the check turns red. A check that cannot fail checks nothing.

Demand evidence. The word “checked” is a claim. Evidence has a command, an output, a point in time and a file state. Have the output shown to you, not a retelling of the output.

What counts as audit evidence for a language model and what is merely a statement:

Audit evidence for a language model

Several models that agree count as one source

A popular way out: you ask three models, and if all agree, the result counts as confirmed. In an audit, three witnesses who copied from each other count as one.

The models receive the same task text and the same documents. If the error sits in the document, all three adopt it. In my assessment there is a further point: large language models have learned from similar data and can therefore share similar errors. I do not cite a measurement for that here.

A check becomes independent through a different basis: a measurement, a fixed test, a look at the original source, a person with subject knowledge. Several models help in finding objections. As confirmation they are of little use.

The silent failure: success reported, nothing done

The second failure is harder to see. A sub-agent fails, and the agent above it still summarises “done”. An instruction file is not loaded, and nobody learns of it. A nightly routine aborts, and in the morning there is no message at all. Guides on AI agents often say, in substance, that in such cases there is usually no error pointing to the problem.

A control that can fail without anyone noticing is a finding in an audit. My question on this is: how would you have noticed that the check did not run yesterday? ISO/IEC 42001 requires in A.6.2.6 that the elements for the ongoing operation of an AI system are defined and documented. Monitoring is part of that. Three countermeasures help.

Count cases. Every report names three numbers: received, processed, checked. In the example above that is 40, 40 and 40. If it says 40, 31 and 31, the question about the nine missing cases is asked at once. Without counters, “zero findings” on zero checked cases looks like a clean result.

Define stop conditions. Before the run it is fixed when the workflow halts: a check step returns no result, the numbers do not match, a mandatory file is missing. Then the workflow stops and reports. A workflow that carries on after an error produces a complete-looking report about incomplete work.

A sample by a human. A fixed number of cases per week, drawn at random, held against the original source, filed with date and name. In the example five out of 40. The sample checks the result and, along the way, the reviewer agent. If the human finds something the reviewer let through, you know more than after a hundred green runs.

Which rights and limits an agent needs before it works on its own:

Securing AI agents

Five questions for your agent workflow

  • Who checks the result, and does that reviewer know the author's working history?
  • Where are the criteria for “done”, and were they fixed before the run?
  • Can the agent technically change tests or criteria?
  • What evidence does a run deliver: counters, outputs, file state?
  • How do you notice that a check step has failed, and who draws the sample?

Two clear answers are a start. Five are a workflow you can defend before a management board and in an audit.

What to do on Monday

Take a workflow an agent already handles today. Write down on half a page what “done” means and which three numbers the report must name. Draw five cases from last week and hold them against the source. That takes an hour. Afterwards you know whether your agent delivers or only reports.

This is my interpretation from practice. For legal advice, please turn to the appropriate professionals.

Share: LinkedIn E-Mail

Frequently asked questions

Is it enough if the same agent checks its work in a second pass?+

No. In the second pass it works with the same understanding of the task and with its own history in the context. A misreading from the first pass stays. The check becomes useful with a separate reviewer that knows only the requirement and the result, and with criteria that were fixed before the run.

Does ISO/IEC 42001 require a separate reviewer agent?+

No. Clause 9.2.2 requires objectivity and impartiality for the internal audit of the management system. A.6.2.4 requires defined measures and criteria for verification and validation. How you implement that is your decision. Separating author and reviewer is my transfer of the audit idea to the individual run.

Are three models that say the same thing an independent confirmation?+

By my standard, no. They receive the same task and the same documents and can share similar errors. A check is independent when it rests on a different basis, such as a fixed test, a measurement or a human sample against the original source.

How do I recognise a silent failure?+

By numbers that do not add up. Have every run report how many cases came in, how many were processed and how many were checked. Define at which deviations the workflow halts. And draw a manual sample regularly. A report without counters can be empty without anyone noticing.

Author & expert review: Lars Zimmermann · ISO/IEC 42001 Senior Lead Auditor & Senior Lead Implementer · ISO/IEC 27001 Lead Auditor & Lead Implementer (PECB)

Last updated: 19 November 2026. Researched and reviewed to the best of our knowledge; not a substitute for individual legal advice.

Questions about your own case?

In a free 15-minute intro call we assess where you stand on ISO 42001, ISO 27001 and the EU AI Act, honestly and without a sales pitch.

Continue reading