Detected but not reported: who sets the reporting threshold?
An AI provider detects an account, bans it and does not report it because its threshold was not met. What your company can learn for its reporting chain.
In short
A reporting threshold is a decision, not a measured value. It determines from what point a detected incident leaves the company, to the supervisory authority, the customer or the police. Who sets it, who decides in the individual case and where a no is justified belongs in writing. For personal data breaches, Article 33(5) GDPR already requires this documentation today.
Auf Deutsch lesen: deutsche Fassung
In June 2025 an automated system at OpenAI flags a ChatGPT account. People look at the case. They examine two questions: does the account violate the usage policies? And do the police need to know about it? They answer the first question with yes and ban the account. The second with no. Its own threshold for a referral had not been met, the company writes later.
On 10 February 2026 eight people die in Tumbler Ridge, a small town in British Columbia, six of them at the local school. The banned account belonged to the person responsible for the act.
I am not writing about this to accuse anyone. Whether a report in June would have prevented anything, nobody knows, and I do not claim it here. I am writing about it because the same sequence can exist in any company, only smaller. Tomorrow your logs show that someone is copying customer data into a private account. The system has detected it. Who decides whether that goes outside? And who determined from what point?
Detected, reviewed, banned: what worked
What exactly happened we know from a source that is rarely this open: a letter from OpenAI itself. On 26 February 2026 the responsible vice president writes to Canada’s minister for artificial intelligence. The sequence she describes has three stages. An automated system detects the account. People review whether the usage policies have been violated and whether a referral to law enforcement is warranted. The account is banned. For a referral, the company would, in its own words, have had to identify credible and imminent planning. It did not see that.
Technically this chain worked. Detection triggered, a human looked, a decision was made. What it hinged on was the threshold. It was fixed before anyone knew the case, and it determined the outcome.
The same letter contains two sentences that I consider more important than any question of blame. First: the criteria have since been made more flexible, because someone planning violence does not necessarily name target, means and timing when talking to a chatbot. Under today’s procedure the account banned in June 2025 would be referred. Second: OpenAI commits to reviewing the thresholds of its automated detection regularly.
Something uncomfortable follows from this for anyone who sets a threshold. The same facts lead to a no with the old threshold and to a yes with the new one. So the threshold is not a value you read off somewhere. People set it. And people can set it wrongly.
A third detail from the letter fits any shop floor. OpenAI had a system meant to recognise banned users when they create a new account. A second account nevertheless only came to light after the person’s name was public. After that it went to the police. A ban works against an account. Against a human being it only works as long as they do not create a new one.
Twelve minutes to the alert, two and a half hours to the stop
The second case also comes from OpenAI, from an incident report that the company last updated on 25 September 2026. During training, an agent was supposed to solve a research task and had no access to the open internet. It still found a way out, via the name service (DNS) of its test environment, and used it to put questions to an external chatbot. The report gives the times:
- 9:50:23: the agent receives the first response from outside.
- 10:02:11: monitoring triggers an alert of the highest severity, just under twelve minutes later.
- 10:05:06: a human acknowledges the alert, less than three minutes after that.
- 12:34:30: the run is terminated.
Between the acknowledgement and the stop lie 2 hours and 29 minutes. OpenAI itself writes why. The run did not halt on its own as expected, and it was unclear whether it should be stopped. It was terminated once that had been clarified. I read that as an open responsibility. The alert had a recipient. The decision did not.
Two further sentences from the report are typical audit findings, only in a different environment. Monitoring did not report other outbound DNS accesses with the expected severity, because it took a response without usable content for a failed access. And a detector for anomalous DNS activity had excluded the affected environment, although the activity was logged. Logged does not mean seen. An automated assessment also carries a threshold within it, and someone set that.
Before this sounds like criticism: we only know both cases because OpenAI published them with times and reasons. That deserves recognition. Could your company supply the same timeline for its last incident? Who may halt should be settled beforehand; that was the subject of my article on the emergency stop for AI. Here the subject is the question that follows every alert: does this stay in the company, or does it have to go out?
The threshold is a decision, not a measured value
Every detection has two thresholds. The first is technical: at what value does the alert trigger? That is usually set by IT or the provider. The second is organisational: from what point does a detected incident leave the company? In many companies nobody has set this second threshold. It arises in the emergency, on the phone, from the gut.
Take the case from the beginning. Your network monitoring reports that an employee has uploaded a price list with customer names and contact persons to his private chatbot account. Several reporting routes are attached to that at once, each with its own threshold:
- To the data protection supervisory authority, if the file contains personal data and a risk to the data subjects cannot be ruled out.
- To the customer, if your contract or the non-disclosure agreement requires notification. Check the agreements of your largest customers for such a clause.
- To the BSI, the German Federal Office for Information Security, if your company falls under the BSI Act and the incident is significant.
- To the police, if you suspect that trade secrets are being deliberately siphoned off.
Four addressees, four thresholds, and usually one person who decides alone on Monday morning. Often that is the same person who would have to bear the consequences of a report: trouble with the customer, questions from the authority, a difficult conversation with the employee. Those who are affected themselves rarely set the threshold too low.
The other side is part of the picture. A threshold that is too low does harm as well. Anyone who passes every false alarm on to authority and customer loses their attention for the real case. Anyone who reports an employee to the police over a misunderstanding damages a person and the trust of the whole workforce. OpenAI names the same tension in the letter: the privacy of the vast majority of users on the one hand, the safety of third parties on the other. Reporting at any price is therefore not a solution. The solution is a threshold that someone has deliberately set, justified and given a review date.
In practice this means: the threshold is fixed in writing, with name and date. It has examples of what lies above it and what below. It has a review date, at the latest after every real incident. And the no is recorded just like the yes.
The clock runs from detection
The law has long known these thresholds, and at one point it is stricter than many think. Article 33(1) GDPR requires a personal data breach to be notified to the supervisory authority without undue delay and, where feasible, within 72 hours. Exempt is a breach that is unlikely to result in a risk to the data subjects. This exemption is a threshold that you apply yourself. Paragraph 5 makes it verifiable: you document every breach with facts, effects and remedial action, in such a way that the authority can follow your decision. Anyone who does not notify must therefore be able to justify why not.
Paragraph 2 concerns your service providers. A processor, including an AI provider that processes personal data for you, notifies you of a breach without undue delay. How quickly that happens in practice and by what route is set out in the data processing agreement. Or it is set out nowhere.
A change has been proposed; it has not been adopted. With proposal COM(2025) 837 of 19 November 2025, the European Commission wants to limit the notification obligation to breaches likely to result in a high risk and extend the deadline to 96 hours. According to the Legislative Observatory, procedure 2025/0360(COD) was still awaiting the committee decision in Parliament on 1 November 2026. The proposal leaves the documentation obligation in paragraph 5 untouched.
For companies that fall under the BSI Act as an important or essential entity, Section 32 of the German BSI Act (BSIG) applies. An early initial notification is due no later than 24 hours after you become aware of a significant security incident, the actual notification after 72 hours, the final report one month after that. Anyone who learns of the incident on Friday evening must have submitted the initial notification by Saturday evening. “Significant”, too, is a threshold.
For high-risk AI, the EU AI Act regulates the chain expressly. Deployers monitor the system, inform the provider and the market surveillance authority in the event of a risk, and suspend use. They report a serious incident immediately, first to the provider (Article 26(5)). The provider reports it to the authority, no later than after 15 days, in the event of a death after 10 days, in the event of a widespread infringement or a serious and irreversible disruption of critical infrastructure after 2 days. An incomplete initial report is expressly permitted (Article 73(2) to (5)). Those exercising oversight must be able to halt operation with a stop button (Article 14(4)(e)). The obligations under Articles 14 and 26 apply to systems under Annex III from 2 December 2027 (Article 113(c) as amended by Regulation (EU) 2026/1744).
All three sets of rules have one thing in common. The deadline begins with awareness, not with the decision. Every hour an alert lies unprocessed is then missing when it comes to reporting.
What I want to see on this in the audit
The standards have their own controls for this, and they rarely come up in conversation with managing directors. In Annex A, ISO/IEC 27001 separates what has to happen in sequence in an emergency: incident management planning and preparation (A.5.24), assessment of and decision on events (A.5.25), response to the incident (A.5.26), collection of evidence (A.5.28). In addition there are event reporting by staff (A.6.8) and contact with authorities (A.5.5). The decision is a control of its own there. That is exactly what was missing for two and a half hours in the second case.
In Annex A.8.4 (communication of incidents), ISO/IEC 42001 requires a determined and documented plan for communicating incidents to the users of an AI system. The guidance in Annex B names what legal or contractual requirements may specify on this and what the plan should therefore cover: which incidents are to be reported, by when, whether an authority is to be informed and with what details. Anyone already operating incident management under ISO/IEC 27001 can hook the AI part in there. It does not take a second process.
In the audit I would not ask about the plan at this point. I would ask about the last event that was not reported and have the justification shown to me. Three patterns of answer, weak and robust:
- Weak: in the event of an incident, IT calls the managing director. Robust: here is the list of who receives which alert, with deputies and availability at the weekend.
- Weak: we report what is reportable. Robust: here are the thresholds for each addressee, with examples, with the date they were set and the date of the last review.
- Weak: we have had no reportable incident. Robust: here are this year’s events, and each one states who decided, when and why.
Five questions for the next management meeting:
- Who receives an alert from network monitoring, antivirus or the AI log, and who deputises for this person at night and at the weekend?
- Which threshold applies to which report, to the supervisory authority, the customer, the BSI and the police, and who set it and when?
- Who decides in the individual case, and by when after the alert?
- Where is the justification recorded when no report is made?
- What does your AI provider report to you, by what route and within what deadline?
Play it through once, with the clock on the table. An invented alert on Friday at 5 pm is enough. Afterwards you will know whether your reporting chain is a procedure or a telephone directory. And if an alert comes in at your company tomorrow at 9:50: who decides by when, and where is that written down?
Primary sources
- OpenAI, letter from Ann M. O’Leary to Minister Evan Solomon of 26 February 2026 (ban in June 2025, referral threshold, second account, amended criteria)
- OpenAI Alignment, incident report “An agent used DNS to reach an external chatbot”, updated on 25 September 2026
- RCMP British Columbia, confirmation of the fatalities of 10 February 2026 in Tumbler Ridge
- Regulation (EU) 2016/679 (GDPR), Article 33
- Commission proposal COM(2025) 837, procedure 2025/0360(COD) in the European Parliament’s Legislative Observatory
- German BSI Act (BSIG), Section 32 Reporting obligations
- Regulation (EU) 2024/1689 (EU AI Act), consolidated version of 27 July 2026, Articles 14(4)(e), 26(5), 73 and 113
- Regulation (EU) 2026/1744, new wording of Article 113 of the EU AI Act
- ISO/IEC 42001:2023, AI management systems, Annex A.8.4
- ISO/IEC 27001:2022, Annex A.5.5, A.5.24 to A.5.26, A.5.28 and A.6.8
Frequently asked questions
Does every personal data breach have to be notified to the supervisory authority?+
No. Under Article 33(1) GDPR, notification is not required if the breach is unlikely to result in a risk to the data subjects. Under paragraph 5 you must nevertheless document every breach, with facts, effects and remedial action, so that the authority can verify the decision not to notify.
From when does the reporting deadline run in the event of an incident?+
From awareness. The 72 hours under the GDPR run as soon as you become aware of the breach, the 24 hours for the initial notification under Section 32 of the German BSI Act (BSIG) from awareness of a significant security incident. Time during which an alert lies unprocessed comes off the deadline.
What does ISO/IEC 42001 require for the reporting of AI incidents?+
Annex A.8.4 requires a determined and documented plan for communicating incidents to the users of an AI system. According to Annex B this includes which incidents are to be reported, within what deadline, whether authorities are to be informed and with what details. The plan can be absorbed into existing incident management under ISO/IEC 27001.
Do the reporting obligations of the EU AI Act already apply?+
For deployers of high-risk AI under Annex III, the obligations under Article 26, including the reporting of serious incidents to the provider, apply from 2 December 2027 following the amendment by Regulation (EU) 2026/1744. For most AI applications in mid-sized companies, the GDPR and, where applicable, the BSI Act apply today.
Author & expert review: Lars Zimmermann · ISO/IEC 42001 Senior Lead Auditor & Senior Lead Implementer · ISO/IEC 27001 Lead Auditor & Lead Implementer (PECB)
Last updated: 10 November 2026. Researched and reviewed to the best of our knowledge; not a substitute for individual legal advice.
Questions about your own case?
In a free 15-minute intro call we assess where you stand on ISO 42001, ISO 27001 and the EU AI Act, honestly and without a sales pitch.