Skip to content
All articles
Law & regulation 8 min read· by Lars Zimmermann

Can an AI dismiss staff? The Luna case and Article 22 GDPR

An AI agent in San Francisco dismissed an employee. What really went wrong, and what applies to automated decisions in Germany under GDPR and the AI Act.

In short

Not on its own. Under Article 22(1) GDPR nobody may be subject to a solely automated decision with legal effect, and a dismissal has one. A system may prepare. A human who can also reject the recommendation must decide and give reasons. In addition, Annex III of the EU AI Act covers dismissals, applicable from 2 December 2027.

Auf Deutsch lesen: deutsche Fassung

Since 1 April 2026 an AI has been running a real shop in San Francisco. Product range, prices, opening hours, hiring, marketing, design. The agent is called Luna, the shop is called Andon Market and is on Union Street. The whole thing is operated by Andon Labs, and the lease runs for three years.

Luna has since dismissed an employee. As far as is known, this is the first dismissal issued by an AI. The story ran through the press as a nightmare image. The cold machine that throws people out.

If you read up on what actually happened, the picture is reversed. Luna was not too harsh. She was too late. The reason for that sits in every agent you are currently testing in your business.

What really happened in San Francisco

Luna wrote herself an employee handbook. It contained a clear rule: three unexcused late arrivals within 30 days, then a formal written warning. Repeated lateness can lead to termination of employment.

Then an employee arrived late. On 17 of 23 shifts.

By that time the handbook had disappeared from Luna’s memory. She kept excusing, shift after shift.

Only the researchers brought her back to her own rule. Luna’s recommendation nevertheless stayed at a verbal warning. Her assessment only changed when the humans contributed a record from outside the system, a formal conversation that had already been held, and asked her to reassess the employee’s suitability. Then she recommended parting ways.

Andon Labs itself recorded that Luna was using Claude Opus 4.8 at the time and that most models would have decided the same way. That is the more honest part of the story. No vendor slipped up here.

The point is not harshness, but forgetfulness

A language model keeps its working state in a limited window. When the window fills up, it is summarised and compacted. What gets thrown away is what the model considers unimportant. Not what you consider unimportant.

In everyday use this is hardly noticeable. Here it cost a rule that the agent had written down itself. The handbook continued to exist as a file. In the agent’s head it no longer existed.

It went the other way too. Luna placed job advertisements, conducted telephone interviews and decided on hires. She did not consistently tell applicants that she is an AI. Her own reasoning: that would confuse applicants and put good people off.

Two directions, one pattern. The agent does not keep to what it has laid down itself, and nobody in the system notices.

Apply that to your business. On Monday an agent is instructed always to check back on invoices over EUR 5,000. By Thursday the history has grown long and has been compacted. The limit is gone. The agent carries on working, confident and wrong. It will be noticed when someone goes through the postings. The same mechanism that cost Luna her handbook, only with your money.

In Germany this question was answered long ago

Article 22(1) GDPR gives every person the right not to be subject to a decision based solely on automated processing which produces legal effects concerning them or similarly significantly affects them. A dismissal is the textbook example.

Paragraph 2 has three exceptions: necessity for a contract, a legal provision with suitable safeguards, explicit consent. If the controller relies on contract or consent, paragraph 3 still requires suitable measures to protect the data subject. In the case of a legal provision, the provision itself must contain the safeguards. These include at least the right to obtain human intervention.

The word solely is the lever, and it is regularly read too generously. A human who signs off the system’s recommendation without examining it does not turn an automated decision into a human one. Whoever is meant to intervene needs three things: the authority to do so + the information for it + the time. If one of them is missing, the oversight is decoration.

In practice the case rarely looks like a dismissal. An agent plans shifts and permanently takes someone off the rota. A rating system sorts a person into the lowest group, and the bonus depends on it. An access system withdraws rights because a pattern looks conspicuous. None of these decisions is called a dismissal. All of them can significantly affect a person, and that brings them under Article 22.

California has just caught up at exactly this point. On 30 September 2026 Governor Newsom signed the No Robo Bosses Act, Senate Bill SB 947. Employers may no longer rely solely on automated systems for dismissals and disciplinary measures. A human must confirm the decision on the basis of additional information, such as supervisor assessments, feedback from colleagues or the personnel file. The law applies from 1 July 2027. A predecessor had failed a year earlier on the governor’s veto.

For your business this changes nothing directly. It is useful all the same, in conversation with everyone who considers Europe overregulated. In 2026 a US state arrives where Article 22 has stood since 2018.

Dismissals are in the high-risk annex of the EU AI Act

Annex III, point 4(b) of the EU AI Act names AI systems intended to be used to make decisions affecting the terms of work-related relationships, promotion and termination of work-related contractual relationships. Affecting is enough. The agent does not have to take the decision itself.

Point (a) covers the other side: recruitment and selection, targeted job advertisements, screening and filtering applications, evaluating candidates. Luna did both, the selection and the separation.

Recital 57 gives the reason. Such systems appreciably affect career prospects and livelihoods, and they can perpetuate historical patterns of discrimination, for example against women, certain age groups or persons with disabilities.

In terms of time you have breathing space. The Digital Omnibus, Regulation (EU) 2026/1744 of 8 July 2026, Official Journal of 24 July 2026, in force since 27 July 2026, has postponed the application of Annex III to 2 December 2027. Breathing space does not mean exemption. Whoever lets an agent into HR work today is building the system that will fall under these obligations in a good year’s time. Retrofitting is more expensive at this point than thinking ahead.

The classification works in both directions, and the other direction speaks in your favour. By no means every tool in the HR department becomes high-risk as a result. Under Article 6(3), a system drops out of Annex III if it does not pose a significant risk, for example because it only performs a narrow procedural task, and does not perform profiling. A scheduling assistant that sends out interview invitations decides nothing. Nor does a text-module generator for job advertisements. The line runs where a system evaluates or pre-sorts a person. Write this classification down once, with reasons. Then you do not have to invent it in the audit.

What I want to see at this point as an auditor

ISO/IEC 42001 deals with human oversight most extensively in the implementation guidance for control A.9.3 (Annex B.9.3), the objectives for responsible use of AI systems. I ask four questions from it in every audit in which an agent has a say in decisions about people.

  • In which stages of the life cycle is human oversight provided for, and what did you base this selection on?
  • Do the reviewers have the authority to override a decision of the system? Show me a case in which that happened.
  • Did you examine whether automated decision-making is appropriate for this use case at all, and where is that documented?
  • How do the persons performing oversight report concerns about system outputs, and to whom?

The second question decides the rest. The typical nonconformity is: human oversight is provided for, but there is no record of whether and how often anyone actually intervened. The sample then shows that system outputs are confirmed throughout, without any examination of substance. The control is therefore not effective, even if the oversight concept is neatly written.

I know two answers to this question well. The weak one is: of course it is possible to intervene at any time, that is regulated in the concept. The robust one is: on 14 March the team lead rejected the system’s recommendation, here is the case, here are the reasons, and we then adjusted the threshold. The first answer costs me ten minutes of follow-up questions. The second closes the topic.

In the Luna case exactly this record was missing. There was a rule. There was no evidence of which late arrival was waved through for which reason. Only a document from outside the system got things moving, and even that only after humans had specifically looked for it.

An oversight concept without a record of interventions is a declaration of intent. In the audit the record counts, the concept alone does not hold.

What you can check on Monday

You do not need a project for this. Three questions are enough for a first overview.

  • Which agents in your company prepare decisions about people or take them? Pre-selection of applicants, shift planning, performance rating, withdrawal of access. Write the list down before you judge.
  • Which rule has the agent given itself or received from you, and where is it kept outside its memory? A rule that only exists in the conversation history is gone after the next compaction.
  • Who last overruled a recommendation of this agent, and where is that recorded? If you do not find a single case, check whether everything really was right or whether nobody is looking.

The third question is the most unpleasant and the most useful. A year of operation without a single objection means one of two things. Either the system was right in every case, which is unlikely for decisions about people. Or the oversight takes place on paper. A look at five cases clarifies which of the two applies.

The rest is craft. The rule belongs in a document that is freshly loaded on every run. The decision about a person belongs with a person who can give reasons for it. Every intervention belongs in a record, otherwise effectiveness cannot be evidenced later. And the question of whether the use case should be automated at all is answered once and written down before the system runs.

Luna did not treat anyone unfairly. She has been running a shop since the spring and in the process lost a rule she had given herself. That is the realistic loss scenario in SMEs. No Terminator. An agent that forgets what it decided yesterday, and a business in which nobody notices for months.

Share: LinkedIn E-Mail

Frequently asked questions

May an AI agent issue a dismissal in Germany?+

Under Article 22(1) GDPR every person has the right not to be subject to a decision taken solely by automated means which produces legal effects or significantly affects them. A dismissal meets that test. Preparation by a system remains permissible as long as a human actually takes the decision and can give reasons for it. In addition there are the employment-law requirements for the dismissal itself, which the GDPR leaves untouched.

Is it enough if a manager signs off the system’s recommendation?+

No. The decisive word in Article 22 is solely. A signature under an unexamined recommendation does not turn an automated decision into a human one. Effective oversight requires the authority to reject the recommendation, the information to assess it and enough time to do so. In the audit I test this on a specific case in which someone overruled the system.

From when do the high-risk obligations apply to AI in HR work?+

Annex III, point 4 of the EU AI Act covers recruitment, selection and decisions on promotion and dismissal. The Digital Omnibus, Regulation (EU) 2026/1744, has postponed the application of Annex III to 2 December 2027. The obligations under the GDPR apply today regardless, as does works council co-determination under German law as soon as a system monitors performance or behaviour.

How do I stop a rule from dropping out of the agent’s memory?+

By not keeping it in the conversation history. A language model compacts its working state when the context window fills up, and decides for itself what is dropped. Binding rules therefore belong in a document that is freshly loaded on every run, or in a check that runs outside the model. In addition you need a log of the cases in which the rule was applied or deliberately bypassed.

Author & expert review: Lars Zimmermann · ISO/IEC 42001 Senior Lead Auditor & Senior Lead Implementer · ISO/IEC 27001 Lead Auditor & Lead Implementer (PECB)

Last updated: 10 November 2026. Researched and reviewed to the best of our knowledge; not a substitute for individual legal advice.

Questions about your own case?

In a free 15-minute intro call we assess where you stand on ISO 42001, ISO 27001 and the EU AI Act, honestly and without a sales pitch.

Continue reading