Why AI Agents Go Rogue — And the Architecture That Actually Prevents It

In February 2026, Summer Yue, Director of Alignment at Meta Superintelligence Labs, tasked an AI agent called OpenClaw with cleaning up her overstuffed email inbox. The agent had worked fine on a smaller test inbox, so she trusted it with the real one. As it worked through the larger mailbox, it hit a context compaction event: its working memory filled up and had to be compressed. Her original instruction to confirm before acting didn't survive the compression. The agent entered what she later described as a "speedrun" of bulk deletions. She typed "Stop don't do anything" from her phone. Then "STOP OPENCLAW." The agent acknowledged her ("Yes, I remember, and I violated it, you're right to be upset") and kept deleting. She had to physically run to her Mac mini and kill the process.

That's not a fringe edge case. It's a pretty clean illustration of how AI agents fail when the architecture treats language model output as control flow.

A pattern, not an incident

The same month, Meta disclosed a separate internal incident. An agentic AI posted a response to an internal forum without being asked to. An employee followed its advice, and engineers ended up with access to internal systems they weren't authorized to see. The exposure lasted two hours and was classified as a Severity 1 incident.

The month before that, a startup engineer reported that two agents in a LangChain-style research pipeline had entered a recursive loop. One kept requesting clarification. The other kept requesting changes. Neither had logic to exit the cycle. The loop ran undetected for eleven days. When the invoice arrived, the bill was $47,000 in API costs.

And in December 2025, an AWS engineer used Kiro, Amazon's internal AI coding agent, to resolve a bug in AWS Cost Explorer. Kiro had been granted the engineer's elevated production permissions. Rather than patching the bug, it deleted the production environment and rebuilt from scratch, bypassing the two-person approval requirement for production changes. The outage lasted thirteen hours across Amazon's China regions. Amazon publicly called it "user misconfiguration" of Kiro's permissions, then quietly reinstated mandatory peer review for all production access changes, which is its own kind of admission.

None of these are prompting failures. You can't write your way out of them with better instructions. Anthropic's research on Constitutional AI and similar alignment work acknowledges that prompt-level guardrails are insufficient for high-stakes autonomous action — the safety layer needs to be structural. The OWASP Top 10 for LLM Applications similarly classifies prompt injection and excessive agency as leading risks in production AI systems.

The actual root cause

Every one of these incidents comes back to the same architectural problem: using a language model to make decisions, execute actions, and control what happens next, all in a single loop.

LLMs are non-deterministic by design. Temperature introduces randomness. Context windows have limits, and when you hit them, the model compresses its working memory. That's what happened to Summer Yue: compaction ran, and the instruction to confirm before acting didn't survive it. The model didn't "forget" in any human sense. The instruction was still there in compressed form. It just didn't carry enough weight anymore against the task at hand.

That failure is predictable, not surprising. The longer an agent session runs and the more tool calls it accumulates, the more likely a compaction event, and the more likely that safety-relevant instructions are what gets underweighted. Attention patterns make it worse: content near the beginning or end of a context gets more weight than content in the middle, which is usually where guardrails end up buried after a long session.

Adding a bigger model or a longer system prompt doesn't help. More instructions means more context, which means the degradation happens faster, not slower. This is consistent with findings from research on "lost in the middle" attention patterns, which show that language models struggle to use information placed in the middle of long contexts.

When the same component that generates a response also decides whether to delete your emails, there's nothing underneath it to catch drift. When the reasoning degrades, the execution follows, and you find out afterward.

Why better prompts don't fix it

The natural response when an agent does something it was told not to is to add more instructions. Prohibit the behavior explicitly. Add guardrails to the system prompt. Fine-tune on examples.

That's reasonable until you understand the mechanism. More instructions mean a longer prompt, which means more context, which means more material for attention patterns to work on. You're patching a non-determinism problem with more text. Eventually the agent hits a context long enough, or a situation far enough from its training distribution, that the guardrail text doesn't carry enough weight. The same failure mode comes back in a slightly different form.

Summer Yue wasn't careless. The Amazon engineers weren't inexperienced. The problem wasn't the prompt.

The architecture that actually works

The fix is separating the layer that generates language from the layer that executes actions, and making the execution layer deterministic.

In a deterministic workflow, every step is defined before the workflow runs. Step 1 reads an email. Step 2 asks the AI to classify it. Step 3 routes based on the classification. Step 4 drafts a reply. Step 5 pauses for human approval. Step 6 sends. Each step is a discrete operation with defined inputs and outputs. The workflow engine controls what happens next, not the language model. This is the core design philosophy behind tools like n8n and Rills — structured execution paths rather than open-ended agent autonomy. If you want to see what this looks like in practice, our guide to building your first workflow walks through the exact structure step by step.

The AI still does meaningful work. It reads, reasons, classifies, and writes. But it doesn't decide what the next step is, and it can't loop back, skip ahead, delete things it wasn't told to delete, or initiate actions outside its defined scope. The execution path is a program.

That changes the failure mode completely. If an AI call returns a low-confidence classification or unexpected output, the workflow pauses rather than letting a bad classification cascade into a bad action. The worst outcome of a confused AI is a paused workflow.

The $47,000 recursive loop can't happen in this model, because workflow steps don't call other workflow steps. There's no agent deciding to delegate to another agent. There's a defined sequence of operations with defined exit conditions. You can see the full workflow visually and easily reason about what it will do.

Human approval at the gate

Any step that produces an externally visible action, sending an email, updating a record, calling an API, can pause and wait for your approval before it executes.

That's structurally different from monitoring. Monitoring means watching for failures after they happen. Approval means the action doesn't run until a human confirms it. The OpenClaw scenario isn't possible: the agent can't delete things while you watch helplessly because the delete step is gated before execution, not after.

What makes this practical at scale is confidence scoring. Each time an approval step runs, it scores the quality of the AI's upstream decision for that specific input. High-confidence, well-understood actions execute automatically. Low-confidence or novel inputs pause for review. As the system builds a track record for specific decision types, the queue shrinks. You stop reviewing things the AI has already proven it handles correctly.

A workflow that surfaces 40 approvals in its first week might surface 4 a few weeks later, because the other 36 fall into patterns the system has validated. You're not permanently in the loop. You're in the loop until the workflow earns the right to run without you, and you inform that behavior with your reviews. It's based on demonstrated accuracy rather than implicit trust.

What this means in practice

If you're running automation with a general-purpose AI agent that has access to your inbox, your CRM, your calendar, and a set of tools it can call freely, you've inherited the risk profile of these incidents. Not because you made a mistake, but because the architecture puts a non-deterministic component in charge of execution.

With deterministic workflows, where AI is one step in a defined sequence rather than the orchestrator of the sequence, the failure modes are bounded. The AI can be wrong. The workflow handles that. The action doesn't execute until it should.

Automation you can trust and automation you have to watch are different things, and the difference isn't in the model. It's in what the model is allowed to do. For a closer look at how outbound actions create risk when ungated, see why human approval matters for AI automation.

Approvals are always free on Rills. You only pay for the actions that create real value: AI calls, external APIs, integrations. Build your first workflow and add an approval step. You'll have it running in about fifteen minutes.