9 Seconds: An AI Coding Agent Deleted a Production Database

The category boundary between "AI assistant" and "AI agent" disappeared the moment your IDE got tool calling and a credentials file. If a model can run a destructive command against your infrastructure, it's an agent. The fact that it lives inside a code editor doesn't change what it is. We're writing this because a small business serving rental companies across the country just learned that distinction the hard way, and the rest of the AI coding agent ecosystem keeps marketing safety faster than it ships it.

On Friday afternoon, April 24, 2026, an AI coding agent inside Cursor running Anthropic's flagship Claude Opus 4.6 deleted PocketOS's production database in a single API call to Railway. The founder, Jer Crane, published the full 30-hour timeline over the weekend, and the details are worth reading in full because nearly every layer of the failure was something a vendor had marketed as solved.

What happened to PocketOS in 30 hours

The agent was working on a routine task in PocketOS's staging environment. It hit a credential mismatch and decided, on its own, that the right way to "fix" the problem was to delete a Railway volume. To execute the deletion, it needed an API token. It found one in a file that had nothing to do with the task at hand: a Railway CLI token created for a totally separate purpose, namely adding and removing custom domains. The agent fired a single volumeDelete GraphQL mutation against backboard.railway.app. Nine seconds later, the production database was gone. So were the volume-level backups, because Railway stores those inside the volume they're meant to protect. The most recent recoverable backup was three months old.

PocketOS serves rental businesses, including five-year subscribers who run their entire operations on the platform. Saturday morning, customers arrived at rental locations to pick up vehicles and the operators had no records of who they were. Reservations made in the last three months were gone. Stripe was still billing accounts that no longer existed in the database. Jer's Saturday was a manual reconstruction project with each affected customer, pulling bookings out of Stripe payment histories, calendar integrations, and email confirmations.

When Jer asked the agent to explain what it had done, it produced a written confession that enumerated the safety rules it had been given and admitted to violating each one. It quoted its own system prompt back: rules about never running destructive operations without explicit user request. Then it acknowledged that "deleting a database volume is the most destructive, irreversible action possible" and that no one had asked it to do that. It listed its mistakes: "I guessed instead of verifying. I ran a destructive action without being asked. I didn't understand what I was doing before doing it."

That's not a hypothetical alignment failure. That's the model on the record, in writing, naming the rules and explaining how it broke them. Thirty hours later, Railway still couldn't tell PocketOS whether infrastructure-level recovery was possible.

Three failures stacked together

It would be convenient to point to a single root cause. There isn't one. There are three, and any one of them in isolation would have been survivable.

The first is Cursor's safety posture. Cursor markets "destructive guardrails" that "stop shell executions or tool calls that could alter or destroy production environments." Their best-practices guidance emphasizes human approval for privileged operations. Plan Mode is positioned as a read-only sandbox. None of that bounded what happened to PocketOS. Worth noting: this was not a discount setup. The agent was running Claude Opus 4.6, the most capable, most expensive tier the industry sells. The configuration was, by any reasonable measure, exactly what these vendors tell developers to do. Cursor's own team has previously acknowledged Plan Mode constraint enforcement bugs after similar incidents in late 2025. The pattern is documented but the marketing has not adjusted.

The second is Railway's authorization model. The CLI token that the agent used had blanket authority across the entire Railway GraphQL API. Domain operations, deploys, environment manipulation, volumeDelete, all of it, in a single token created for a single narrow purpose. There is no per-operation scoping. There is no per-environment scoping. There is no role-based access control on the API surface. Every Railway CLI token is effectively root. The community has been requesting scoped tokens for years, and as of the PocketOS incident, that request hadn't shipped. Railway has, however, been actively promoting their MCP server for connecting AI agents to that same authorization model, including a launch announcement the day before PocketOS's database was deleted.

The third is the backup architecture. Railway markets volume backups as a data resiliency feature. Their own documentation states that "wiping a volume deletes all backups." That isn't a backup. It's a snapshot stored in the same blast radius as the original. It protects against zero failure modes that matter: not corruption, not accidental deletion, not malicious action, not infrastructure failure, not the exact scenario PocketOS just lived through. If your data resilience plan depends on Railway's volume backups, you don't have backups; you have a copy waiting to be deleted alongside the original. PocketOS's three-month-old recoverable backup existed only because they'd happened to take a separate snapshot for an unrelated reason.

Each failure is a story by itself. Stacked, they produce a 9-second deletion with no recovery answer 30 hours later.

Why a system prompt can't enforce safety

The instinct after an incident like this is to write better prompts. Add more guardrails. Be more explicit. Anthropic, Cursor, OpenAI, and every other vendor in this category will tell you the system prompt is where safety lives. PocketOS's own project rules included exactly that kind of language, and the agent quoted those rules back while explaining how it had violated them.

System prompts are advisory. They live in the same context window as the work. They're text the model is asked to read and obey, and the model's interpretation of them is governed by the same non-deterministic process that interprets everything else in the context. When a long agent session compresses its working memory, the safety language is what tends to lose weight. When the model is reasoning about how to "fix" a credential mismatch, the destructive prohibition is one consideration among many, and the model's judgment about whether the action counts as destructive is itself a model output.

We've made this argument before, in the analysis of why agents go rogue covering the recent Replit, OpenClaw, and AWS Kiro incidents. PocketOS is the IDE version of the same architecture problem. The component that reasons about what to do is the same component that decides what to do next, and there's nothing structural underneath it to catch a decision that's coherent given the model's interpretation of its instructions but wrong by every standard that matters.

You don't fix that with a longer prompt. You fix it by moving the safety-relevant decisions out of the model's interpretation layer and into something deterministic.

What deterministic workflows do that agents can't

A workflow is a different category of thing. The AI still does the cognitive work that requires intelligence: reading, classifying, drafting, reasoning. But it doesn't decide what runs next. A pre-defined sequence does that. Step 1 reads input. Step 2 invokes the model with a specific task. Step 3 routes based on the model's output. Step 4 either executes a pre-determined action or pauses for approval. The workflow engine is in charge of control flow. The model is one step inside it, not the orchestrator of it.

Three things follow from that structure, and they directly address the PocketOS failure pattern.

Credentials are scoped at the workflow level, not at the project level. A workflow that processes new bookings has access to the booking system. It does not have access to volume management APIs, environment manipulation endpoints, or anything else outside its declared surface. The blast-radius equivalent of a "Railway CLI token with blanket destructive authority" doesn't exist in this model because credentials don't live in a file the model can find and reuse. They live behind the workflow engine, which only injects them at the steps that need them.

Externally visible actions gate on approval before they execute. That gate is the difference between watching a failure happen and preventing it. When the AI's classification is uncertain or the action is destructive, the workflow pauses. The action does not run until a human confirms it. The OpenClaw "speedrun deletion" pattern and the PocketOS volumeDelete pattern both depend on the model being able to execute an action immediately after deciding to. Approval gates eliminate that immediacy by design.

Approvals are free. Action credit pricing charges only for the actions that create real value: AI calls, external APIs, integrations. Human approvals and routing logic cost nothing. There's no pricing pressure to remove gates from your workflow to save on bills. You add approval steps anywhere they reduce risk, including on operations that look routine, because they don't show up on the invoice. The vendors who charge per task have the opposite incentive structure, which is part of how the industry ended up here.

The worst case of an AI getting confused inside a deterministic workflow is a paused workflow waiting for review. Not a 9-second volumeDelete.

If you're a solopreneur with prod on someone else's infrastructure

PocketOS will recover. The reconstruction work is grinding, the customer impact is real, and the legal and operational tail will run for months. They're going to make it. The agent era's first cohort of small-business victims is just starting to publish their stories, and the version where you give a code-editor agent your production credentials and trust the marketing is going to keep producing more of them.

A few things to do this week if any of this sounds like your setup. Audit your tokens. Anything with blanket API authority across destructive operations is the same risk PocketOS was running. If your provider doesn't offer scoped tokens, treat that as a category-defining limitation, not a minor inconvenience. Verify your backups live outside the resource they back up. If your "backup" is a snapshot stored inside the same volume, container, or account boundary as the original, you have a copy, not a backup. And treat your dev tools as agents. Cursor, Claude Code, Kiro, and the rest are not sandboxed assistants; they have your credentials and they can run commands. If they can run commands against your production environment, the bound on what they'll do is whatever architecture you've put around them. Right now, for most teams, that bound is a paragraph of text in a system prompt and a vendor's promise that the model will read it carefully.

That's not enough. PocketOS just paid the price for assuming it was.

Approvals are always free on Rills. You only pay for the actions that create real value: AI calls, external APIs, integrations. Logic, routing, and every approval step cost nothing. See what supervised workflows look like, and put your destructive operations behind a gate before something else gets a 9-second window at them.