The case names, parties, applicants, filenames, and specific details in this post are fictitious demonstration data shown to illustrate the bug pattern. No real client information is shown.

On a Friday afternoon a paralegal noticed two cases were tangled. The activity log on Doe, A v. Acme Logistics described a defense panel strike letter from opposing counsel. The activity log on Roe, B v. Sample Equipment Co. described a subpoena noncompliance notice from a records copy service. Both descriptions were correct for the case they were on. But the actual PDF attached to each activity belonged to the other case.

Two documents. Two cases. Wires crossed. The filenames matched the activity descriptions — Doe's activity had a file named 2026-05-08-panel-strike-doe.pdf. Open the PDF and every page referenced Roe. Every applicant name. Every claim number. Every signature block.

This is the kind of bug that sits silently inside a case file forever. Nobody opens an old upload to verify the PDF still matches the activity that describes it. The wrong document is filed, the right deadline is calculated from the wrong evidence, and the case proceeds on a fiction.

The system that produced this bug was our own. We had built a workers' compensation case-management assistant that read incoming mail, identified the document, found the matching case, proposed actions to file the upload, and executed them on attorney approval. The system had a written rule in its instructions: read the PDF, extract the applicant name, match to the case before proposing actions. The rule was there. The rule was clear. The rule did not prevent the misfile.

Why the rule did not work

When a language model processes a batch of items — say, fifteen unprocessed uploads in the mail queue — its working memory holds context from all of them simultaneously. It read upload A, extracted "ROE" as the applicant. It read upload B, extracted "DOE" as the applicant. It searched for both cases. It proposed actions. And somewhere between extraction and action, it crossed the wires: it proposed renaming upload A with a Doe filename and assigning upload A to Doe's case.

The instruction "match the PDF content to the case" was satisfied at extraction time. Both extractions were correct. The cross happened later, when the model translated extracted facts into API calls. From the model's perspective, no rule had been broken — it had read both documents, it knew both applicants, it filed both uploads. The fact that upload_id and applicant_name had become disconnected by the time the write was issued was invisible to the rule.

You can write more rules. "Re-state the binding before writing." "Cite the upload_id alongside the extracted name." "Verify each upload_id against its read." These help. They also remain rules — instructions to a system that interprets instructions probabilistically. The system can follow the rule 99 times and miss the hundredth.

This is a structural limit of prompt-based safety. A rule that exists only as text in an instruction is a request to a probabilistic process. For low-stakes tasks the request is enough. For tasks that produce permanent record-keeping changes in a law firm's case management system, it is not.

Where the safety actually has to live

The architectural answer is to move enforcement out of instructions and into code. Code refuses. Code does not have a 99% adherence rate to its own logic — when the condition is false, the operation does not happen.

We rebuilt the mail processing pipeline around four code-level guards.

1. Extraction by isolated subprocess

When a PDF arrives for processing, a new program runs: aai-bind. It downloads the PDF, spawns a fresh Claude Code subprocess with one job — read the PDF, identify the applicant, pick the right Merus activity tag from the firm's catalog — and writes the result to disk. The subprocess has no access to the parent system's working memory, no other context from other uploads, no way to confuse this PDF with another one. It is given exactly one document and one task. It returns structured JSON: applicant last name, applicant first name, claim number, document date, suggested activity tag, certainty.

The result is stored at ~/.aaicase/queue/bindings.json, keyed by upload_id. The binding is the authoritative statement of what the PDF contains. The main system reads from this binding rather than re-deriving the applicant from memory.

If the subprocess fails — corrupt PDF, ambiguous applicant, network error — there is no binding. Without a binding, downstream operations are blocked.

2. Write-time refusal

Every Merus API write that touches an upload-to-case relationship now passes through a guard. The guard sits inside merus-fetch, the helper script that issues HTTP requests to the MerusCase REST API. When the helper is called with /uploads/edit/{id} and a case_file_id= argument, it does this:

  1. Look up the binding for that upload_id. If no binding exists, refuse the write.
  2. Fetch the target case from Merus and read the applicant's name.
  3. Compare the case applicant's last name to the binding's applicant last name.
  4. If they don't match, refuse the write.

The same guard checks filenames. If description= is in the arguments, the filename must start with YYYY-MM-DD- (a format check) and must contain the bound applicant's last name (an integrity check). A filename like 2026-05-08-panel-strike-doe.pdf attempting to be applied to an upload bound to Roe is refused.

The same guard runs on /activities/add when both upload_id and case_file_id are present, and again separately when activity_type_id values are proposed. If the proposed tags don't include the tag Claude picked during binding, with high certainty, the write is refused.

Refusal means exit code 3 and a printed error explaining what was expected and what was provided. The model sees this output and knows the write did not happen. There is no possibility of silent corruption — either the data shape is consistent or no data was written.

3. Override is logged

The guard accepts an override environment variable, AAI_FORCE_BINDING_OVERRIDE=1. Setting it allows the write to proceed despite a binding mismatch. We did not make the system unbreakable; we made the break visible.

When override is used, the bypass is logged to stderr in large red letters and to a permanent audit log at ~/.aaicase/audit.log. Every override leaves a trail. The override exists because there are legitimate edge cases — a misfiled historical document being intentionally moved to its correct home, a case where the binding extracted a co-defendant's name instead of the primary applicant. The override is a tool for the attorney; it is not a tool for the model. The model has no instruction to use it, and the override path is conspicuously different from the normal path.

4. Read-after-write

After a successful write, the system does not trust the API response. It re-fetches the upload from Merus, confirms the case_file_id and description match the proposed values, and runs the binding extractor a second time against the now-filed PDF. If the re-extracted applicant doesn't match the case applicant, processing stops. This catches the theoretical case where Merus returns success but persisted different fields than were sent.

The end of regex for content interpretation

There was a second class of bugs that had nothing to do with upload swaps but everything to do with the same root cause: code interpreting content. The old mail and email skills used regular expressions to decide what an email or document was. If the sender's domain matched /scif|sedgwick|gallagher/i, it was tagged as adjuster correspondence. If the subject contained QME, it got a QME tag.

This worked for the common case and silently failed for the uncommon one. A QME report forwarded from a paralegal's personal email did not match the QME domain pattern. A defense letter with an unusual subject line got tagged as general correspondence. The regex was a model of the document space that was always slightly wrong, and the cases where it was wrong were exactly the cases where the wrong tag mattered most.

We replaced every content-interpretation regex with a Claude subprocess. Email tag selection now runs aai-bind-message, which receives the email metadata, the firm's activity-type catalog, and the firm's user directory, then picks the appropriate tag IDs directly. Activity description classification (does this note describe client contact?) and party classification (is this contact a doctor?) now run through aai-analyze, which performs the same kind of structured extraction on text records.

The cost is real. Regex is instantaneous; a Claude subprocess takes 5-15 seconds. Case value queries that used to return in 3 seconds now take 25-30. We accepted this tradeoff because the alternative — fast, slightly-wrong classification on documents that legal deadlines depend on — is not actually faster. It just defers the cost to the moment when the wrong classification matters.

The skills that came out of it

Four new commands shipped alongside the architecture.

process mail (rewritten)

The old version ran a background prefetcher that downloaded uploads in parallel and used sub-agents to read long PDFs in chunks. The parallelism was where wires crossed. The new version is strictly serial: one upload at a time, downloaded just-in-time, bound just-in-time, presented to the attorney just-in-time. The attorney sees a BINDING CHECK header at the top of every proposal — upload_id, local PDF path, applicant extracted from PDF read, case match, filename plan — before any action is proposed. If anything in the binding doesn't line up with the case match, the system refuses to propose actions and asks the attorney what to do.

fix misfile

A dedicated skill for resolving the kind of bug that started this story. The attorney names one or two upload_ids and says "these are on the wrong case" or "these are swapped." The skill re-runs the binding extractor on each upload (no cache — always fresh), shows the actual PDF content versus the current filing, and proposes a reassignment. The reassignment goes through the same guard, so even the fix-skill cannot misfile.

audit case

An entire case can now be swept. The skill takes a case name, fetches every upload linked to that case via activities, and runs the binding extractor on every PDF. It compares each upload's actual content to the case applicant, the activity tag, and the filename. Findings are displayed in a table — critical (wrong case), warnings (wrong filename or wrong tag, right case), clear (everything matches). The attorney reviews and approves corrections one by one. No cache, no shortcuts. The full sweep on a 50-upload case costs a few dollars in API spend and takes ten minutes; it is honest work that surfaces problems that would otherwise sit dormant.

process messages (rewritten)

Email processing now uses aai-bind-message for tag selection. The old version's regex-based sender classifier is gone. The same merus-fetch guard refuses email-to-case writes that don't match the binding.

What changed in the system prompt

The instruction file the model reads at startup grew several hard rules. They are not the safety — the code is the safety — but they describe the architecture to the model so it knows the shape of the system it is operating inside.

The model now operates inside a fence that limits what it can do, not just what it should do. Inside the fence the model is helpful and fast. At the fence the model cannot act. That is the property we wanted.

What this costs and what it buys

Every upload now incurs an additional Claude subprocess call for binding extraction. Every email incurs a subprocess call for classification. Every case-value query incurs three subprocess calls for analytics. The cost runs $0.05-0.15 per upload, roughly $2-4 for an audit on a fifty-upload case.

What it buys is a system whose guarantees can be stated in the active voice: aaicase cannot file an upload to a case whose applicant does not match the PDF content. It cannot rename an upload to a filename that does not match the bound applicant. It cannot tag an activity with a tag that contradicts the Claude classifier's high-confidence pick. It cannot misfile mail in the way that started this incident.

None of these are guaranteed by any individual sentence in the model's instructions. They are guaranteed by the helper script that refuses to make the HTTP request. They survive prompt drift, context window changes, model upgrades, and the kind of probabilistic accidents that prompt-only safety cannot prevent.

What this does not solve

We did not eliminate the possibility of bad input. If the PDF itself is mislabeled — Acme Industries' name printed on a Doe file by a sloppy fax operator — the binding will reflect what's printed, not what's true. The guard cannot detect that. We did not eliminate the possibility of override misuse — the override exists, and an attorney with the variable set can still write through. We did not eliminate the possibility of bugs in the guard code itself. The lib/binding.mjs module contains real Javascript, and real Javascript has bugs.

What we did eliminate is one specific shape of error: the model's working memory crossing wires between uploads it is currently processing. That shape of error is structurally impossible now because the model is not the source of truth about what is in upload N — the binding file is. And the write path consults the binding file, not the model's current understanding.

The wider lesson is that prompts and code do different things. Prompts shape the system's intent. Code enforces the system's invariants. A safety property that exists only in a prompt is a request; a safety property encoded in code is a guarantee. For workers' compensation case management, where misfiles harm clients and produce malpractice exposure, the guarantees have to live in code.

Why we are writing about it

The story above describes a bug in our own system and the architecture we built to prevent its recurrence. We chose to write about it for two reasons. First, the incident is the kind of thing AI tools in legal contexts will keep producing, and explaining how it happens makes the failure mode legible to attorneys who use AI in their practice. Second, the response — moving safety from prompt to code — is a pattern that generalizes far beyond this one tool. Any AI system that takes actions in the real world is going to need write-time guards, structured extraction, and code-level refusal. The shape of the architecture is more durable than any particular implementation.

The system shipped with these guards in versions 4.5.10 and 4.5.11. The misfile that started this story has been corrected. The audit-case skill has been run on every active case in the firm where the incident occurred. We expect to find more historical misfiles as we audit older cases; the architecture cannot retroactively prevent past bugs, only future ones.

The work continues.