By mid-afternoon we had shipped thirteen releases. The original misfile guards, the writer-identity transparency, the rollback runner, the email and mail flow integrations — all of it was in production and verified. The earlier blog post about closing the interface ambiguities described the work as done.
It was not done. The right move at this point was to stop shipping new features and audit the system we had just built.
This is a habit we are trying to keep. Each layer of safety produces its own assumptions. The misfile guards assume the binding is correct. The runner assumes the recorded result ID still refers to an unmodified record. The override mechanism assumes that "every override is logged" is a sufficient property of an override system. Each assumption is reasonable in isolation. Each one breaks when you look at it closely.
The audit produced fifteen items. Four were significant. This post walks through each.
Single-use override tokens
The override system from earlier in the day worked like this: when a guard refused a write, the attorney could mint a signed token bound to the specific operation. The token was HMAC-SHA256 of the operation's kind and parameters. Re-using the token on a different operation failed the signature check.
What the token did not do was expire after a single redemption. Once minted, it would work forever for the exact operation it was bound to. The audit log captured each redemption — but a token used a hundred times produced a hundred log lines that looked like a hundred deliberate decisions when they were actually one decision: the mint.
For a workers' compensation case management system, where every override should correspond to one specific attorney decision, unbounded reuse is the wrong semantics. The fix was a redemption record: ~/.aaicase/queue/redeemed-overrides.json stores the SHA-256 fingerprint of every redeemed token under a file lock. On the next attempt to redeem the same token, the guard refuses with "already redeemed" and surfaces the prior redemption's timestamp.
Fixing single-use exposed a second problem. The token's signature was deterministic — same operation parameters always produced the same token. With single-use enforcement, an attorney who legitimately needed to override the same operation twice (different decisions, same operation shape) couldn't, because the second mint produced the already-redeemed token.
The fix was a nonce. Each mint now generates twelve random bytes, includes them in the canonical signing form, and embeds them in the wire-format token as nonce.mac. The verifier splits on the period, recovers the nonce, recomputes the canonical form, and timing-safe-compares the MAC. Tampering with either segment fails verification because the canonical form includes the nonce.
The combined property: every token authorizes exactly one specific operation, exactly one time. Five verification cases, all passing — mint and use, refuse on reuse with prior-redemption details, fresh mint produces a different token for the same operation, tampered MAC rejected, tampered nonce rejected.
Pre-rollback state verification
The rollback runner kept a record of each step's result ID and used that ID to invoke undo if a later step failed. The implicit assumption: the record at that ID is still in the state we expect.
This is true most of the time, on the scale of milliseconds between forward and rollback. It is not always true. Another process — the MerusCase web UI, a parallel binding extraction running in another terminal, an audit-case sweep on the same firm — might have modified or deleted the record between when we created it and when we tried to undo it. Blind rollback against the recorded ID would still execute, but it might delete a record that someone else had already modified into something the attorney cared about, or attempt to undo a write that had already been undone by the cascade of another operation.
The fix added an optional verify hook to each step in the runner. Before invoking undo, if the step has a verify function, the runner calls it with the recorded result. The verify checks whatever the step author thinks is important — does the record still exist, is it still un-deleted, is it still on the case we put it on. If verify returns a failure or throws, the undo is skipped and the record is surfaced in a new pre_rollback_verify_failed array in the result.
The aai-sequence CLI gained three built-in verify checks that callers can specify in the spec without writing custom code: exists, deleted_false, and case_file_id_matches:NNN. Each maps to a re-fetch of the record and a check on the relevant field. Misses on the verify check skip the undo cleanly rather than running it blindly.
The runner's failure mode now matches what an attorney would actually want. If the rollback can't safely proceed because the record drifted, the runner declines to run a destructive operation and tells the attorney exactly which record needs manual inspection.
The PDF hash check finally reaching production
Back in 4.5.15, when we added multi-source applicant verification to the binding, we also added a verifyPdfHash function that compared a PDF's SHA-256 to what the binding recorded. The function was tested in isolation. It worked correctly. It was a library function with no production caller.
This was exactly the same pattern the runner had before we wired it into the mail and email flows. Library exists; production doesn't call it. The audit caught it because — having just done that wiring for the runner — we knew to look for similar gaps.
The fix: before any /uploads/edit write that includes a case_file_id, if the bound upload has a recorded hash, the guard now downloads the PDF from /documents/download/{upload_id}, hashes the bytes, and compares to the binding's stored hash. Mismatch refuses with "PDF content drift."
The cost is one HTTP GET and one SHA-256 of the bytes per filing. A few hundred milliseconds. We considered cheaper alternatives — passing the local PDF path via an environment variable, storing the path in the binding — but both depend on the local file still existing at write time and being the same bytes the binding extracted from. Re-downloading is the only check that compares against what MerusCase actually has, which is the bytes that will be filed.
The check catches: a PDF replaced in MerusCase after the binding was extracted (someone else uploaded a new version), a binding pointing at the wrong upload, server-side corruption, manual editing of the file on the server. None of these would have been caught by the applicant-mismatch guard alone, because the wrong PDF might still belong to the right applicant.
Back-compat: bindings without a recorded hash skip the check, so older bindings still work. Soft-fail on download error: if the PDF can't be fetched for verification (a transient network blip), the guard prints a note and proceeds without the hash check. We don't want a flaky network to block a legitimate filing.
Three verification cases — fake hash refused with "PDF content drift," null hash skipped, real hash passes. All live against MerusCase.
The static analyzer for the rule we already had
Back in 4.5.11 we added a hard rule to the system prompt: no regex for content interpretation. Tag selection, sender role classification, "what kind of document is this" — none of these could be done with regex pattern matching. They had to go through a Claude subprocess that read the catalog and picked structured values.
The rule existed as documentation. Nothing enforced it. A future release that introduced if (/qme/i.test(subject)) tag = 'qme' would not produce any error at all. The misclassification would only become visible when a customer filed an email and got the wrong tag.
The fix is a small static analyzer that runs as part of npm run check. It scans the code for regex patterns applied to a curated list of content-bearing variable names — subject, body, sender, applicant, doc_type, activity, and so on — via decision verbs like test, match, and search. Transformation verbs like replace are allowed, because they're format operations (stripping HTML, normalizing whitespace), not content decisions.
The analyzer is honest about its own limits. It's pattern-based, not type-checked. A regex applied to an aliased variable name will slip past it. Exemption comments (// aai-allow-content-regex) let specific lines bypass the check for cases where the regex really is a format check on a content-named variable.
Initial development surfaced four false positives — HTML stripping, base64 decoding, whitespace normalization — all of which were correctly format transformations on text-named variables. The pattern was refined to only flag decision verbs, and the code is now clean. Two deliberate violations confirmed the check fires correctly, and an exemption-commented violation confirmed the suppression mechanism works.
The check runs on every release going forward. A future commit that regresses the no-content-regex rule will fail the build.
What this round teaches us
Every layer of safety produces a new class of weakness. The misfile guards produced the override system. The override system produced the unbounded-reuse problem. The unbounded-reuse fix produced the deterministic-token problem. The rollback runner produced the unverified-state problem. The PDF hash library produced the production-call-site gap. The no-content-regex rule produced the no-enforcement problem.
None of these are signs that the safety architecture is failing. They're signs that the architecture is real enough to interact with itself. A toy safety system has no weaknesses because there's nothing to break. A real one accumulates weaknesses at the interfaces between its parts, and those weaknesses are visible only after you build the parts and try to use them together.
The discipline we are keeping: every fix verified before publish, every release recorded in the changelog with what was verified and how, every audit cycle producing a new backlog. The system grows in concrete properties. The blog posts describe what the system actually does, not what we hope it will do.
Four fixes from one audit cycle. Eighteen releases shipped in a single day. Each release is a small piece of an architecture that is becoming, slowly, the right shape for an AI tool operating inside a law firm.
What's still open
The same audit produced eleven other items we did not address this round. Most are smaller polish issues — audit log rotation, performance during parallel binding extraction, output messages in the manual undo tool. The audit log of audits, so to speak, is growing too. Each item gets its own ticket and waits for the next cycle.
The two open commitments outside the audit remain the same. The Windows compatibility script still hasn't been run against an actual Windows machine. The model's adherence to the new rules will be visible only in actual attorney usage over the coming days, and any new failure modes will go into the next audit pass.
The work continues at the pace of one verified release at a time.