The system prompt for this tool is assembled at startup from two sources: a template file (system-prompt.md) and a knowledge file (~/.aaicase/knowledge.json). The template contains placeholders like {{SKILLS_DIR}} and {{FETCH_HELPER}} that get replaced with absolute paths. The knowledge file contains structured information about the firm — its name, its type (applicant vs. defense), and the list of staff members with their initials, emails, and roles.

The assembled prompt gets handed to the model on every query. The model reads it as system instructions. It expects clean text — sentences about the firm and the staff, formatted lists of staff with their roles and contact info, the kind of structured context an attorney would give a junior associate on day one.

The probe this week was to feed the assembly layer some knowledge.json shapes that aren't quite right and see what comes out. The cases I tried were all things that could happen in practice: a staff entry hand-edited to remove a field by mistake, an old schema where one of the now-required fields didn't exist, an export from an older firm-import tool that was missing some columns.

What went wrong

Three field omissions produced three different leaks of the literal string undefined into the prompt.

A staff entry missing the name field produced:

- undefined (SA) — user_id [USER-ID] — Managing Attorney — attorney@aai.dev

The initials map appended NH=undefined. The model, reading this, would parse "undefined" as a person's name. If the attorney later asked "who is NH?" the model would answer "NH is undefined" or, worse, "per undefined's notes on this case..."

Missing initials produced Sample Managing Attorney (undefined) and undefined=Sample Managing Attorney in the initials map. Missing email produced ... — undefined where the email should be.

These are all the same kind of bug: template-string interpolation that doesn't defend against missing fields. ${s.name} evaluates to the literal string "undefined" when s.name is undefined. JavaScript will happily render that into the result string. Nothing fails, no error is thrown, the prompt assembles successfully. The defect is downstream — the model receives a prompt with "undefined" in it and may reproduce it.

What I'm not arguing

This isn't a critical bug. A misconfigured knowledge.json is itself a problem; the fact that the prompt then has "undefined" in it is downstream of that. An attorney whose knowledge.json was correctly populated wouldn't see any of this. The fix is more about defense in depth than about a vulnerability someone is exploiting.

But: the threat model for this tool isn't external attackers. It's misconfigurations from migration, hand-editing, schema drift, partial setup completion. Each of those is the kind of thing that happens once per firm during onboarding and then never again. A defensive layer that catches them when they happen — and produces a sensible prompt anyway instead of a polluted one — is the right level of rigor for an operations tool.

The fix

Four small accessor functions in src/prompt.mjs. Each one returns the field value if present, or a sensible fallback if missing.

const safeName = s => s.name || s.full_name || s.initials || '(unknown)';
const safeInitials = s => s.initials || (s.name ? s.name.split(/\s+/)
  .map(w => w[0] || '').join('').toUpperCase().slice(0, 3) : '??');
const safeEmail = s => s.email || '(no email)';
const safeRole = s => s.role || 'Staff';

The interesting one is safeInitials. When initials are missing but the name is present, it auto-derives initials from the name's first-letter-of-each-word. So a staff entry with name "Sample Attorney" and no initials field still produces "SA" in the prompt, the same as if the initials had been there. That's better than falling back to "??" because the auto-derived value is usually the right answer.

The fallbacks are visible to the model — it sees "(no email)" instead of "undefined" — which is honest about what the prompt knows. The model can recognize "(no email)" as a placeholder and refrain from generating output that pretends an email exists.

The regression detector

Beyond the fix itself, this release adds a regression detector to aaicase --debug-prompt. After the prompt is assembled, the command scans the output for two patterns: literal "undefined" anywhere, and unresolved {{X}} placeholders. If either is found, a warning prints alongside the success message.

The detector doesn't fail the prompt generation — the prompt still writes. But the operator sees that something's off and can investigate. For a tool whose primary verification is "the prompt assembled cleanly," having a warning that surfaces leftover template artifacts is a useful regression catch.

The detector also surfaced a small piece of dead code: there's a substitution for BASE_URL in src/prompt.mjs that's no longer referenced in system-prompt.md. The substitution is a no-op at runtime (replace doesn't find any match), but it's an inconsistency between the two files. Left in for now — a future template might use BASE_URL again, and removing the substitution would just put it back when the time came.

Where the probe pattern keeps landing

This is the second post in the "probe the misconfiguration path" thread. The first was the sequence runner validation (4.5.83): empty steps, missing label, wrong-type args. Same shape of bug: an input that's mostly right but quietly wrong produces output that looks legitimate.

The lesson keeps repeating. Verification against well-formed input doesn't catch the cases where input is malformed in subtle ways. The bugs are the kind nobody would write deliberately — they only show up when a config file is hand-edited or migrated or partially set up. But that's most of the real world. The probe is doing what code review can't, because code review reads the happy path.

The version on the install URL is 4.5.84. The work continues at the pace of one verified release at a time, and the prompt-assembly layer is now defended against the small kinds of corruption that knowledge.json files tend to develop.