Skip to content

v1.25.0 ·

The wrong facts were too unimportant to read

The problem

The knowledge graph is the thing every agent reads before it reads any code. aidlc knowledge query costs one command; re-deriving the same fact costs a session. So the graph earns its keep by being right, and the failure mode nobody had a name for is an entry that is wrong in a way that looks exactly like being correct.

On 22 August the website cutover deleted the old site package, directory and all. The graph went on describing it for seventeen days. A query for the site package answered with the deleted one first:

$ aidlc knowledge query "web"
entities:
  - id: module-rasensio-aidlc-web
    description: Workspace package at packages/web/
    source: packages/web/package.json
$ ls packages/web
ls: No such file or directory

Eight module entities, plus seven dependency entities whose source named that package's manifest. Fifteen in a graph of two hundred and three.

The interesting half is not the cleanup. It is that two staleness signals already existed and neither could see this. byAge flags anything unverified for thirty days, which on a real graph is most of it — correct entries included — so it does not separate the wrong from the merely old. degradeChanged intersects each entity's source against git diff --name-only, which sounds exactly right until you notice that a path which no longer exists stops appearing in diffs at all. The change signal goes quiet precisely when the entry becomes wrong. Both signals were watching, and the shape of this defect was in the blind spot of each.

Two things the report that opened this got wrong, and both mattered.

It counted seven module entries. The population was fifteen: the seven dep-* entities sourced to the same deleted manifest are the same defect, and nobody had looked for them, because the report had named the symptom rather than the mechanism.

And it claimed the stale entries were compiled into CLAUDE.md and therefore fed to every agent's opening context. They were not. aidlc knowledge context ranks by confidence against a two-thousand-token budget, and every one of the fifteen carried confidence: 0.6 from auto-discovery. They lost the budget to the hand-written 0.9 entries and never reached a prompt.

That correction cuts both ways, which is why it is worth stating rather than quietly fixing. The ranking was working: it kept low-confidence facts out of the expensive place. It is also the reason nobody noticed for seventeen days. The wrong answers were not important enough to be read, and being unread is what let them survive — until somebody ran a query, where confidence buys no protection at all.

It also ruled out the obvious check. Anything that greps CLAUDE.md for the deleted package name finds a match today, and the match is true: the correct entry for @rasensio/aidlc-website truthfully ends "packages/web and vercel.json were deleted at cutover (T27)". A guard on the compiled text would have fired on an accurate sentence and stayed silent about the fifteen wrong ones.

How it could be solved

Deleting fifteen entries takes ten minutes. The whole question was what to build so that the sixteenth does not last seventeen days, and there were four candidates.

Do the cleanup and write the follow-up down. The cheapest option, and this project has measured what it is worth. A lesson recorded in prose is not a work item: the cost-tracking instance named its own fix exactly, and three days later a different instance rediscovered it from scratch and re-scoped it as new work. The release script fixed the day before this one had its remedy written down as a lesson ten days before it was needed, and the release died the same way twice more in between. So "note it and move on" is not a cheaper version of a check. It is the absence of one, with a paper trail that makes the next person feel worse.

A test in this repo's suite. Assert over this repository's own committed graph files that every source resolves. Genuinely good, and it is in the release — but on its own it protects exactly one repository. The graph is a framework feature; every project that runs aidlc discover accumulates the same class the moment it deletes a directory. A guard that only ever runs here would leave the shipped product with the same blind spot and this repo with a clean bill of health.

A confirm-tier doctor migration that removes the offending entries. The tempting one, because it ends with a green run and nothing to do. It is also the option the data rules out. Of the fifteen entities, twelve wanted deleting and three wanted their source corrected to a different manifest — marked, react and react-dom are still real dependencies, declared in packages/website/package.json at the exact versions each entry already recorded. Nothing in the data distinguishes those two cases. A confirm-tier migration applies unprompted under --yes, so the first unattended aidlc doctor --yes would have deleted three correct facts to tidy a path. Deleting knowledge is the loss this graph exists to prevent, and a repair that cannot tell a stale entry from a moved one has no business writing.

A report-tier migration. State the condition, name the entries, apply nothing under any tier or flag, and leave doctor's exit code alone. The precedent was already there: two of the four compliance-pack conditions and the roadmap projection's missing-gh check are all report tier for the same reason — the remedy is a decision, and doctor may not make it. This is that shape exactly.

Report tier won, and the honest cost is that it fixes nothing by itself. It converts a defect that was invisible into one line somebody has to read, which is the whole of the improvement. Against seventeen days of silence, one line is the difference; against a maintainer who never runs doctor, it is nothing. That was accepted rather than argued away.

Two smaller decisions came with it, both worth recording because either could have been made carelessly.

The check reports dangling relationships as well as unresolvable sources, from the same read. That was not scope creep looking for a home: dangling edges measured zero across all one hundred and ninety-five relationships before the fix, which made the second class the guard on the first. Deleting fifteen entities and forgetting their sixteen edges would have traded one defect for another, and the check that reported the first would have been silent about the second.

And a source that is absolute, or that escapes the project root once resolved, is classified as a defect in the entry and never handed to a filesystem call. A tracked YAML file is not a trusted one, and the alternative — resolve first, ask questions later — would have let a committed graph decide which paths aidlc doctor goes looking for on somebody else's machine.

How AIDLC solves it

A data fix to this repo's committed knowledge graph, and the check that would have caught it.

  • 12 entities removed, 3 re-sourced, 16 relationship edges removed, 3 added — all in .aidlc/knowledge/.
  • One new report-tier aidlc doctor migration, knowledge-stale-sources, which ships to every AIDLC project that runs aidlc update.
  • Two docs pages updated (below).
  • 49 new tests.

The PR itself bumped no version and published nothing; packages/cli/package.json was untouched by it. The owner then asked for a release, so this went out as v1.25.0 — minor rather than patch, because a new check appears in every project's aidlc doctor output. Report tier, so no project's exit code changes and no CI can newly fail; a project with no .aidlc/knowledge/ gets applicable: false and pays no parse.

The heading above was renamed from ## What is being deployed to make the release blog's shipped act resolve from source instead of needing a third override file. That is the lesson v1.24.1 recorded a day earlier — writing the headings the extraction chain already accepts costs nothing — and it is a rename, not an amendment: no criterion changed, none was retired, and the section says what it said.