Skip to content

v1.17.0 ·

An unassessed control is never a pass

The problem

Three problems stack on one cause: this framework already produces audit evidence, and has no idea that it does.

Look at what a completed piece of work leaves behind. Numbered acceptance criteria. A design. A task breakdown that names, for each task, which criteria it satisfies. Review findings, each with a severity and a status saying whether it was resolved. A log of every phase transition, with timestamps. A record of what was deployed and how to roll it back. An auditor would recognise that immediately: it is a change-management trail, and a better one than most teams keep by hand.

But nothing in it names a control.

So when somebody asks "show me your change-management control, and the evidence that it was applied," what they get is a directory of markdown files and a person explaining them from memory. The problem is not that the evidence is missing. The evidence is unusually good. The problem is that no artifact maps any of it onto a control identifier, which means it cannot be produced on demand, and cannot be shown to be complete. Evidence you have to narrate is not evidence you can hand over.

The second problem is that the obvious fix looks finished before it starts.

This framework already has a place to put rules that apply while code is being written — a set of guidance layers that phases are told to consult. It was built anticipating exactly this use: the comment explaining why layers are loaded from one place names compliance packs, in those words, as the reason. Installing a bundled layer into a project already works, and already merges into the layer index without clobbering what is there. Shipping a security-and-availability regime as one markdown file would take an afternoon.

It would also be indistinguishable from a style guide. Prose an agent is told to apply, with no control identifiers, no stated failure condition for any individual control, and nothing produced afterwards that says which controls were actually checked. Applied on best effort, unverifiable after the fact. That is the version that is worse than shipping nothing at all, because it invites somebody to believe something has been established when it has not — and the person it misleads is the buyer who is relying on it.

The third problem is that a pack, once installed, could not be taken out again.

There was no uninstall. Not a partial one, not an awkward one — searching the whole install area for anything resembling removal returned nothing. And the existing install path makes removal harder rather than easier, for a good reason: if it finds that a project has edited an installed layer, it leaves the edited file alone. That is the right instinct. It also means whatever removes a pack has to tell the difference between "this is our file, safe to delete" and "this is the project's file now, and deleting it destroys their work."

The request was for add and remove, and the ordering in that sentence is misleading about the difficulty. Add is the easy half. Remove is where the design is. And without it there is no product, because nobody trials a compliance pack they cannot back out of.

How it could be solved

Option A — Guidance layer only

A pack is one extra markdown file plus aidlc compliance add|remove <regime>. No control ids, no review dimensions, no report.

Cheapest by a wide margin — most of it already works. Also the version described above as worse than nothing: it is a text file, so a project could write it themselves in an hour, there is nothing to sell, and nothing afterwards establishes that any control was applied. Rejected as a product, but worth naming because it is the fallback if the structured version proves too large.

Option B — Guidance plus controls, review dimensions, and an evidence report (recommended)

A pack directory carries four separable things: constraints (imperative, testable, implementation-time rules, installed as a guidance layer through the existing seam); controls (one entry per control id naming the phase, where to look, the fail condition, and severity); evidence requirements (which lifecycle artifact records which control decision); and pack metadata (id, regime, version, industries, entitlement).

The review skill gains one dimension per control applicable to the phase under review, with findings tagged by control id and flowing through the review-gate machinery that already exists. A fourth gate-criterion module reads them. aidlc compliance report renders the evidence trail keyed by control id from state that is already on disk, and a diff-level check reads the same controls data so a change can be assessed without running a phase.

Costs: real design work, one new command group, a new gate criterion, an uninstall path that has to reason about locally edited files, and the SOC 2 content itself. The distinguishing bet is that the report is generated from existing artifacts — if that turns out to require new developer bookkeeping, Option B collapses toward Option A and we should notice early rather than at implementation.

Option C — Option B plus registry, licence keys, signed packs, and an industry catalog

Rejected for this instance. The roadmap item explicitly says do not attempt the catalog, and a licence server is weeks of work before any pack content has been validated by anyone. Option B's resolver indirection is what keeps C from being a rewrite later.

How AIDLC solves it

compliance-packs

A SOC 2 compliance pack and the machinery to install, remove, upgrade, check, gate and report on it. Opt-in per project: with no pack installed, nothing changes.

New module packages/cli/src/compliance/:

  • types.ts, schema.ts — pack shape and whole-pack validation.
  • resolver.ts — the only reader of pack sources; bundled-only, exact-match slug.
  • config.ts — the compliance: config section, per-file hashes, the recorded-hash modified detector.
  • install.ts — installPack / removePack / upgradePack / packStatuses.
  • findings.ts — the <artifact>.compliance.md stream and its two productions.
  • gate-criterion.ts — evaluateComplianceCriterion, wired at both gate sites.
  • check.ts — checkDiff, where an unassessed control is never a pass.
  • report.ts — the read-only evidence report.

New command group commands/compliance.ts (list, add, remove, upgrade, check, report).

Extracted seams, reused rather than duplicated:

  • compile/layer-install.ts — the install primitive, with an injected modified detector; guidance-install.ts became a thin caller.
  • review/findings-parser.ts — one findings grammar, two callers.
  • core/paths.ts — toPosixRelative, one definition.
  • doctor/types.ts — a report migration tier.

Content: packages/content/packs/soc2/. Four doctor migrations. An init wizard question. The review, deployment and maintenance skills gained the index-reading instruction they lacked.

agent-portability

Half A — Codex skill parity

Codex now gets what Claude Code and Kiro get. codexAdapter.plan emits one .agents/skills/aidlc-<name>/SKILL.md per lifecycle skill, carrying the body rather than a pointer to it, so Codex selects a skill by matching its description — the activation that was previously absent. AGENTS.md keeps every section it had and its per-phase list now names installed skills instead of files to open by hand.

Only SKILL.md is emitted. Codex skills may also carry scripts/, assets/ and agents/openai.yaml, and implicit activation is on by default, so a committed skill directory is an auto-loading instruction surface for anyone who clones the repo. The emitted path set is asserted so an executable sibling cannot be added quietly.

Measured, not assumed: 2,583 of 5,168 preview characters used across 17 skills, 2,585 headroom. The budget is min(2% × context_window, 8000) and the observed window is 258,400 — the 8,000 figure the first requirements draft used as "Codex's floor" was the unknown-window fallback, roughly twice the real number.

formatCanonicalSkill is now the single canonical formatter for all five adapters. This fixed a live defect rather than tidying an inconsistency: claude-code wrote skill.body raw with no frontmatter while codex wrote frontmatter to the same .aidlc/skills/<name>.md path, so on a two-platform project the canonical copy depended on adapter iteration order. Field order is pinned to codex's, so AC-22 holds and kiro's output changes.

Platform-visible files were left alone, deliberately. Cursor's validator requires description and alwaysApply and forbids name; claude-code's requires name to equal the directory. One shared shim format breaks one of them.

Half B — one platform per instance

cost/pin.ts derives ownership from sessions.ndjson and writes nothing. The write path — resolveAgent, resolveSessionIdentity — is untouched, and a test asserts it. That separation is the design: feeding the pin into the write path would make the log self-confirming, and sync.ts:197 resolves a provider from the recorded agent, so a Codex session on a claude-code-pinned instance would have been priced at Claude rates.

Three verdicts, and unpinned is deliberately not skipped: nobody-could-tell reports as unattributed exactly as before, while two-platforms reports as unmeasured with both platforms named.

The unmeasured rule reaches all five cost readers from one seam — instanceCost consults the pin — rather than five printers. Patching printers alone would have left aidlc metrics --json and published blog frontmatter carrying a figure the CLI describes as unmeasured. Alongside that: true_cost_micro_usd nulls when any constituent is unmeasured (it previously skipped a null-cost bugfix and still reported a non-null parent, which can only understate); --by-template averages over measured members and prints the count rather than coercing null to zero; and aidlc metrics prints true cost unconditionally, since gating it on caused_bugfixes.length > 0 hid every null.

aidlc cost also names the owning platform, which is what makes the pin observable through a shipped surface at all.

Half C — Codex cost provider, pricing, sync

The provider honours CODEX_HOME — real, read by codex-rs/rollout/src/config.rs — because hardcoding ~/.codex would give anyone who moves it detect() === false and silent zero capture.

It reads payload.info.last_token_usage and never the cumulative block, and it de-overlaps the counters: cached_input_tokens is inside input_tokens and reasoning_output_tokens is inside output_tokens, the inverse of Anthropic's disjoint shape that the claude-code provider maps straight through. A straight mapping would price cached input twice — 206,483 input-side tokens against a true 122,167 on one observed turn.

The cursor is <rollout-basename>#<ordinal>. Codex records carry no id at all — verified across both real rollouts, a token_count line has only payload, timestamp, type — so there was nothing to resume on. Rollouts are append-only, which makes the ordinal stable, and the cursor names the record rather than the session, so the duplicate check is session-agnostic.

sync resolves Codex by session id (the uuid is in the filename, the only id the format offers) and by usage-record timestamp, with per-segment repository scoping by git common dir and the no-git-identity path fallback that makes worktrees count. That scoping is not hypothetical: the rollout sitting in the sessions root on this machine carries cwd: /Users/rasensio/projects/SkyForce.

sync also uses the pin to narrow an unknown-agent session to one provider instead of widening detectedProviders — the widening that once left two instances with no cost data at all.

Six literal platform-name branches became one hook registry, including the read site that gave Kiro's hook status to any provider that was not claude-code. The registry is a module-level literal, deliberately independent of the provider registry, because allProviders() is empty until registerBuiltinProviders() runs and three doctor migrations kept static tables for exactly that reason.

project-constitution

A new capability and no change to any existing one.

  • aidlc constitution import <path> — read a vision document (.docx, .md, .markdown, .txt) into .aidlc/context/vision-source.md.
  • aidlc constitution check — report which vision statements moved since that import, and which constitution entries cite one that is gone.
  • aidlc-constitution skill — derives .aidlc/context/constitution.md (mandate, in scope, prohibitions, amendments) from the imported statements, and proposes amendments into .aidlc/inbox/ when the vision is revised.
  • The consultation rule in aidlc-overview, with pointer lines in the requirements, design and roadmap skills.