← Projects

Projects · specification

No pass without a measurement: AI Architect Spec, traced through its source

Clément Deust · September 2026 · verified against ai-architect-mcp-spec at 7d8fa1f

A specification that reads well and describes a system nobody can build is worse than no specification. This server is built to catch that case before it reaches code: it checks that every symbol exists, that every acceptance criterion traces back to a requirement, and that no performance claim passes without a measurement — deterministically, with no model in the checking loop.13 It is a reducer rather than an agent: it emits actions and your host executes them, and it never phones home for a verdict.2

Runsone Node process on standard input and output, exposing 17 tools under three profiles67
Shapea stateless reducer: state and result in, next state and one action out2
Pipeline20 steps, of which 11 produce the specification and 9 are opt-in behind a human gate9
Checks73 deterministic rule functions with no model in the loop, and a five-verdict taxonomy above them1416
Refusesa report where everything passes, and any performance claim marked PASS16
Keepsnothing across a restart: run state is in memory only11
Exhibit 01 · the shape

A reducer that emits actions, and calls no model itself

The interesting architectural choice is what this server refuses to do. It does not draft your sections and it does not judge them. It computes the next state and hands the host one action to execute, which means the same pipeline runs against any agent runtime rather than one vendor's.52 It writes no source file and pushes no branch; the nine output files are written by the host.25

Seventeen tools are registered, and three profiles decide which of them a given host sees: everything, everything minus five internal diagnostics that are refused at call time, or just the two validators for a host that only wants the checker.67 The order is not advisory. The server states it in the message it sends on connect, and the failure mode is named: calling the tools out of order does not raise an error, it leaves the run in a wrong state.8

The host Claude Code, or any stdio MCP client it executes the actions; the server never does ONE NODE PROCESS · STDIO a stateless reducer step(state, result?) → next_state, action “It runs on your machine, and it never phones home for a verdict.” 17 TOOLS, THREE PROFILES full · everything agent · 5 internal tools refused at call time verifier · the two validators only WHAT THE HOST RUNS spawn_subagents section drafts and judge verdicts come back this way, never from a No model in the loop “No tool calls an LLM”, so the same pipeline runs against any agent runtime rather than one vendor's OUTGOING, ITS OWN a memory server a code-graph server connect timeout 10 000 ms ON DISK, OPTIONAL ~/.prd-gen/evidence.db ~/.prd-gen/reliability.db the run state itself is not here stdio THE ORDER IS THE PRODUCT “The tools are a strongly ordered pipeline: start_pipeline then submit_action_result until done; calling them out of order does not error, it leaves the run in a wrong state.” The generation loop and the verification stage are separate, and the server says so on connect.
Two connections go out, and they are its own. The server opens stdio clients to a memory server and to a code-graph server, under a ten-second connect timeout that the file labels a provisional heuristic rather than a measurement.13 Everything else stays local: two optional SQLite files under your home directory, and no network call for a verdict.2
Exhibit 02 · the pipeline

Twenty steps, and the nine you have to ask for

Eleven steps produce the specification, from the first banner through preflight, context detection, input analysis, a feasibility gate, clarification, budgeting, section generation, ticket generation, export and a self check.9 Nine more carry the same run into implementation, testing, review and a pull request — and they are opt-in, behind a gate a human opens. The terminal marker is not a step, which is the kind of distinction that only matters until someone counts the steps wrong.9

Inside section generation sits the loop that does the work: recall from memory, draft by a spawned agent, validate deterministically, retry. The retry budget is three, and the file that defines it says what that three is: a provisional anchor pending an ablation study, replaceable at runtime by a calibrated value.10 That is a small piece of honesty with a large effect, because it tells a reader which numbers in this system are load-bearing and which are placeholders.

11 STEPS · MANDATORY · THE SPECIFICATION banner → preflight → context_detection → input_analysis → feasibility_gate → clarification → budget → section_generation → jira_generation → file_export → self_check 9 STEPS · OPT-IN BEHIND A HUMAN GATE · THE IMPLEMENTATION implementation_gate → pre_impl_grounding → implementation → post_impl_verification → testing → review → pr_gate → pr_creation → finalize INSIDE ONE SECTION recall from memory → draft by a spawned agent → validate deterministically → retry, at most 3 attempts MAX_ATTEMPTS = 3 WHY THREE, HONESTLY The constant is labelled in its own file as a provisional anchor, pending an ablation study that is named but not present in the tree. A calibrated replacement can be injected at runtime, so the 3 is a default, not a law. WHAT THE HOST WRITES, NOT THE SERVER nine files at export: six core, three companion “This server only emits spawn_subagents actions — it never edits source files or pushes branches itself.” complete is the terminal marker, not a step the gate a human opens
The host does the writing. Nine files land at export, six core and three companion, and the server emits an action rather than touching the disk.25 The same boundary explains why the pipeline can be driven by a different host entirely: nothing in the reducer depends on who executes the action it returns.5
Exhibit 03 · verification

What it proves, and what it refuses to pretend

The deterministic layer is seventy-three rule functions across twenty-four files, pure parsing with no model involved, which is what makes its verdict identical on every host and inside continuous integration.14 What matters more than the count is the sentence the server sends about it: a passing report establishes structural conformance to those rules, and in the same breath states that it does not establish factual accuracy, implementation feasibility or semantic correctness.15 It also says plainly that it does not judge prose.26

Above that sits the judging layer, and it is built around a refusal. The taxonomy has five verdicts, a performance claim may not receive a plain pass, and a report where everything passes is rejected outright.16 A panel that agrees unanimously across five or more verdicts is flagged as suspicious rather than trusted,17 and half of the confidence-weighted vote forces a failure.18 Judges are treated as fallible by design, which the assurance case states as a limit rather than a feature.24

DETERMINISTIC · NO MODEL IN THE LOOP 73 rule functions across 24 files pure parsing and regular expressions, so the same verdict comes back on every host and in continuous integration Example, verbatim: “FR table contains Story Points column — SP belongs only in Implementation Roadmap” JUDGED · FIVE VERDICTS, ONE OF THEM REFUSED PASS · SPEC-COMPLETE · NEEDS-RUNTIME · INCONCLUSIVE · FAIL “A report with 100% PASS verdicts is REJECTED. NFR claims (latency, throughput, storage) MUST NOT receive PASS.” A panel agreeing unanimously over five or more verdicts is flagged; half the confidence-weighted vote forces a failure. WHAT A PASSING REPORT ESTABLISHES “structural conformance to those rules” and, in the same sentence, what it does not: “it does not establish factual accuracy, implementation feasibility, or semantic correctness.” WHAT NOBODY HAS DONE, FROM THE ASSURANCE CASE no formal verification · no adversarial review no second reviewer, because there is one maintainer nothing about the models: judge verdicts are treated as fallible One diagnostic does not follow this discipline: the health check always returns a literal status of “ok”, never derived from the three booleans it computes. A caller reading only that field cannot see a degraded component; it has to read the booleans itself.
One diagnostic does not hold the line. The health check always returns a literal “ok”, never derived from the three booleans it computes, so a caller that reads only that field cannot tell a degraded component from a healthy one.19 It is a small inconsistency in a codebase that is otherwise careful about exactly this, and it is worth knowing before you wire an alert to it.
Exhibit 04 · state and numbers

Nothing survives a restart, and the numbers say which is which

A run lives in one in-memory map and nowhere else. The file says why — the server is per-session, so a map is correct for one host — and names the swap that would change it.11 The consequence is worth stating plainly: restarting the process loses every run in flight, and what survives is the files the host already wrote plus the two optional databases. Terminal runs are evicted after thirty minutes or when sixty-four accumulate; runs in flight are never evicted.12

Two override paths exist and both are explicit. A calibrated gate can be held back from promotion even when it passes its threshold,20 and a held-out comparison refuses to run unless the caller acknowledges the seal, with the reason carried in the error message itself.21 On the measurement side, one calibration run is dated and complete — and carries its own two flags saying it diverges beyond tolerance and should be recomputed.22 Every performance ceiling around it is labelled a provisional heuristic in the file that defines it.23

RUN STATE · MEMORY ONLY “a single in-memory map is correct for one host” Restarting the process loses every run in flight. What survives: the files the host already wrote, and the two optional databases. TTL 30 min · at most 64 runs · ≈12.8 MB runs in flight are never evicted TWO WAYS TO OVERRIDE A RESULT hold_provisional holds a calibrated gate back from promotion even when it passes its threshold SEAL_VERIFIED a held-out comparison refuses to run unless the caller acknowledges the seal explicitly MEASURED, AND SELF-FLAGGED One dated calibration run: 1 050 attempts, 500 events, rate 0.476 with a 95 % interval. The file carries its own two flags: diverges_beyond_tolerance: true recompute_recommended: true A measurement that argues against itself. CHOSEN, AND LABELLED AS CHOSEN Retry budget 3, iteration ceiling 100, wall-time ceiling 500 ms, section-failure ceiling 5: every one of them carries the words “provisional heuristic” or “provisional anchor” in the file that defines it. The repository labels its own guesses rather than dressing them up. AND ONE NUMBER THIS DOSSIER COULD NOT SETTLE The published test count is 1506. Counting call sites statically in the tree at this revision gives 1 061 across 98 test files; the gap is runtime-expanded cases, which only running the suite can resolve. The figure is reported here as unverified rather than repeated.
A measurement that argues against itself is the strongest thing on this page. The calibration file carries both its result and the two flags saying that result should not be trusted yet.22 Most projects publish the number and drop the flags.

One number this dossier does not repeat

The published test count is 1506, and it appears in several user-facing places. Counting call sites statically in the tree at this revision gives 1 061 across 98 test files. The gap is almost certainly runtime-expanded cases, which only executing the suite can resolve, and the repository does gate that claim in continuous integration against a real run. It is reported here as unverified from the source alone rather than repeated as a fact, which is the same standard the project applies to a latency claim.3 The project's own honesty about energy is worth quoting for the same reason: it publishes no carbon figure, because it measures no joules and avoided rework is a design argument rather than a measurement.4

1README.md:20:20-21 — “Catch a hallucinated spec before it becomes code… deterministically, with no model in the checking loop.”
2README.md:49:49 — “The server is a stateless reducer… It runs on your machine, and it never phones home for a verdict.”
3README.md:43:43 — “The verdict taxonomy *refuses* to pass a latency, throughput, fps or storage claim — it returns SPEC-COMPLETE or NEEDS-RUNTIME instead of a confident guess.”
4README.md:57:57 — “we publish no energy or CO₂ figure: this repository measures no joules, and avoided rework is a design argument rather than a measurement.”
5README.md:368:368 — “No tool calls an LLM — section drafts and judge verdicts come back via the host's spawn_subagents action so the same pipeline runs against any agent runtime.”
6packages/mcp-server/src/index.ts:56:56 — seventeen tools: five diagnostics, two validation, two evidence, eight pipeline, verification and budget
7packages/mcp-server/src/tool-profiles.ts:43:43-57 — the five internal tools the agent profile refuses at call time, and the two the verifier profile exposes
8packages/mcp-server/src/tool-profiles.ts:126:126-129 — the call order the server sends the host on connect: budget, start, state, submit, repeated until done; verification is a separate stage
9packages/orchestration/src/types/state/pipeline-step.ts:3:3-93 — the twenty steps, with complete as a terminal marker rather than a step
10packages/orchestration/src/handlers/section-generation-constants.ts:26:26 — three attempts per section, labelled at :18-25 as a provisional anchor pending an ablation study, with a calibrated replacement injectable at runtime
11packages/orchestration/src/run-store.ts:4:4-6 — “The MCP server is per-session, so a single in-memory map is correct for one host. If we ever need multi-host or persistent runs, swap this for a SQLite-backed store.”
12packages/orchestration/src/run-store.ts:68:68 and :86-94 — a thirty-minute time to live, at most sixty-four runs, about 12.8 MB resident; :13 states that runs in flight are never evicted
13packages/ecosystem-adapters/src/transport/stdio-mcp-client.ts:36:36 — a ten-second connect timeout on the server's own outgoing connections, labelled a provisional heuristic
14packages/validation/src/hard-output-rules/rules/sp-rules.ts:47:47 — one of the seventy-three rule functions, and the exact sentence it emits: “FR table contains Story Points column — SP belongs only in Implementation Roadmap”
15packages/mcp-server/src/tool-profiles.ts:152:152-155 — what a passing report establishes, “structural conformance to those rules”, and in the same sentence what it does not: “it does not establish factual accuracy, implementation feasibility, or semantic correctness.”
16packages/core/src/domain/verdict.ts:14:14-15 — “A report with 100% PASS verdicts is REJECTED. NFR claims (latency, throughput, storage) MUST NOT receive PASS.”; :17-23 lists the five verdicts
17packages/verification/src/orchestrator.ts:375:375-378 — a panel agreeing unanimously over five or more verdicts is flagged. The same rule is implemented a second time in core/src/domain/verdict.ts:38-44 and the two are not wired together
18packages/verification/src/consensus.ts:139:139 — half of the confidence-weighted vote forces a failure
19packages/mcp-server/src/index.ts:199:193-208 — the health check returns a literal “ok”, never derived from the three booleans it computes, so a caller reading only that field cannot see a degraded component
20packages/benchmark/src/calibrated-gates-loader.ts:51:51-58 and :107-109 — a calibrated gate marked hold_provisional is held back from promotion even when it passes its threshold
21packages/benchmark/calibration/paired-bootstrap.ts:184:184-188 — a held-out comparison refuses to run unless the caller acknowledges the seal: “precondition violated — verifyHeldoutPartitionSeal must be called before invoking this function.”
22packages/benchmark/calibration/data/event-rate-K50.json:16:1-18 — a dated calibration run, 1 050 attempts and 500 events, rate 0.476 with a Clopper-Pearson interval, carrying its own flags diverges_beyond_tolerance and recompute_recommended
23packages/benchmark/src/pipeline-kpis.ts:360:360, :369, :377 — the iteration, wall-time and section-failure ceilings, each labelled “provisional heuristic” in the file that defines them
24docs/ASSURANCE-CASE.md:118:118-126 — no formal verification, no adversarial review, no second reviewer with one maintainer, and nothing established about the models themselves
25README.md:561:561 — “This server only emits spawn_subagents actions — it never edits source files or pushes branches itself.”
26README.md:562:562 — “It does not validate prose quality.”

Projects · lire en français · about the author