Clément Deust

I build instruments that catch what harnesses get silently wrong

Designed and ran a matched-conditions A/B benchmark of the popular open-source agent stack versus my own: same corpus (4 repos incl. an adversarial shell/markdown one), smoke-test-before-measure rule, retrieval probes scored by independent read-only agents verifying every claim with exact greps. The rig exposed opposite silent failure modes. The popular stack was confidently wrong where its index is blind (real fan-in undershot by 17×); mine served real facts months stale, with no timestamp to warn anyone. Full matrix, fairness caveats and my own side’s losses published; no aggregate-winner claim where the data couldn’t support one.

• Judge panels for spec verification carry Bayesian reliability priors per claim type, held-out seals so calibration can’t be peeked at pre-evaluation, and Kaplan-Meier retry budgets.
• Measured my own agent-router’s inter-rater ceiling (two LLM raters, Cohen κ 0.554; agreement on routing shape only 39%) and report every routing score against that ceiling, never against 100%.

Cortex — an agent memory system with write governance and an audit trail

~266k LOC Python, 7,366 tests, 49 releases in 3 months, 17 benchmark suites (LoCoMo, LongMemEval, MemoryAgentBench + bespoke). The parts other memory systems don’t have: an origin-allowlisted write gate so fetched web text can’t promote itself into long-term memory; supersession enforced at read time so a corrected fact can never be re-injected; and injection receipts: every auto-injected memory carries a marker the why tool resolves to what was actually in context, without claiming causation.

• Dual backend (SQLite / PostgreSQL+pgvector), lazy SQL-side decay, consolidation modeled on complementary learning systems.
• The A/B benchmark exposed a freshness-policy gap in my own serving path; root-caused to three unwired seams, fix plan public.

A code-graph MCP server that tells you when it might be wrong

~84k LOC Rust, 841 tests, 24 MCP tools: tree-sitter indexing across 11 languages, cross-file resolution ladder (static → macro tables → stdlib → LSP), community detection, blast-radius queries. Its distinctive feature is epistemic honesty as API: impact results carry exact vs lower-bound markers with the named carriers of uncertainty, every heuristic edge has a confidence and resolution method, and a coverage sidecar records what the index did not parse.

• Pre-registered falsifiable benchmark, content-hashed corpus, committed before execution. It falsified my own recall hypothesis (0.83 vs substring 1.00), and that loss is published in the README.
• Supply chain: CodeQL, nightly fuzzing, cargo-audit/deny with empty ignore list, Sigstore attestation + CycloneDX SBOM in release CI.

Deterministic guardrails around a non-deterministic model

A plugin family for Claude Code: context management that checkpoints while headroom remains (the warn level blocks the stop once, persists a mechanical checkpoint plus a token-budgeted memory-writer pass, then resumes, so the hard cap becomes a formality); true session spend accounting that parses each subagent’s own transcript; burn-rate pacing in the statusline. Plus a 120-agent orchestration layer with worktree-per-agent isolation and identity injected at the spawn site so a subagent cannot forge its own memory scope.

• 97% coverage over hooks/tools; every documented count is derived by a CI-checked command (doc-count-check fails the build when docs and tree disagree).
• Gates degrade open but never silently: one-line stderr trace on every non-fatal path, a policy adopted after an audit found a silently disabled hook.

An evidence-first technical watch for the agent ecosystem

A deterministic daily scout over a fleet of upstream repos (GitHub metadata only, no code execution, fails closed on missing evidence) feeding a five-state proof ladder: CLAIMED → IMPLEMENTED → VERIFIED → COMPARED → ACTIONABLE. A README can only ever establish CLAIMED; parity claims require a matched benchmark citation, enforced in CI.

• Dossiers keep retracted root causes in the tree, because the chronology of being wrong is itself evidence.
• Surveyed ~90 of the highest-momentum agent-tooling repos in Aug 2026; none ship an equivalent verification discipline.

Japonais 2027, the harness dogfooded

An offline, single-file PWA I use daily to work toward JLPT N3, built with my agent team under the same zetetic standards as everything above. I led it from an iPad by usage reports: I described the symptom, the agent reproduced it against real data before believing it, fixed the minimum, then locked the fix with a test replaying my exact report. Ten executable invariants gate every release; red output reads “do not ship” and nothing ships.

No content was invented: every kanji meaning, etymology and reading comes from a cited source (KANJIDIC2, CJKVI IDS, makemeahanzi, Janome), and a hole in a source stays a visible hole. Supabase Postgres behind row-level security on every row. 117 study sessions and 600+ progression records in the first week.

• When I disagreed with the app’s numbers, we didn’t debate. The agent built an audit screen crossing two independent measurements of the same work, and the screenshot settled it.
• The code stays private because the Irodori curriculum (Japan Foundation) isn’t redistributable. The method that built it is public: read the full case study.