consensus-fix
Fans N independent LLM fixes at a failing test, gates each against the rails, and picks the smallest consensus winner.
consensus-fix
ADLC phase: P4 Build · Gate: a fix is only accepted when it passes both the repro test and the full rail suite, and agrees with peers.
What it defends against
Single-shot fixes stall on hard bugs, and a fix that passes the one failing test
may quietly redden the rest of the suite. consensus-fix applies N-version
programming: it fans N independent, stateless LLM completions at the same broken
test, applies each candidate in isolation, and keeps only those that pass BOTH the
repro command and the full rail regression suite. Survivors are grouped by
agreement and the smallest diff from the largest consensus group wins. When every
survivor is a singleton (with --n >= 3), that divergence is treated as evidence
the spec is ambiguous, and the tool escalates rather than guessing. A candidate
proposing no changes is a no-op, not a fix, and is discarded before it can ever
become a survivor — it would otherwise apply cleanly and be judged solely by
whether --test-cmd happens to pass a second time against the untouched source.
Usage
adlc consensus-fix --test-cmd "..." --files a.mjs,b.mjs [--rails "..."] [--n 3] [--tier mid] [--provider <name> | --providers <a,b,c>] [--apply] [--allow-dirty] [--json] [--prompt-only]| Flag | Description |
|---|---|
--test-cmd <cmd> | Repro gate: the failing-test command (required). Must exit non-zero while the bug is present. |
--files <paths> | Comma-separated files the LLM may modify (required). Any candidate touching a file outside this list is discarded. |
--rails <cmd> | Regression gate: the full frozen rail suite. A candidate survives only if both gates exit 0. Omitting it prints a WARNING and skips the rail check. |
--n <int> | Number of independent candidates (default 3). Ignored when --providers is given. |
--tier cheap|mid|frontier | LLM tier for completions (default mid). |
--provider <name> | Force one provider (anthropic/openai/gemini/agy) for the run. Mutually exclusive with --providers. |
--providers <a,b,c> | Draw one candidate per named provider family (cross-family diversity). Each named provider must have its API key set or the run fails closed first. Mutually exclusive with --provider. |
--apply | Write the winning fix. Default is dry-run. Never applies when all-divergent. |
--allow-dirty | Skip the dirty-tree guard (useful in CI with staged-only changes). |
--json | Machine-readable output. |
--prompt-only | Print the prompts that would be sent and exit 0. Works with zero API keys. |
Exit codes
--n >= 3 (spec ambiguity; escalate).Example
$ adlc consensus-fix --test-cmd "node --test test/math.test.mjs" \
--rails "node --test test/*.test.mjs" \
--files src/math.mjs --n 5
survivors: 3/5 groups: 1 rails: checked
winner: src/math.mjs (1 changed line, consensus group of 3)
dry-run — re-run with --apply to write it
exit 0Safety guarantees
- Dirty-tree guard: refuses to run against a dirty git tree unless
--allow-dirty. Prevents accidental loss of unstaged work. - Snapshot + restore in
finally: every candidate evaluation restores all--filesto their original content, even if the test crashes or the LLM returns garbage. - Termination signal restore: SIGINT, SIGTERM or SIGHUP at any point before the report (including while a candidate's test or rails command is running) stops the run: the in-flight command's process group gets SIGTERM, then SIGKILL after 2 s, the original files are restored from the snapshot, no winner is applied, and the process exits 1. SIGKILL sent to consensus-fix itself cannot be handled, so it can leave a candidate's hunks on disk; recover with
git checkout -- <files>. - Atomic writes: file restoration, candidate hunk application, and winning candidate --apply writes are atomic (write temp file in same directory, then renameSync) to guard against partial writes or truncation.
- File-list enforcement: any candidate that references a file outside
--filesis discarded with a diagnostic message, never written.
Go deeper
Why prosecution, not code review, is the discipline that makes fixes trustworthy: Prosecution, Not Code Review.