Cadora · design spec

Compare & evaluate, by their surfaces

Everything needed to render or extend the two post-run reads — the exact tokens, the anatomy of the eval checklist and the compare diff, the six checks and which four gate the verdict, every state, and the number formats. The rendered components are live HTML; edit them here or lift the specs into Figma.

Palette

Semantic first · green passes, red fails a critical, amber warns without blocking

Green--scr-green · #16c784✓ · PASS · outcome held

Red--scr-red · #fb7185✗ critical · FAIL · ok changed

Amber--scr-amber · #ffd08a(warn) · est. · ⚠ · Δ attention

Blue--scr-blue · #6ba5f0Models · paths

Ink--scr-bg · #0f172aCard ground

Canvas--scr-canvas · #111c2fInset · table body

Panel--scr-panel2 · #162237Header · section rows

Line--scr-line · #334158Borders · grid rules

Text--scr-text · #f5f7fbPrimary · run id · values

Muted--scr-muted · #9aa8bcLabels · captions

Type & metrics

Monospace is the instrument voice — both tools are read at the shell

Role	Family	Size / weight	Notes
Check row	mono	12.5px / 400	`✓ <name>: <detail>` — one per invariant
Verdict line	mono	12.5px / 700	`score P/T (NN%) → PASS` green · `FAIL` red
Node diff line	mono	12.5px / 400	`A[✓ model $cost] B[✓ model $cost]`
Cost / Δ	mono	amber	`$%.4f`; `Δcost (B−A)` attention accent
Value	mono	text	Run id, executor, tokens, per-node figures

Metric	Value	Metric	Value
Card radius	10px	Verdict line-height	1.85
Card padding	14px 16px	State padding	11px 13px
Cost precision	4 decimals	Missing value	— (em dash)
Header separator	· (spaced)	Verdict arrow	→ (spaced)

Anatomy — the eval checklist

The one component a reviewer reads to accept or reject a run

1234eval flags-verify · executor=claude · topology=multi-backend-feature-flags ✓ run_ok: manifest.ok=True ✓ all_nodes_ok: all nodes ok ✓ gates_passed: no failing gates ✓ integrity_clean: no integrity findings ✓ cost_attributed: all nodes have cost (3 estimated from price table) (warn) ✗ aidlc_artifacts: no aidlc-docs artifacts found (warn) score 5/6 (83%) → PASS

1Header — the run being scored, its executor, and its topology. One run in, one verdict out.

2CRITICAL block — run_ok · all_nodes_ok · gates_passed · integrity_clean, in green. Every one must pass or the verdict is FAIL.

3Non-critical block — cost_attributed · aidlc_artifacts, each tagged (warn) and rendered amber. Surfaced for attention; a miss never fails the run.

4Score + verdict — passed/total across all six checks and the percentage, then → PASS (green, exit 0) or → FAIL (red, exit 1). Gated on the CRITICAL rows only.

Anatomy — the compare diff

Two manifests in, outcome + cost per node out

12345compare A=guardgoal-claude B=guardgoal-codex A: executor=claude topology=guardgoal ok=False pass=4/4 cost=$14.1011 out_tok=140117 B: executor=codex topology=guardgoal ok=False pass=4/4 cost=$4.9692 out_tok=50295 Δcost (B−A): $-9.1319 nodes: · implement: A[✓ claude-sonnet-5 $11.7842] B[✓ gpt-5.5 $2.3543] · build-test: A[✓ claude-sonnet-5 $0.5467] B[✗ gpt-5.5 $1.6986] ⚠ ok changed

1Run ids — A and B. A trailing ⚠ different topologies line appears when they don't match.

2Per-run summary — executor · topology · ok · pass=n/N · cost · out_tok for each side.

3Cost delta — Δcost (B−A), signed + when B costs more, negative when B is cheaper. Amber-accent metric.

4Node line — A[✓ model $cost] B[✓ model $cost]. Missing side reads · <node>: A only / B only.

5ok-changed flag — ⚠ ok changed when a node passed on one run and failed on the other. The regression signal.

The six checks

Four gate the verdict · two only warn

Check	Gates verdict	Passes when	Fails when
run_ok	critical	manifest.ok=True	manifest.ok=<v>
all_nodes_ok	critical	all nodes ok	failed nodes: <ids>
gates_passed	critical	no failing gates	bad gates: <node:status>
integrity_clean	critical	no integrity findings	findings in: <ids>
cost_attributed	warn	all nodes have cost	missing cost: <ids>
aidlc_artifacts	warn	AI-DLC artifacts captured	no aidlc-docs artifacts found

Verdict = pass iff all four critical checks pass. score is passed/total across all six — so a run can score 5/6 and still PASS (a warn missed), or score 4/6 and FAIL (a critical missed). The two warn checks add detail — (N estimated from price table), (N in credits) — but never change the outcome.

States

Every verdict the two tools produce

eval · PASS — all critical hold

eval pr1-verify: ✓ run_ok ✓ all_nodes_ok ✓ gates_passed ✓ integrity_clean score 6/6 (100%) → PASS

eval · PASS with warnings — 5/6

eval flags-verify: ✓ run_ok ✓ all_nodes_ok ✓ gates_passed ✓ integrity_clean ✗ aidlc_artifacts: no aidlc-docs artifacts found (warn) score 5/6 (83%) → PASS

eval · FAIL — a critical failed

eval guardgoal-claude: ✗ run_ok: manifest.ok=False ✗ gates_passed: bad gates: build-test:blocked_prerequisite score 4/6 (67%) → FAIL

compare · outcome held, price differs

compare A=guardgoal-claude B=guardgoal-codex · implement: A[✓ … $11.7842] B[✓ … $2.3543] Δcost (B−A): $-9.1319

compare · ok changed — regression

compare A=nightly-0706 B=nightly-0707 · construction: A[✓ … $0.7897] B[✗ … $1.2144] ⚠ ok changed Δcost (B−A): +$0.4481

compare · node only / topology mismatch

⚠ different topologies: A=guardgoal B=aidlc · e2e-tests: B only

Score is not the verdict. The percentage counts all six checks, but only the four critical rows decide PASS/FAIL — a green → PASS can sit above an amber warn row, and a single red critical turns the bottom line red. There is no partial pass on the critical set.

Number & glyph formats

The exact strings both tools emit

Element	Format	Example
Cost	$%.4f · — when None	$11.7842 · —
Cost delta	Δcost (B−A): [+]$%.4f	+$0.4481 · $-9.1319
Pass ratio	pass=n/N · score P/T	pass=4/4 · 5/6
Score %	(NN%) rounded	(83%) · (100%)
Outcome glyph	✓ pass · ✗ fail	A[✓ …] B[✗ …]
Attention glyph	⚠ ok changed · ⚠ different topologies	⚠ ok changed
Verdict	→ PASS · → FAIL	→ PASS

What each command produces

Two verbs · compare needs two runs, eval needs one · both add --json

Command	Emits	Exit
cadora compare <a> <b>	Per-run summary + per-node outcome/cost diff + Δcost. --json = structured diff.	0
cadora eval <run>	Six-check checklist + score + verdict. --json = checks[] + passed/total/score/verdict.	0 pass · 1 fail

Both take --archive-dir (default runs). Both are pure functions over the manifests — no LLM, no network — so the text and JSON are byte-stable across runs. Only eval sets a non-zero exit, which is the CI gate.

Voice

The output is a measurement, not a grade

Do	Not
Name the check — gates_passed: no failing gates	“Quality: good”
Show the delta with its sign — Δcost (B−A): $-9.1319	“Codex is cheaper”
Mark estimates — (3 estimated from price table) · est.	Present est. figures as measured
Keep warn and fail distinct — amber (warn) vs red critical	Fail the run on a non-critical miss
Keep the exact verdict glyphs — → PASS / → FAIL	“Success!” / “Rejected”

Tokens and behavior are taken verbatim from cadora/compare.py and cadora/evaluate.py. The CRITICAL set is the module's _CRITICAL constant. Edit the rendered components in place, or hand the palette and metrics tables to a Figma library.

Design specification for Cadora's compare and evaluate tools: the color and type tokens, the anatomy of the eval checklist and the compare diff, the six checks and which four are CRITICAL, every verdict state, the number and glyph formats, what each command produces in text and JSON, and the voice.