Eval Scorecard

npx shadcn@latest add https://ui.zyeon.ai/r/eval-scorecard.json

Build a React + TypeScript + Tailwind "EvalScorecard" component with zod and
lucide-react. The dial is hand-rolled SVG — no chart library — and the component
has no other runtime dependency.

Contract
- A zod schema (`evalScorecardRunSchema` in a sibling contract file) is the
  single source of truth for ONE run: { id, model, overall, criteria[],
  failures[], dataset?, completedAt?, baselineOverall?, baselineLabel?,
  threshold?, sampleCount? }.
- A criterion is { id, name, score?, threshold?, baseline?, description?,
  sampleCount?, failedCount? }. `score` is OPTIONAL and that is the point: a
  judge that timed out produced NO score, and rendering it as 0 reads as a total
  failure. Scores are fractions of 1 — higher is better. This card is for
  quality metrics; latency and cost belong elsewhere, because there "up" means
  the opposite.
- A failure is { id, criterionId?, input, expected?, got, score?, note? }.
  `expected` is optional because an open-ended judge has no gold answer, and an
  empty "Expected" box invents one.
- `failures` is a SAMPLE — the worst cases the report carried. `failedCount` is
  how many cases really failed. The two are different numbers and the card says
  both; showing three cases and calling that the failure count is how a
  regression ships.
- A separate envelope status — "loading" | "empty" | "error" | "ready" — is the
  card's own render state, independent of the run's verdict. `status="error"`
  means the REPORT failed to load; the run failing is a "Gate failed" verdict in
  the ready branch, and the two read differently on screen.
- Props: run: Run | null; status; threshold? (0.8); deltaEpsilon? (0.005);
  order? ("given" | "failing-first" | "delta"); maxFailuresPerCriterion? (3);
  excerptChars? (240); density? ("comfortable" | "compact"); showDial? (true);
  defaultOpenFailures? (false); formatScore?; formatDelta?; formatDate?;
  onSelectFailure?(failure, criterion?); onRetry?; emptyState?; errorMessage?;
  label? ("Evaluation run"); className plus the rest of the element props, ref
  forwarded to the <section>.
- Every numeric prop is clamped (thresholds to 0..1, maxFailuresPerCriterion to
  >= 1, excerptChars to >= 40, deltaEpsilon to >= 0) so a 0 or a NaN can never
  render an empty panel or a bar wider than its track.

Behavior
- THE GATE RESOLVES IN THREE STEPS: criterion.threshold ?? run.threshold ??
  the `threshold` prop. Data wins over configuration, because a run's gate is a
  fact about that run. Each row draws a tick at ITS OWN gate, so a rubric that
  mixes a 0.99 JSON-validity bar with a 0.70 conciseness bar stays readable.
- A criterion is passed / failed / unscored. Unscored rows are excluded from the
  "4 of 6 criteria passed" tally, draw a dashed empty track and say "not scored"
  — never a zero-width solid bar, which reads as "scored zero".
- REGRESSION IS ITS OWN OUTCOME: a criterion that fails now AND was at or above
  its gate on the baseline is labelled a regression and counted in the header.
  That is the number that blocks a merge; a criterion that has always failed is
  a known gap, not news.
- Deltas have four readings and only one of them is a number: no baseline →
  "new"; |delta| < deltaEpsilon → "unchanged"; otherwise an up/down arrow plus
  signed points ("+3.2 pts"). Eval scores jitter between runs, so an arrow for a
  0.1-point move is noise dressed as a signal. Deltas are POINTS, never a
  percent of a percent.
- The overall score is NEVER recomputed from the criteria — your harness may
  weight them, and a card that silently disagrees with the harness is worse than
  no card. The tallies (passed / failed / unscored / regressions) ARE derived,
  so they cannot drift from the rows they summarise.
- Failing cases live on the row that owns them, behind a real <button
  aria-expanded> whose panel is UNMOUNTED when closed (a zero-height collapse
  leaves every case body and copy button reachable by Tab — an invisible
  keyboard trap). `aria-controls` is only set while open so it never points at
  an id that is not in the document.
- Cases are ordered WORST SCORE FIRST with a stable index tiebreak; cases with
  no score sink to the bottom instead of sorting as zero. The panel draws
  maxFailuresPerCriterion of them and then an explicit "Show N more sampled
  cases", plus a line stating how many of the real failures were sampled at all.
- A criterion with failedCount > 0 but no sampled case renders the count as
  PLAIN TEXT ("7 failing cases · none sampled"), not as a disclosure — a chip
  that expands to nothing is a dead control.
- Cases whose criterionId matches no criterion in the rubric are grouped into a
  final "Unmatched failing cases" row rather than dropped. A rubric that changed
  between runs is exactly when you need to see them.
- Each case field is excerpted to excerptChars with one "Show full case" toggle
  for the whole case and an on-screen note of the cap. Copy always writes the
  FULL case (id, criterion, score, input, expected, got, judge note) so it can be
  pasted into an issue or a rerun script.
- The copy button reports every outcome: a blocked clipboard flips to "Copy
  failed" instead of doing nothing. Its confirmation timer is cleared on unmount
  and its async result is dropped once the button is gone, so collapsing a panel
  mid-write never sets state on a dead node.
- "Expand all failing cases" toggles every group at once from one control; the
  set of open groups is state on the card, keyed by criterion id.
- Identity changes reset the card (adjust state during render, no effect): a new
  run id restores the default disclosure state, so a recycled card can never
  show the previous run's panels against the new run's rows.
- The card owns NO CLOCK and fetches nothing. `completedAt` is data,
  `onSelectFailure` hands a case back to your own inspector, and the "Open case"
  button is not rendered at all when that handler is missing.
- Edge cases that must render: an empty rubric (say so, do not draw an empty
  list); a run with no baseline anywhere; a score of exactly the gate (passes);
  a gate of 0 or 1 (the tick overhangs the track and must not be clipped).

Rendering & styling
- Semantic tokens only: bg-card / border / text-muted-foreground for chrome,
  bg-chart-2 + text-chart-2 + stroke-chart-2 for passing, bg-destructive /
  text-destructive / stroke-destructive for failing, bg-foreground/60 for the
  gate tick, bg-muted for tracks and skeletons. No hard-coded colours anywhere,
  so a theme swap re-skins the whole card and dark mode is free.
- PASS/FAIL IS A POSITION. Every track carries a tick at its own gate and the
  dial carries the same tick on the ring, so "78.1% against an 80% gate" is
  legible in a monochrome theme and to a colour-blind reader. Colour and icon
  (check / alert / dashed circle) repeat the reading; they never carry it alone.
  The track has no overflow-hidden — a marker at 1.0 would be clipped exactly
  when it matters most.
- The dial is a <div role="meter"> with aria-valuenow and an aria-valuetext that
  spells out the whole reading ("89.1% overall, at or above the 85% gate"). The
  role is children-presentational, so the number drawn inside it is not read
  twice, and the SVG is aria-hidden.
- Each row carries ONE sr-only sentence — score, gate, outcome, direction word
  plus signed delta, failing-case count — while the visual number, arrow and bar
  are aria-hidden. An arrow glyph reads as nothing and a "+" is announced
  inconsistently, so the direction is a WORD and the signed figure rides in
  parentheses.
- Motion is decorative and CSS-only: `from`-only keyframes grow each bar
  (scaleX, staggered ~45ms per row) and sweep the dial's stroke-dashoffset. The
  resting style IS the final geometry, so `motion-reduce:animate-none` lands on a
  fully drawn card rather than an empty one, and no JS drives either animation.
- Numbers use a FIXED locale and the date a fixed UTC formatter, because a card
  rendered on the server and hydrated in another time zone must produce the same
  string. tabular-nums keeps columns from dancing as values change.
- Long model names, dataset ids and case bodies use `wrap-anywhere` plus
  `min-w-0`, NOT `break-words`: only the former lowers the element's min-content
  width, which is what actually stops a 60-character run id from widening the
  card.
- cn() merges the consumer's className into the root, which also spreads the
  remaining element props and forwards its ref.

Customization levers
- Gate policy: `threshold` is only the last fallback — put the real gate in the
  data (per run, or per criterion) and the card follows it, tick and all.
- Signal floor: `deltaEpsilon` decides what counts as movement. Raise it for a
  noisy judge, drop it to 0 to show every arrow.
- Reading order: `order="given"` keeps the rubric's own order (use it when the
  rubric is a narrative), "failing-first" for triage, "delta" for a release
  review that starts at the biggest drop.
- Case budget: `maxFailuresPerCriterion` and `excerptChars` decide how much is
  drawn before an explicit reveal — but never make either silent; the count is
  part of the reading.
- Density and chrome: `density="compact"` shrinks the rows and the dial,
  `showDial={false}` drops the dial entirely when the card sits under a bigger
  run header; `emptyState` replaces the empty body and `errorMessage` +
  `onRetry` own the envelope failure.
- Wording: `formatScore` switches percent to the raw 0–1 (or to a 1–5 Likert)
  without touching the geometry, `formatDelta` re-words the movement, and
  `formatDate` swaps UTC for the reader's own time zone if you accept the
  hydration cost.
- Wiring: `onSelectFailure(failure, criterion)` is what deep-links a case into
  your own inspector; leave it off and the card stays read-only instead of
  growing a button that opens nothing.