Token Latency

npx shadcn@latest add https://ui.zyeon.ai/r/chart-token-latency.json

Build a React + TypeScript + Tailwind "ChartTokenLatency" serving-latency card in
plain SVG with zod. Not recharts, and not a generic scatter: the crosshair is a
pair of SERVICE BUDGETS rather than a median split, the percentile lines are
computed in the component from the same rows the dots are drawn from, and the
end-to-end time nobody can read off either axis is derived rather than accepted.

Contract
- One zod schema is the source of truth:
  { status: "loading" | "empty" | "error" | "ready"; title: string;
    requests: { id: string; ttftMs: number; tpotMs: number;
                outputTokens: int >= 1; queueMs?: number; model?: string }[];
    budgets: { ttftMs: number > 0; tpotMs: number > 0 };
    window?: string; sampledFrom?: int }.
- The two axes are independent measurements a user actually feels: ttftMs is how
  long the answer took to START, tpotMs is how fast it then arrived, in
  MILLISECONDS PER OUTPUT TOKEN (20 ms/token is 50 tokens a second). They come
  from different parts of the stack — prefill and scheduling against decode under
  batch pressure — which is the entire reason for plotting one against the other.
- tpotMs is the mean gap AFTER the first token, so end to end is
  ttftMs + tpotMs * (outputTokens - 1). The minus one is the standard off-by-one:
  the first token is already paid for by TTFT. Never accept a totalMs field — a
  second source of truth can disagree with the dot drawn from the same request.
- queueMs is the ONLY phase the payload supplies, because it is the only number
  the scatter cannot see: TTFT already contains it and nothing in the other
  fields separates waiting from working. Refine it to be <= ttftMs (a queue
  longer than the TTFT containing it would give prefill a negative duration).
  Reject a queue/prefill/decode object: two of its three numbers are already on
  the card, and the first time they drift the panel stops being believed.
- Export the zone tuple ["within","slowFirstToken","slowStreaming","slowBoth"]
  and a label map, plus the phase tuple and its labels, so the chips, the
  sentences and the screen-reader table all read one source.
- refine: a ready card needs at least one request; ids must be unique (an id is
  the dot's identity and its table row header).
- Props = z.infer plus height (default 210, clamped 150..360), onRetry, className
  and the div's native props (Omit "title"); forwardRef to the card.
- Export the maths beside the component so a test can print the numbers the
  picture is made of: buildTokenLatencyModel(), latencyPercentile(),
  apportionPercent(), floorShares(), latencyAxisStep().

Behavior
- FOUR ZONES, ONE PARTITION. The budget pair cuts the plane: within budget, slow
  first token, slow streaming, slow both ways. Budgets are INCLUSIVE — a request
  exactly on the line met it. Every request is in exactly one zone, so the counts
  add to N and the shares add to 100. Do not collapse the two single-axis misses
  into one "slow" bucket: a request that thinks for a second and a request that
  dribbles need opposite fixes, and that difference is the only thing the second
  axis bought you.
- NEAREST-RANK PERCENTILES, COMPUTED HERE. rank = ceil(p/100 * N), 1-indexed, so
  p50 and p95 are REAL requests you can point at in the table rather than
  interpolations. Draw them as two crosshairs (dashed p50, dotted p95) and print
  the same numbers in the legend, so the line and the readout can never drift.
- THE PERCENTILE LINES ARE MARGINAL, AND SAY SO. Each axis is ranked on its own,
  so the p95 first-token value and the p95 per-token value almost never belong to
  the same request: the point where the two p95 lines cross is not a request
  anyone made. Say it in words — a reader will otherwise read that intersection
  as "the p95 request".
- PERCENTILES AND AXES DO NOT MOVE WITH THE FILTER. Both are computed over every
  request, so hiding a zone removes dots and nothing else. An axis that rescaled
  on filter would move every remaining dot and make a filter look like new data.
- ZONE CHIP IS THE LEGEND IS THE FILTER. One chip per zone, a real button with
  aria-pressed: swatch, name, count, its share of what is currently shown, and
  that zone's median end-to-end time — which is the payoff, because a first-token
  miss costs a fraction of a second while a per-token miss multiplies by every
  token the answer needed. Refuse to switch the last zone off and say why in a
  role="status" line rather than no-opping silently. A zone with no requests is
  still drawn (the four are a partition) but as a plain, non-interactive chip —
  a button that cannot change anything is a dead control.
- EXACTLY 100, IN EVERY FILTER STATE. Shares are apportioned by largest remainder
  (Hamilton): floor every share, then hand the leftover points to the largest
  fractional parts, ties broken by the larger raw count then by position. A
  26/6/5/3 split of 40 is 65 + 15 + 12.5 + 7.5, which per-share rounding prints
  as 101. A zone with nothing in it has no fraction to claim a point, so a real
  zero survives as a zero.
- WHERE THE WALL CLOCK GOES. Pick the requests whose end-to-end times ARE the
  nearest-rank p50 and p95, and decompose each one exactly: queue (supplied),
  prefill (ttft - queue), decode (tpot * (tokens - 1)). The three sum to that
  request's own total — no medians are added together, because medians do not
  add. When queueMs is absent show one undivided "first token" phase instead of
  filing a queue wait under prefill. When p50 and p95 land on the same row, say
  so in one caption instead of drawing it twice.
- SIZES GET NUDGED, NUMBERS NEVER DO. A request streaming 800 tokens spends ~99%
  of its clock in decode, so give every non-zero phase a floor of ~2% of the bar
  and borrow it proportionally from the phases above the floor; the floor
  self-limits to an equal split so there is always something to borrow from.
  Every printed millisecond and percent still comes from the raw values.
- DOT AREA CARRIES THE RESPONSE LENGTH — radius scales with sqrt(tokens / median
  tokens), clamped, so twice the tokens is twice the ink and not twice the width.
- Four first-class branches of one card: loading (skeleton mirroring tiles, plot
  and chips, aria-hidden, plus an sr-only role=status line), empty (worded so it
  cannot be mistaken for a failed fetch, and it explains that budgets alone draw
  a crosshair but not a cloud), error (Try again only when onRetry was passed),
  ready. status "ready" with nothing plottable renders the empty branch and names
  how many rows arrived unreadable.
- CLEANUP: one ResizeObserver measuring the plot column so SVG user units are CSS
  pixels, disconnected on unmount and whenever the node changes. No timers, no
  rAF, no window listeners.

Rendering & styling
- Semantic tokens only: bg-card / text-card-foreground panel, border gridlines
  and tiles, muted skeleton, muted-foreground axis text, text-destructive for the
  error headline, ring for focus. Zones take var(--chart-1/3/4/5); the phase bar
  takes var(--chart-2) for decode and then steps OUT of the chart palette
  entirely — muted-foreground for queue, primary for prefill — so a phase swatch
  can never be mistaken for a budget zone. Never a hex.
- The over-budget region is drawn as two solid var(--muted) rectangles whose
  union is an L; solid, because their overlap then cannot darken and because a
  low-alpha wash disappears on a near-black card.
- Emphasis is fill, not colour: a request that met both budgets is a translucent
  ring that stacks legibly where the cloud is dense, and a miss is solid with a
  card-coloured seam so it reads through the grey region and off its neighbours.
  Misses are painted last so they are never buried under the cloud they are being
  compared to.
- Line vocabulary: p50 dashed, p95 dotted, budget solid and heavier, each with a
  matching swatch in the legend beside its own numbers — so no label has to be
  written inside the plot, where four reference lines would collide at a 420px
  card. Ticks are 1/2/2.5/5 x 10^n and thinned by the widest label, and the first
  and last x labels re-anchor so they never hang off the frame.
- ACCESSIBILITY: role="img" on the svg with a full sentence label — honest,
  because nothing inside the frame is a tab stop; the filter lives in real
  buttons and every number is repeated in an sr-only table (sr-only on the
  WRAPPER div, never on the table itself: CSS width is only a lower bound for a
  table box). The svg label describes the FILTERED plot; the table is the
  unfiltered ledger, so give it its own caption. The two percentile rows in that
  table are labelled "each column ranked separately", because they are not
  requests. Chips are focus-visible ringed and carry a spoken sentence.
- Motion: the only animation is the skeleton pulse and the chip hover; both carry
  motion-reduce:animate-none / motion-reduce:transition-none.

Customization levers
- Budgets: the crosshair is the pair you pass, so the same card serves a chat
  endpoint (300 ms / 20 ms per token) and a batch summariser (2 s / 40 ms per
  token) with no code change. Pass a percentile of your own history as the budget
  only if you want the card to grade itself against itself — usually you do not.
- Density: height is the plot box; the two stat tiles and the footnote paragraph
  are the first things to drop for a dashboard tile, the zone chips the last. The
  phase panel is a full-width block below the plot because at 420-540px a side
  column would strangle both; lift it beside the plot behind a measured-width
  check if your cards are wider.
- Third encoding: dot area is output tokens today; swap it for prompt tokens,
  cost, or a fixed radius by replacing one radiusFor() call.
- Colour by model instead of by zone if a routing decision is the point — keep
  the budget crosshair and move the zone verdict into the chip counts, since
  position already tells the reader which zone a dot is in.
- Phase set: add a "tool time" phase by extending the phase tuple, its label map
  and the colour map together — they are one edit — and supply it per request the
  way queueMs is supplied.
- Percentiles: the p50/p95 pair is a constant in one place; a tail-heavy service
  usually wants p95/p99, and adding a third line means adding a third dash
  pattern to the legend so the plot stays self-describing.
- Population honesty: pass sampledFrom and the card says out loud that its p95 is
  the 38th slowest of 40 rather than your metrics store's p95 for the window.