Computer Use Panel

npx shadcn@latest add https://ui.zyeon.ai/r/computer-use-panel.json

Build a React + TypeScript + Tailwind "ComputerUsePanel" component with zod and
lucide-react. It is a viewer for a computer-use session: a screenshot with the
agent's action drawn on top of it, plus a synced action log you can step through.

Contract
- A zod schema (`computerUseFrameSchema` in a sibling contract file) is the single
  source of truth for ONE step: { id, imageUrl, at (ISO 8601 with offset),
  action, state?: "done" | "running" | "failed", error?, alt? }.
- action = { type, x?, y?, toX?, toY?, direction?, text?, description }. The type
  union is the vocabulary of a SCREEN, not of an API: "click" | "double_click" |
  "right_click" | "type" | "key" | "scroll" | "drag" | "wait" | "screenshot".
  Anything richer (a shell command, an HTTP call) is a tool call and belongs in a
  different component.
- COORDINATES ARE PERCENTAGES OF THE SCREENSHOT, 0–100, never device pixels. A
  computer-use API hands you pixels against the machine's resolution; store
  100 * px / screenshotWidth instead. Percentages are the only coordinate space
  that survives the same session being replayed in a 320px panel, on a retina
  export, or after the screenshots are re-encoded at half size. Say this in the
  contract's own comments — it is the single most common way to get this wrong.
- `description` is one sentence in the reader's language ("Clicked Re-run build
  on deployment 4182"), not a serialised call. `text` is the literal payload:
  what was typed, the key combo ("cmd+shift+p"), or the scroll amount.
- A separate envelope status — "loading" | "empty" | "error" | "ready" — is the
  PANEL's own render state. It is independent of any frame's state: status
  "error" means the session log could not be fetched, a frame's "failed" means
  the agent's click missed. They are two different problems and must read
  differently on screen.
- Props: frames: Frame[]; status; mode?: "live" | "replay" (default "replay");
  index? (controlled) + defaultIndex? + onIndexChange?(index: number); showLog?
  (true); playbackIntervalMs? (1600, clamped >= 200); typePreviewChars? (72);
  aspectRatio? (16/9); formatTime?; label? ("Computer use session", used as both
  the accessible name and the visible heading); emptyState?; errorMessage?;
  onRetry?; className plus the rest of the element props, ref forwarded to the
  <section>.
- onIndexChange takes the INDEX ONLY, never the frame object. A callback that
  carries the frame forces the auto-play timer to depend on the frames array,
  and a parent that rebuilds that array every render would then restart the
  countdown on every render and never advance.

Behavior
- One selection, two surfaces. The current step is highlighted on the screenshot
  (the numbered marker) and in the log rail (filled ordinal, tinted row, primary
  edge) at the same time, from one index. Sources that can move it: a rail click,
  arrow keys, the header's prev/next, auto-play, and a live append.
- The log rail is a real vertical tablist: role="tablist" with roving tabindex
  (only the selected row is tabbable), Up/Down/Left/Right/Home/End moving the
  selection with AUTOMATIC activation, and the screenshot stage as the tabpanel
  it controls. The stage itself is focusable and takes Left/Right/Home/End —
  Up/Down deliberately NOT, because a widget that eats ArrowDown traps the page's
  own scrolling the moment the reader tabs into it.
- LIVE IS FOLLOW-AND-TAKEOVER. In mode "live" the panel opens on the newest frame
  and jumps to each new one as it streams in — until the reader steps back. That
  is the takeover: following stops, a "Jump to live" button appears, and
  returning to the newest step re-arms following. There is no separate
  "auto-scroll" toggle to fall out of sync with. The follow jump happens by
  adjusting state DURING RENDER (compare the last frame's id with the previous
  one), not in an effect, so an appended frame is painted in the same commit
  instead of showing the previous step for one frame first.
- A NEW SESSION resets everything: the reader's position, the takeover, playback
  and the adopted aspect ratio. Detect it by the FIRST frame's id changing —
  comparing lengths would call a truncated stream a new session.
- Playback is DERIVED, not stored: playing = playRequested AND canPlay AND
  current < last. It therefore stops by itself on the final step, with no effect
  writing state back (a cascading render), and can never be left stuck on by a
  session swap or a one-frame run. The timer is ONE timeout re-armed by the index
  it just changed — cleared on unmount and on every step, so it can never
  outlive the panel. Any manual step pauses playback; pressing play on the last
  step restarts from the first.
- The frame box adopts the screenshot's OWN aspect ratio as soon as it loads
  (naturalWidth / naturalHeight, reported up from the image). This is not
  cosmetic: `object-contain` inside a box of the wrong ratio letterboxes the
  picture and silently shifts every percentage marker off its target. The
  `aspectRatio` prop only holds the space until the first image lands.
- Each screenshot is keyed by frame id so its load state can never be inherited
  by the next one, and the NEXT frame's URL is warmed in an effect (handlers
  dropped on cleanup) so stepping doesn't flash a grey box. A screenshot that
  fails to load becomes an explicit "this screenshot didn't load — the action
  below is still what the agent did" placeholder, never a broken image.
- Markers are per action type: numbered ripple ring for the pointer family (a
  dashed outer ring for double-click), a monospace callout anchored at the caret
  for `type`, keycaps split on "+" for `key`, a bouncing direction arrow for
  `scroll`, a dashed line with a start dot and an end ring for `drag`, and a
  corner chip for `wait` / `screenshot`. The ordinal badge FLIPS to the other
  side of the ring past x=82 and the typed callout flips past x=60 / y=70, so a
  marker at the edge of the screen is never clipped by the frame.
- A pointer action that arrives without coordinates renders NO overlay at all —
  an empty role="img" would still announce "Click marker" for something that is
  not on the picture.
- Typed text is capped in the ON-SCREEN callout only (typePreviewChars, with an
  ellipsis); the caption under the screenshot and the log row always carry it
  whole. A cap is acceptable only because the full value is on screen elsewhere.
- A failed STEP tints and rings the frame and prints its reason in full under the
  screenshot — wrapped, never behind a disclosure, never height-capped. The
  reason a step missed is the reason the reader opened the session.

Rendering & styling
- Semantic tokens only: bg-card / border / text-muted-foreground for chrome,
  bg-primary + text-primary-foreground for markers, ordinals and the selected
  row, bg-primary/10 + border-primary/40 for the live chip and "Jump to live",
  bg-muted for the frame backdrop and skeletons, text-destructive /
  border-destructive / bg-destructive/5 for a failed step, and text-background on
  a destructive fill (a destructive-foreground token does not exist in this
  theme). Markers use stroke-primary / stroke-destructive for the drag line. No
  hard-coded colours anywhere.
- Container queries, not viewport breakpoints: the same panel is a full-width
  session viewer and a 360px side panel, so the rail moves under the screenshot
  at the container's own width. On wide layouts the rail is an absolutely
  positioned scroller inside a stretched flex item, so it fills the screenshot's
  height and scrolls on its own.
- Every decorative animation is guarded: the ripple is hidden under
  prefers-reduced-motion (the static ring, the dot and the number survive, so
  nothing is lost), the scroll arrow's bounce and the live dot's pulse stop, and
  every colour transition is motion-reduce-guarded. Nothing about stepping
  through the session depends on motion.
- The root is a <section aria-label> holding ONE persistent sr-only role="status"
  line ("Step 8 of 10. Click. Clicked Re-run build on deployment 4182"), mounted
  for the whole ready branch so a step CHANGE speaks from an element the screen
  reader is already watching. The overlay is a single role="img" whose label
  spells out the position ("Click marker at 88% across, 62% down the
  screenshot") — that is how a non-sighted reader gets what a sighted one gets
  from a ring sitting on a button.
- Prev/next use aria-disabled, never the native `disabled` attribute: `disabled`
  blurs the button the moment you reach the last step, dropping focus to <body>
  mid-run and taking it out of the tab order. Focus rings are ring-inset on the
  stage and on the rail rows, because the panel clips its own overflow.
- The default timestamp is a fixed en-US, UTC formatter. A default that reads the
  visitor's zone renders one string on the server and another in the browser —
  a hydration mismatch on every server-rendered session. Local time is what
  `formatTime` is for, applied by you, after mount.
- Long descriptions and long typed strings use `wrap-anywhere` plus `min-w-0`,
  not `break-words`: only the former lowers the min-content width, which is what
  actually stops a 200-character note from making the panel wider than its
  container. cn() merges the consumer's className into the root, which also
  spreads the remaining element props and forwards its ref.

Customization levers
- Position ownership: leave it uncontrolled for a self-driving viewer, or pass
  `index` + `onIndexChange` to put the step in a URL, a scrubber, or in sync with
  a second panel. Live following is uncontrolled behaviour by design — when you
  own the index, following is your call.
- Density: `showLog=false` turns the component into a bare screenshot stage (the
  caption strip stays) for a compact card; keep the rail for an audit surface.
  Swap the rail's fixed narrow-layout height for a taller one if steps are long.
- Playback: `playbackIntervalMs` sets the beat of a replay; drop the play button
  entirely if your sessions are read, not watched.
- Marker budget: `typePreviewChars` decides how much typed text sits on the
  picture. Raise it for short form fills, lower it to nearly nothing when the
  screenshots matter more than the payload.
- Chrome: `label` names the panel and heads it (put the machine or the run id
  there), `formatTime` swaps the clock, `aspectRatio` matches your fleet's
  resolution so the box never resizes on first load, `emptyState` /
  `errorMessage` + `onRetry` own the envelope states.
- Palette: markers read from --primary and failures from --destructive. Point
  them at a chart token instead (var(--chart-1) etc.) if the session viewer
  needs its own accent without touching the app's primary.