Indexing Progress
An ingest pipeline board — per-source parse → chunk → embed chips, a byte-weighted aggregate with a derived ETA, failures that keep the stage they died at behind a one-shot retry, and a completed summary.
Loading preview…
Installation
npx shadcn@latest add https://ui.zyeon.ai/r/indexing-progress.jsonPrompt
Build a React + TypeScript + Tailwind "IndexingProgress" component with zod and
lucide-react. It is the status board of a knowledge base's ingest pipeline: one
row per source advancing parse → chunk → embed → ready, an aggregate header, and
a summary once the batch settles.
Contract
- A zod schema in a sibling contract file is the single source of truth for one
source: { id, name, size, stage, progress, error?, source?, chunks? }.
- `stage` is a position, NOT an outcome: "queued" | "parse" | "chunk" | "embed" |
"ready" | "skipped". There is deliberately no "failed" stage — a non-blank
`error` IS the failure, and the row keeps the stage it died at so the message
can say "failed while embedding" and a retry knows where to resume. "skipped"
is the real bulk outcome of "content hash unchanged since the last run".
- `progress` is 0–100 WITHIN the current stage, not overall. Backends reset it to
0 on every stage change; that is correct and must not rewind the bar.
- `size` is bytes and doubles as the source's WEIGHT in the aggregate.
- The envelope is separate: status "loading" | "empty" | "error" | "ready", plus
optional `startedAt` / `finishedAt`. `status="error"` means the status endpoint
failed — a different failure from a file failing, and it reads differently.
- Props: status; items; startedAt?; finishedAt?; now? (injected clock);
etaSeconds? (number | null override); stageWeights? (Partial per work stage);
stageLabels? (Partial per work stage); unit? ("file"); maxVisible? (8);
collapseOnComplete? (true); onRetry?(id); onRetryAll?(ids); onReload?;
formatSize?; emptyState?; errorMessage?; label? ("Indexing progress"); plus the
rest of the element props, ref forwarded to the <section>.
Behavior
- OVERALL PROGRESS IS COMPOSED, NEVER TAKEN RAW. A source's fraction is
`sum(weights before its stage) + weight(stage) × progress/100`, with default
weights parse .2 / chunk .15 / embed .65 (embedding dominates because it is the
stage that leaves the machine). Weights are normalized to sum 1, non-finite or
negative entries fall back to the default, an all-zero set falls back to
thirds. Composition is what makes a backend's per-stage reset read as a step
forward instead of a jump back to zero.
- THE BATCH BAR IS BYTE-WEIGHTED. A 40 MB manual at 10% is not the same amount of
remaining work as a 4 KB snippet at 10%, so each source contributes
`max(size, 1)` (an empty file still weighs 1 and cannot vanish). Failed sources
leave the denominator entirely — they can never reach 100%, and keeping them in
would pin the bar below full forever with no way to read why. They are counted
and named in the header instead.
- THE COMPONENT OWNS NO CLOCK. The ETA is recomputed every render as
`elapsed × (1 − fraction) / fraction` from `startedAt` and the injected `now`,
never accumulated. Below 3% done or 1.5 s elapsed it says "estimating time
left" rather than inventing a precise-looking number, and the result is
BUCKETED (5 s / 1 min / 0.1 h) so it stops flickering between 41 s and 38 s.
`etaSeconds` overrides it; `etaSeconds={null}` suppresses it.
- RETRY IS ONE-SHOT PER FAILURE. The guard is a ref written synchronously inside
the click handler (several clicks dispatched in one task all read the same
stale state, so a state-only guard would re-enqueue the same source twice), and
the value stored is the row's FAILURE SIGNATURE (`stage + error`). When the
consumer clears the error and rewinds the stage, the signature changes and the
lock expires by itself — no pruning pass, no effect, and a *second* genuine
failure of the same source is retryable again. "Retry all" fires one call with
every not-yet-locked failed id.
- FAILURES ARE FILTERED, NEVER SORTED. Rows keep the order they came in;
reordering them mid-flight makes the eye lose the file it was watching. A
header toggle (`aria-pressed`) filters down to failures and turns ITSELF off
when the last one clears, so the panel can never show an empty filtered list.
- THE COMPLETED STATE IS A DIFFERENT READING. Once nothing is in flight and
nothing failed, the header switches from "Indexing 24 files" to "Index up to
date" plus totals (files, unchanged, chunks, bytes, `finishedAt − startedAt`)
and the row list collapses — a list of green rows is noise. The collapse is a
ONE-SHOT: it fires on the transition into completion, never again once the
reader has touched the disclosure, and a new batch restores both the list and
the automatic behaviour. A panel that MOUNTS already complete starts collapsed.
- Long lists are capped at `maxVisible` rows with an explicit reveal that states
the remainder ("Show all 24 pages (16 more)"). A cap you can see is a cap you
can trust; a silent one makes you debug the wrong thing.
- Duplicate ids are dropped before render (first occurrence wins): two rows with
the same key would let a retry flip the wrong row into "Retrying…".
- Every numeric input is defended: `progress` is clamped to 0–100, non-finite
sizes count as 0, `maxVisible` is floored to >= 1, and `Infinity` never caps.
- The collapsed list is UNMOUNTED, not hidden: a zero-height collapse leaves
every Retry button reachable by Tab, which is an invisible keyboard trap.
`aria-controls` is only set while open so it never points at a missing id.
Rendering & styling
- Semantic tokens only: bg-card / border / text-muted-foreground for chrome,
bg-primary for fills and completed chips, bg-primary/10 + border-primary/40 for
done, border-dashed + text-muted-foreground for pending, bg-destructive and
text-destructive with border-destructive/50 for failures, bg-muted for tracks
and skeletons. No hard-coded colours anywhere.
- Each running row draws the pipeline as three chips (Parse › Chunk › Embed):
done carries a check, the active one spins and shows its stage percentage,
future ones are dashed, and the broken one turns destructive — the position is
a SHAPE, not only a colour. The strip is aria-hidden because the row's
progressbar already carries the same information as `aria-valuetext`
("Embedding, 62%" / "Failed while embedding"), and reading three pill labels
aloud per row is noise. Settled rows drop the strip for a single
Indexed / Skipped pill: the pipeline is only interesting while a source is in it.
- One persistent sr-only role="status" line, derived from COUNTS ONLY ("18 of 24
files ready, 2 failed"). Wiring it to the percentage would interrupt a screen
reader twice a second; derived from counts it changes only when a source
actually settles.
- Motion is decoration: the spinner and the highlight riding the head of the bar
are `motion-reduce:animate-none` / `motion-reduce:hidden`, and every transition
is motion-reduce-guarded. Nothing about the reading depends on them.
- Names and URLs truncate with the full string in `title`; error text uses
`wrap-anywhere` + `min-w-0` so a 200-character message cannot widen the panel.
- cn() merges the consumer's className into the root <section>, which also
spreads the remaining element props and forwards its ref.
Customization levers
- Vocabulary: `unit` renames the row noun ("file" → "page", "document",
"record"), `stageLabels` renames the pipeline itself ("OCR" / "Split" /
"Vectorize", "Fetch" / "Split" / "Embed"). Both are cosmetic — the arithmetic
and the state machine are unchanged.
- Pacing: `stageWeights` re-balances how much of a row's bar each stage owns.
Measure one real batch and set them; a local embedding model flips the default
ratio completely.
- Time: pass `now` you tick once per second in your app (one interval for the
page, not one per panel), or hand over `etaSeconds` from a backend that models
its own queue, or `null` to say nothing at all.
- Density: `maxVisible` decides how many rows are drawn before the reveal;
`collapseOnComplete={false}` keeps the finished list open in an admin console
where the per-file record is the point.
- Affordances: omit `onRetry` and no Retry appears anywhere; omit `onRetryAll`
and the header falls back to calling `onRetry` per id; omit `onReload` and the
error envelope is read-only. `formatSize` swaps the byte wording (return "" to
drop sizes), `emptyState` replaces the zero-source body, `label` names the region.Concepts
- Stage is a position, failure is a modifier — a
failedstage value would erase the one fact a retry needs: where to resume. Keeping the stage and letting a non-blankerrormark the row lets the panel say "failed while embedding", point the strip at the stage that broke, and hand the consumer a row it can rewind by exactly one step. - Composed progress can't rewind — the bar is
weight before the stage + stage weight × the stage's own percentage, so a backend that resetsprogressto 0 at every stage boundary reads as a step forward. Raw per-stage numbers plotted directly would drop from 98% to 0% three times per file. - Byte-weighted aggregate — completion is weighted by
size, because "3 of 6 done" says nothing when one of the remaining three is a 12 MB scan. Failed sources leave the denominator entirely (they can never reach 100%) and are counted separately, so the bar can honestly reach full while the header still says two files failed. - ETA derived, never measured — the panel owns no clock. It recomputes
elapsed × (1 − fraction) / fractionfrom an injectednowevery render, refuses to guess below 3% done, and buckets the answer, because a countdown that flickers between 41 s and 38 s is noise a reader learns to ignore. - Signature-keyed one-shot lock — the retry guard is a ref (five clicks in one task all read the same stale state) storing the row's
stage + errorsignature. The moment the consumer moves the source on, the signature no longer matches and the lock releases itself — no cleanup pass, and a second genuine failure is retryable again. - Filter, never sort — surfacing failures by moving them to the top would reorder rows under a reader who is watching one file. The failure count is a toggle that filters instead, and it switches itself off when the last failure clears so the panel can never show an empty filtered list.
- Completion is a different reading, collapsed once — a settled batch stops being a list of rows and becomes a sentence with totals. The auto-collapse fires on the transition into completion and never again once the reader has opened the list themselves; a new batch hands both the list and the automatic behaviour back.
Knowledge Picker
A grounding-scope selector for collections, files and links — tri-state collections that cascade only over what is actually selectable, per-source document and chunk counts with index freshness, an accounting bar that prices the selection in chunks, and four data states.
Web Search Card
One web-search step inline in a conversation — the queries the model typed, the pages that came back, which ones it actually read and which ended up cited, with a data-driven reading shimmer and four data states.