A four-state training-runs card: loss against optimiser step for several concurrent experiments, each a debiased EMA over its own faint raw trace, with divergence and restart markers, an optional learning-rate schedule on a secondary axis, and a scan that reads every run at the same step and says whether the lead survives the noise.
Preview in your theme
Loading preview…
This component is part of Pro
The install command, prompt and concept map are included with a Pro license — one purchase unlocks every Pro component, forever.
status="ready" — four 8B runs logged every 2k steps. The scan rests at 54k, the last step every run reached: WSD + μP leads the baseline by 0.084 nats against ±0.027 of combined step-to-step noise. Drag the smoothing slider: the lines and that noise bar tighten together, while the raw trace they are computed from stays exactly where it is.
Pretraining loss · 8B sweep
2 diverged
4 runs · 2k–120k steps · EMA 0.80 · 2 of 4 diverged
step 54k · 4 of 4 runs logging
8B · cosine 3e-42.076+0.084lr 1.9e-4
8B · cosine 4.5e-42.199+0.206lr 2.9e-4
8B · WSD + μP1.992best herelr 3.0e-4
8B · data mix v32.207+0.215no lr
8B · data mix v3 · divergence at 54k, raw loss 2.589 — loss spike, run abandoned
8B · WSD + μP leads at 54k0.084 against ±0.027 of noise
8B · WSD + μP sits 0.084 below 8B · cosine 3e-4 here, and the two smoothed lines carry a combined ±0.027 at this weight — the lead is bigger than the wiggle. Smoothing harder shrinks that bar; it cannot shrink the step-to-step scatter it is computed from, which is measured off the raw logs.
step budget · 390k steps across 4 runs
8B · cosine 3e-431%
8B · cosine 4.5e-431%
8B · WSD + μP24%
8B · data mix v314%
raw log
smoothed · EMA 0.80
learning rate · right axis
divergence
restart
Each bold line is a debiased exponential moving average at weight 0.80 — roughly the last 5 logs — with the raw log kept faint underneath, because a smoothed line on its own cannot show how much it is hiding. The noise bar under every comparison is σ̂ √((1 − w)/(1 + w)) per run, where σ̂ comes from successive differences of the raw readings: smoothing tightens the reading without touching the data. Shares of the step budget are apportioned by largest remainder, so they add to exactly 100%. Point at the plot, or tab in and use the arrow keys, to read any step.
2 of 4 runs hit a divergence. 8B · cosine 4.5e-4 restarted 6k steps later. 8B · data mix v3 never came back and stops at 54k. The shaded band is those 6k steps — logged, paid for and thrown away.
Training runs: train loss · nats against optimiser step for 4 runs, steps 2k to 120k. Each line is a debiased exponential moving average at weight 0.80, about 5 logs, drawn over the raw trace. At 54k, 8B · WSD + μP leads 8B · cosine 3e-4 by 0.084, clear of the 0.027 the step-to-step scatter allows. 2 of 4 runs hit a divergence. 8B · cosine 4.5e-4 restarted 6k steps later. 8B · data mix v3 never came back and stops at 54k. The shaded band is those 6k steps — logged, paid for and thrown away. Exact values follow in the data table. Up to 14 steps, evenly spaced by rank; values are smoothed.
Step
8B · cosine 3e-4
8B · cosine 4.5e-4
8B · WSD + μP
8B · data mix v3
Leads
2k
2.598
2.623
2.508
2.692
8B · WSD + μP
12k
2.421
2.406
2.351
2.461
8B · WSD + μP
20k
2.292
2.296
2.233
2.357
8B · WSD + μP
30k
2.199
2.211
2.134
2.241
8B · WSD + μP
38k
2.143
2.163
2.081
2.184
8B · WSD + μP
48k
2.105
2.287
2.038
2.129
8B · WSD + μP
56k
2.064
2.169
1.987
ended
8B · WSD + μP
66k
2.022
2.070
1.958
ended
8B · WSD + μP
74k
2.001
2.035
1.960
ended
8B · WSD + μP
84k
1.980
1.999
1.948
ended
8B · WSD + μP
92k
1.956
1.972
1.929
ended
8B · WSD + μP
102k
1.951
1.958
ended
ended
level within noise
110k
1.929
1.938
ended
ended
level within noise
120k
1.917
1.950
ended
ended
level within noise
the same four runs at smoothing 0.30 — the lines sit almost on top of their raw traces and the noise bar doubles to ±0.059, which is what the weight actually buys. The hot 4.5e-4 run spiked at 46k and came back 6k steps later (the shaded band is those steps); data mix v3 spiked at 54k and never restarted, so its line simply stops. Step shares print 31 / 31 / 24 / 14 — the exact figures are 30.77, 30.77, 24.62 and 13.85, and rounding each on its own would print 101%.
Pretraining loss · 8B sweep
2 diverged
4 runs · 2k–120k steps · EMA 0.30 · 2 of 4 diverged
step 54k · 4 of 4 runs logging
8B · cosine 3e-42.059+0.115lr 1.9e-4
8B · cosine 4.5e-42.095+0.151lr 2.9e-4
8B · WSD + μP1.944best herelr 3.0e-4
8B · data mix v32.436+0.492no lr
8B · data mix v3 · divergence at 54k, raw loss 2.589 — loss spike, run abandoned
8B · WSD + μP leads at 54k0.115 against ±0.059 of noise
8B · WSD + μP sits 0.115 below 8B · cosine 3e-4 here, and the two smoothed lines carry a combined ±0.059 at this weight — the lead is bigger than the wiggle. Smoothing harder shrinks that bar; it cannot shrink the step-to-step scatter it is computed from, which is measured off the raw logs.
step budget · 390k steps across 4 runs
8B · cosine 3e-431%
8B · cosine 4.5e-431%
8B · WSD + μP24%
8B · data mix v314%
raw log
smoothed · EMA 0.30
learning rate · right axis
divergence
restart
Each bold line is a debiased exponential moving average at weight 0.30 — roughly the last 1.4 logs — with the raw log kept faint underneath, because a smoothed line on its own cannot show how much it is hiding. The noise bar under every comparison is σ̂ √((1 − w)/(1 + w)) per run, where σ̂ comes from successive differences of the raw readings: smoothing tightens the reading without touching the data. Shares of the step budget are apportioned by largest remainder, so they add to exactly 100%. Point at the plot, or tab in and use the arrow keys, to read any step.
2 of 4 runs hit a divergence. 8B · cosine 4.5e-4 restarted 6k steps later. 8B · data mix v3 never came back and stops at 54k. The shaded band is those 6k steps — logged, paid for and thrown away.
Training runs: train loss · nats against optimiser step for 4 runs, steps 2k to 120k. Each line is a debiased exponential moving average at weight 0.30, about 1.4 logs, drawn over the raw trace. At 54k, 8B · WSD + μP leads 8B · cosine 3e-4 by 0.115, clear of the 0.059 the step-to-step scatter allows. 2 of 4 runs hit a divergence. 8B · cosine 4.5e-4 restarted 6k steps later. 8B · data mix v3 never came back and stops at 54k. The shaded band is those 6k steps — logged, paid for and thrown away. Exact values follow in the data table. Up to 14 steps, evenly spaced by rank; values are smoothed.
Step
8B · cosine 3e-4
8B · cosine 4.5e-4
8B · WSD + μP
8B · data mix v3
Leads
2k
2.598
2.623
2.508
2.692
8B · WSD + μP
12k
2.326
2.340
2.287
2.407
level within noise
20k
2.205
2.209
2.164
2.291
level within noise
30k
2.158
2.154
2.077
2.196
level within noise
38k
2.118
2.119
2.038
2.159
8B · WSD + μP
48k
2.093
2.478
2.001
2.080
level within noise
56k
2.030
2.065
1.960
ended
8B · WSD + μP
66k
1.990
2.013
1.924
ended
8B · WSD + μP
74k
1.979
2.014
1.954
ended
level within noise
84k
1.987
1.997
1.922
ended
8B · WSD + μP
92k
1.933
1.943
1.903
ended
level within noise
102k
1.934
1.954
ended
ended
level within noise
110k
1.907
1.917
ended
ended
level within noise
120k
1.898
1.949
ended
ended
level within noise
one run, no schedule logged and nothing to compare against — the right axis, the event glyphs and the ranking all drop out, the budget strip is a single 100% segment, and the readout falls back to the raw log beside the smoothed one.
Context-extension anneal · 3B
1 run · 2k–40k steps · EMA 0.60 · no divergences
step 40k · 1 of 1 run logging
3B · 128k ctx anneal1.242raw 1.262
step budget · 40k steps across 1 run
3B · 128k ctx anneal100%
raw log
smoothed · EMA 0.60
Each bold line is a debiased exponential moving average at weight 0.60 — roughly the last 2.5 logs — with the raw log kept faint underneath, because a smoothed line on its own cannot show how much it is hiding. The noise bar under every comparison is σ̂ √((1 − w)/(1 + w)) per run, where σ̂ comes from successive differences of the raw readings: smoothing tightens the reading without touching the data. Shares of the step budget are apportioned by largest remainder, so they add to exactly 100%. Point at the plot, or tab in and use the arrow keys, to read any step.
Training runs: train loss · nats against optimiser step for 1 run, steps 2k to 40k. Each line is a debiased exponential moving average at weight 0.60, about 2.5 logs, drawn over the raw trace. Exact values follow in the data table. Up to 14 steps, evenly spaced by rank; values are smoothed.
Step
3B · 128k ctx anneal
Leads
2k
1.535
—
4k
1.473
—
8k
1.436
—
10k
1.420
—
14k
1.371
—
16k
1.369
—
20k
1.313
—
22k
1.309
—
26k
1.277
—
28k
1.256
—
32k
1.271
—
34k
1.246
—
38k
1.228
—
40k
1.242
—
status="loading" — three descending traces and a tick column, so the skeleton is the shape of the thing being loaded rather than a grey box
Pretraining loss · 8B sweep
train loss · nats against optimiser step, one smoothed line per run over its raw trace
Loading training runs
status="empty" — no runs yet, worded so it cannot be read as a failed fetch
Pretraining loss · 8B sweep
train loss · nats against optimiser step, one smoothed line per run over its raw trace
No runs to plot yet
This card compares experiments against optimiser step, so it needs at least one run with two logged readings — a single reading is a number, not a curve, and nothing can be smoothed, ranked or marked on it.
status="error" — Try again is wired to a real onRetry and flips the card to ready
Pretraining loss · 8B sweep
train loss · nats against optimiser step, one smoothed line per run over its raw trace
Zyeon UI