A four-state training-mixture card: one composition bar per training stage split by data domain, with per-domain epochs so upsampling reads as the gap between unique tokens and tokens seen.
Preview in your theme
Loading preview…
This component is part of Pro
The install command, prompt and concept map are included with a Pro license — one purchase unlocks every Pro component, forever.
Replay reshapes the mix most for Math & reasoning: 5.9% of unique tokens against 12.6% of tokens seen, +6.7 points. 5 of 6 domains are passed over more than once.
Segment length is the domain's share of unique tokens in that stage, apportioned by largest remainder so each bar totals exactly 100%. Point at a domain, or Tab into the key and press Enter, to trace it through every stage; Escape clears it. The switch re-apportions every bar between the corpus as it exists and the corpus as the optimiser saw it.
Training data mixture · 32B run — 4 stages totalling 9.72T unique tokens and 13.61T tokens seen, a replay factor of 1.40. Every row carries its stage, its unique tokens, its epoch count, the tokens that combination puts through the optimiser, and its share of the stage under both measures.
Stage
Domain
Unique tokens
Epochs
Tokens seen
Share of unique
Share of seen
Pretrain
Web crawl
5.40T
1
5.40T
60.0%
44.9%
Pretrain
Code
1.15T
2
2.30T
12.8%
19.1%
Pretrain
Math & reasoning
420B
3
1.26T
4.6%
10.5%
Pretrain
Books & papers
610B
2
1.22T
6.8%
10.1%
Pretrain
Multilingual
980B
1
980B
10.9%
8.1%
Pretrain
Synthetic & instruction
440B
2
880B
4.9%
7.3%
Mid-train
Web crawl
120B
1
120B
16.9%
7.7%
Mid-train
Code
190B
2
380B
26.8%
24.4%
Mid-train
Math & reasoning
150B
3
450B
21.1%
28.8%
Mid-train
Books & papers
60.0B
2
120B
8.4%
7.7%
Mid-train
Multilingual
80.0B
2
160B
11.3%
10.3%
Mid-train
Synthetic & instruction
110B
3
330B
15.5%
21.1%
SFT
Code
1.15B
2
2.30B
27.4%
25.1%
SFT
Math & reasoning
780M
3
2.34B
18.6%
25.5%
SFT
Multilingual
420M
2
840M
10.0%
9.1%
SFT
Synthetic & instruction
1.85B
2
3.70B
44.0%
40.3%
Preference
Code
260M
1
260M
27.7%
22.6%
Preference
Math & reasoning
210M
2
420M
22.3%
36.5%
Preference
Multilingual
90.0M
1
90.0M
9.6%
7.8%
Preference
Synthetic & instruction
380M
1
380M
40.4%
33.1%
empty
Training data mixture · 32B run
Token mix per training stage, with the epoch count that decides how often each domain is replayed.
No mixture recorded
A mixture needs at least one stage carrying domains with token counts. Epochs are what make it worth drawing: a domain with 3 epochs puts three times its own tokens through the optimiser, so the mixture the model read is not the mixture on disk.
loading
Training data mixture · 32B run
Token mix per training stage, with the epoch count that decides how often each domain is replayed.
Loading the training data mixture
error
Training data mixture · 32B run
Token mix per training stage, with the epoch count that decides how often each domain is replayed.
Zyeon UI