A base corpus in a language or domain
Nothing off the shelf covers the language or the domain at the scale and cleanliness a base model needs.
Use case · LLM and GenAI teams
Data for pretraining, adaptation and post-training
One team usually needs three different things at once: a corpus to build on, a corpus to adapt with, and behaviour data to tune against. They are separate products with separate acceptance criteria.
01
Where teams usually are when they call
your situationEach one points at a different product — the decision helper below turns the symptom into a starting point.
Adapting a model you already have
The base model is fine; it has simply never seen your domain's vocabulary, formats or conventions.
Behaviour data — instructions, preferences, reasoning, tools, safety
The model knows enough and behaves wrong. That is a post-training data problem, not a pretraining one.
02
The products that answer them
what we build for thisLLM Training Data
The umbrella product: eight training stages, one specification, generation separated from validation.
- 8 stages
- umbrella
Q&A & Instruction Data
Grounded Q&A, instruction–response and RAG evaluation records with evidence and lineage.
- grounded
- evidence
Monolingual Corpora
Cleaned, deduplicated, documented text in one language — the base to build or adapt on.
- pretraining
- CPT
Multilingual & Parallel
Coverage across languages, and aligned pairs where translation matters.
- coverage
- pairs
03
Start from the symptom, not the catalogue
decision helper| If you need | Start here | Why |
|---|---|---|
| A base or foundation model | Monolingual / Multilingual corpora | Volume and cleanliness, with a mixture report |
| Domain adaptation | Continued pretraining corpus | Narrow, current, in-domain text — not more of everything |
| Behaviour and instructions | SFT & Q&A data | The unit becomes a conversation; correctness becomes checkable |
| Ranking quality | Preference data | The unit becomes a comparison, with the reason recorded |
| Verifiable tasks | Reasoning / RFT (reinforcement fine-tuning) data | Problems with answers a validator can execute |
| Agents | Tool-use data | Calls, observations, state and the outcome of the run |
| Safety behaviour | Safety data | Boundary prompts with allowed answers and safe alternatives |
| A way to measure any of it | Evaluation holdouts | Private, decontaminated, and never trained on |
04
What we measure on this kind of project
quality controls that matter hereContamination-aware splits
No source family spans training data and the private evaluation set.
how measured · split manifest validation
Token & task distribution
What the mixture actually contains, by task family and language.
how measured · mixture report, per release
Generator / judge separation
The model that wrote a record never approves it.
how measured · registry extract, per record
Bring us a language, a domain or a schema.
We will help define a feasible pilot, quality criteria and a delivery format for your model.