Skip to content
Discuss your dataset

Datasets · child product

Grounded Q&A and instruction data

Question–answer, instruction–response and RAG evaluation records with evidence, lineage and separate validation.

Request a pilot
01

Nine shapes this product takes

product forms
  • grounded Q&A
  • extractive QA
  • abstractive QA
  • multi-hop QA
  • instruction–response / SFT
  • multi-turn conversations
  • RAG evaluation
  • negative / adversarial examples
  • structured outputs

RAG evaluation records carry answerable and unanswerable questions, citation targets and relevance labels — an evaluation set without unanswerable questions measures fluency, not retrieval.

02

Generation is not validation

the core rule
Generation and validationThe generation column produces records — human, synthetic or hybrid — and tags each with a creation method. The validation column checks them with rule checks, a judge model and human review. The two columns meet at a shared validation_status field; neither column can set the other’s field.GenerationHuman authoredSynthetic generationHybrid, creation method taggedValidationRule checksJudge modelHuman reviewvalidation_statusNeither lane can set the other’s fieldGeneration and validationThe generation column produces records — human, synthetic or hybrid — and tags each with a creation method. The validation column checks them with rule checks, a judge model and human review. The two columns meet at a shared validation_status field; neither column can set the other’s field.GenerationHuman authoredSynthetic generationHybrid, creation method taggedvalidation_statusValidationRule checksJudge modelHuman reviewNeither lane can set the other’s field
03

What a record has to carry

required fields

A record missing any of these cannot be audited later — which is the same as not being validated at all.

  • question
  • answer
  • context / context IDs
  • source / document / chunk lineage
  • answerability label
  • evidence span / citation IDs
  • language / domain / difficulty
  • creation_method
  • validation_status
  • split
  • generator + prompt/template version
  • decoding params + seed (where disclosure permits)
  • judge model / rule checks
  • human review status
04

One grounded Q&A record

sample schema

JSONL

{
  "id": "dp_qa_000001",
  "question": "What is the stated warranty period?",
  "answer": "The warranty period is 24 months.",
  "context_id": "doc_204#chunk_07",
  "evidence": ["sent_03"],
  "answerable": true,
  "language": "eng",
  "domain": "product_support",
  "difficulty": "direct",
  "creation_method": "hybrid",
  "validation_status": "reviewed",
  "split": "train"
}

Illustrative — schema; no customer or production data.

05

Ten checks, each with its own failure mode

quality checks
check

Groundedness

The answer is supported by the cited span.

check

Correctness

The answer is right, judged against the source.

check

Relevance

The question is one a real user would ask of this document.

check

Uniqueness

Near-duplicate questions are clustered and thinned.

check

Ambiguity

Questions with more than one defensible answer are caught.

check

Citation validity

Every evidence ID resolves to real text.

check

Answerability

Unanswerable questions are labelled, not removed.

check

Style compliance

Format and register match the specification.

check

Safety

Boundary content is labelled per policy.

check

Distribution balance

Coverage across difficulty, domain and document.

06

Two rules that do not bend

07

What you provide

  • The documents, or the corpus the questions should be grounded in
  • What a good answer looks like — length, register, citation style
  • Whether synthetic generation is permitted, and which models are approved
  • Your evaluation holdout situation
  • Difficulty bands, if you have them; otherwise we propose them
  • Any domain expert who will adjudicate disagreements
parent

LLM Training Data

The umbrella product: pretraining through evaluation, eight stages.

LLM Training Data →

Frequently asked

questions we are asked

Only after they have passed validation that the generator had no part in. Synthetic generation is a production approach, recorded on the record as creation_method — it is not a quality claim, and it never sets validation_status.

Difficulty bands are calibrated against a reviewed sample rather than asserted, using the bands you provide or ones we propose against your document set.

Yes. Every record can be grounded exclusively in customer-provided documents, with no external source contributing to the answer or the evidence span.

A question the provided context does not support an answer to. It is labelled answerable: false rather than removed, so the model is trained and evaluated on knowing what it cannot answer.

A named domain expert, agreed with you before review starts, resolves disagreements that reviewers cannot settle between themselves.

Bring us a language, a domain or a schema.

We will help define a feasible pilot, quality criteria and a delivery format for your model.

Discuss your datasetRequest a pilot