What we do
Acquisition, extraction, classification, cleaning, enrichment, QA — each with its own inputs, outputs and quality gate.
Services
Data engineering operations behind every dataset
Acquisition, extraction, classification, cleaning and documentation — configured per language, domain and intended use.
01
Operations, not products
how to read this pageA service is something we do to data. A product is something you receive. Every dataset on the Datasets pages is assembled from these six operations, configured differently — which is why the same six appear whether you bring sources or ask us to find them.
What you receive
A corpus, a parallel set, a post-training dataset or a custom build — plus the package that documents it.
02
What happens to your data, in order
the six operationsAcquisition: mining, crawling & ingestion
Source discovery and scoring, focused crawling under an approved policy, and file ingestion — with provenance captured at URL level.
Web Data Mining & Crawling →Extraction & normalization
Format-aware extraction from HTML, PDF and office documents: reading order, tables, formulas and captions, then normalization to a single document shape.
Scope confirmed per project
Classification
Cascaded language identification, script validation and hybrid domain classification — calibrated per project, with explicit review bands.
Domain & Language Classification →Cleaning & deduplication
Six independent layers, each with a defect type, a decision, a metric and a machine-readable reason code; dedup runs as an ordered pipeline.
Data Cleaning & Curation →Annotation & enrichment
Entities, terminology, citations and metadata added against your taxonomy, with guidelines and reviewer agreement recorded.
Scope confirmed per project
QA, documentation & delivery
Automated validation, human sampling, the deliverables package and a versioned release with a changelog.
Scope confirmed per project
03
Bring your own sources, or ask us to discover them
two ways to startThe operations are the same. What changes is where the source policy comes from and how much of the registry we build.
| Row label | Bring your own sources | Ask us to discover sources |
|---|---|---|
| Starting point | You hand us a source list, an archive or a live feed | We propose candidate sources against your language, domain and risk criteria |
| Source policy | Yours — we record it and work inside it | Drafted together, then fixed as the source registry before crawling |
| Provenance captured | What your sources carry, plus everything the pipeline adds | Full URL-level capture from the first fetch |
| Scoping dependency | Source condition, format diversity and access method | Discovery breadth, source-policy review and crawl feasibility |
| What you receive | Curated release, quality report and rejection summary; record-level rejects where rights and the agreement permit | The same package, plus the source registry with include, exclude and review decisions |
Bring us a language, a domain or a schema.
We will help define a feasible pilot, quality criteria and a delivery format for your model.