From a folder of documents
to a model that answers.
Four stages, none of which ask you to provision a machine. What follows is what
actually happens between penstock push and an endpoint you can curl.
The four stages
Stage one · Ingest
Point at a folder or a bucket. We parse PDFs, HTML, Markdown, transcripts and CSV, strip boilerplate, and run near-duplicate detection across the whole corpus rather than per file.
You get a report of what was dropped and why before a single GPU-hour is billed. In most corpora, between 12% and 30% of the tokens are duplicates of something else in the set.
Stage two · Tokenise
Tokenisation uses the base model's own vocabulary, so nothing is re-learned that the base already knows. Sequence packing fills context windows instead of padding them, which is where a third of naive training budgets quietly go.
Stage three · Train
The scheduler bids for interruptible capacity across regions and checkpoints often enough that a pre-emption costs minutes, not the run. LoRA by default; full fine-tune when the task needs the whole weight matrix moved.
You watch loss curves live. If a run is going nowhere, kill it and you pay for what burned.
Stage four · Evaluate
Every run ends with a held-out evaluation against the base model on your own data, not a public benchmark. The number that matters is whether the tune beat the thing you already had.
Ship it to an endpoint, or export the weights and run them yourself. Both are one command.
What it costs to be wrong
| Stage | Wall clock | Billed | What you can still change |
|---|---|---|---|
| Ingest & dedup | 18 min | $0 | Everything. Nothing has trained yet. |
| Tokenise | 6 min | $0 | Corpus, packing, base model. |
| Train | ~4 h on 24 GPUs | $180 | Kill it and pay only the hours burned. |
| Evaluate | 11 min | $0 | Re-run the eval set as often as you like. |