penstock

Documentation

Everything below runs against the same API the dashboard uses. If a thing is possible in the UI and not here, that is a bug — tell us.

Install

The CLI is a single static binary with no runtime dependency. It targets macOS and Linux on both architectures.

# macOS and Linux
curl -fsSL https://get.penstock.dev | sh

# or, if you would rather see what you are running
curl -fsSL https://get.penstock.dev -o install.sh && less install.sh

Verify with penstock version. The binary self-updates only when you ask it to.

Quickstart

From an empty directory to a served endpoint, assuming a folder of documents:

penstock login
penstock push ./handbook # upload + dedup + tokenise
penstock train --base penstock-8b --method lora
penstock eval  --run last
penstock serve --run last --name handbook-8b

Nothing bills until train. Push, dedup, tokenise and every evaluation after a run are free; you are charged per GPU-hour at the rate the scheduler won.

Authentication

penstock login opens a browser and writes a scoped token to ~/.penstock/credentials. For CI, mint a machine token in the dashboard and export it as PENSTOCK_TOKEN; the CLI prefers the environment variable when both exist.

Tokens are scoped to a project and can be restricted to read, train or serve. A read token cannot start a run, which is the scope you want in a build pipeline that only fetches artefacts.

Ingesting a corpus

push parses, strips boilerplate, and runs near-duplicate detection across the whole corpus. It prints what it dropped:

$ penstock push ./handbook
  parsed      1,284 files      412 MB
  dropped       218 files      near-duplicate (>0.94 cosine)
  dropped        31 files      under 40 tokens after boilerplate strip
  kept        1,035 files      238.4M tokens

Pass --dry-run to get that report without uploading, and --keep-duplicates if the repetition is meaningful in your domain — legal boilerplate sometimes is.

Running a training job

LoRA is the default because it is right most of the time. Reach for --method full when the task changes what the model knows rather than how it answers.

penstock train \
  --base penstock-8b --method lora \
  --epochs 3 --lr 1e-4 \
  --eval-split 0.05

--epochs: more than 3 on a small corpus usually memorises rather than learns.
--eval-split: held out before training, never seen by the optimiser.
--resume <run>: picks up from the last checkpoint after a pre-emption.

Runs checkpoint every few minutes, so an interrupted spot instance costs minutes rather than the job. penstock logs --follow streams the loss curve to your terminal.

Evaluating a run

Evaluation compares the tuned model against its own base on your held-out split. A tune that does not beat the base is a result, not a failure. It usually means the corpus is too small or too close to what the base already knew.

$ penstock eval --run last
  base   penstock-8b     exact 0.612   f1 0.703
  tuned  handbook-8b     exact 0.781   f1 0.844
  delta                  +0.169        +0.141

Serving and export

serve puts the run behind an OpenAI-compatible endpoint, so existing client code works with a base-URL change and nothing else. export writes safetensors you can run anywhere.

penstock serve  --run last --name handbook-8b
penstock export --run last --out ./weights

CLI reference

Command	What it does	Bills
`login`	Writes a scoped token to the credential file	No
`push`	Parse, dedup, tokenise, upload	No
`train`	Schedule a run on interruptible capacity	Per GPU-hour
`logs`	Stream loss curves and scheduler events	No
`eval`	Score a run against its base on held-out data	No
`serve`	Expose a run as an OpenAI-compatible endpoint	Per token
`export`	Download safetensors for the run	No

HTTP API

Everything the CLI does is one HTTPS call. Base URL https://api.penstock.dev/v1, bearer token in the Authorization header.

$ curl https://api.penstock.dev/v1/runs \
    -H "Authorization: Bearer $PENSTOCK_TOKEN" \
    -d '{"base":"penstock-8b","corpus":"handbook","method":"lora"}'

{ "run": "run_8fQ2", "state": "queued", "eta_s": 240 }

Limits & quotas

Corpus upload — 50 GB per push, no cap on total stored tokens.
Concurrent runs — 2 on Flow, 12 on Reserved, unlimited on Enterprise.
Endpoint throughput — 600 requests/minute per served run, raised on request.
Checkpoint retention — 90 days, then only the final weights are kept.

Security

Corpora are encrypted at rest and never used to train anything but your own runs. Weights you export are yours; we keep no copy after the retention window. Machine tokens can be revoked individually without rotating the rest.

Changelog

2026‑07‑22: sequence packing on by default; typical run ~19% cheaper.
2026‑06‑30: penstock export writes safetensors instead of pickles.
2026‑05‑14: 256K context on the 34B base.
2026‑04‑02: near-duplicate detection moved from per-file to corpus-wide.