Documentation

Distillery docs

Everything you need to distill a large model into a small one you can prove: datasets, recipes, runs, and the gate that decides what ships.

01 · Quickstart

Three lines to a sealed distillation

Point Distillery at a dataset, pick a recipe, and wait for the proof report. Everything else, teacher labeling, manifest sealing, training, evaluation, happens inside the run.

quickstart.py

from distillery_sdk import Distillery

distillery = Distillery(base_url="https://distillery.fly.dev")
dataset = distillery.datasets.generate(corpus="smoke")
run = distillery.distill(dataset, recipe="auto").wait()
print(run.proof_report)

02 · Datasets

Immutable, hashed, split-frozen

Datasets are immutable resources with frozen train, validation, and test splits. Teacher labels fill only missing training responses, carry full provenance, and must pass deterministic validation. Any curation produces a new dataset id, never a mutation.

datasets.py

dataset = distillery.datasets.create("./traces.jsonl")
meta = distillery.datasets.get(dataset["dataset_id"])
stats = distillery.datasets.synthesize(
    dataset["dataset_id"], mode="teacher", dry_run=True)

03 · Recipes

Bring your own distillation method

A recipe is ordinary Python written against five synthesis primitives. Registered recipes run through the same resolver and capability gates as the built-ins. A recipe that demands something the backend cannot provide fails loudly with RECIPE_INCOMPATIBLE. It never silently downgrades.

recipes.py

from distillery.recipes.custom import SynthesisContext, register

class RejectHard:
    name = "reject_hard.v1"
    requires = frozenset({"teacher"})
    description = "Drop hard cases the teacher got wrong."

    def run(self, ctx: SynthesisContext, examples):
        ctx.teacher_label(examples)
        return [ex for ex in examples
                if ctx.is_valid(ex) and ctx.agrees_with_oracle(ex)]

register(RejectHard())

04 · Runs

Plan first, then one finite job

Every run starts with a plan: resolved recipe, cost estimate, blockers. Submission seals a manifest with dataset hashes, model pins, and training config. Training executes from the manifest on local hardware or SageMaker and produces an artifact plus checksums.

runs.py

plan = distillery.plan(dataset, recipe="rejection_sampling.v1")
run = distillery.distill(
    dataset, recipe="rejection_sampling.v1", max_run_usd=25.0)
run.wait()

05 · The proof gate

Pass, or BLOCKED

Candidates are evaluated on a frozen held-out set the training process never touched: schema validity, decision-field accuracy against the executable oracle, cost ratio versus the teacher, and break-even request count. In our reference experiment the distilled 0.5B student scored 55% schema validity and 25% decision-field accuracy, equal to its teacher on decisions, at roughly a thousandth the size.

proof.py

report = run.proof_report
assert report["passed"], "gate says BLOCKED, nothing ships"

06 · Local demos

See it on your own machine

The repo ships a terminal comparison (base model versus distilled, same weights, adapter toggled) and a browser playground. Both validate every output against the oracle and keep a running scoreboard.

local.py

PYTHONPATH=.:src .venv/bin/python examples/tui_demo.py
PYTHONPATH=.:src .venv/bin/uvicorn examples.compare_demo:app --port 8010

Next: read the experiment or the product page.