Capstone C: choosing the right technique, and graduating
After this capstone you can take any model goal and choose the right technique (or combination) from everything the Academy taught — and you're ready to claim your completion certificate.
You've now seen the whole toolkit, from what a model is to running one in production. The last skill is the meta-skill: given a goal, knowing which tool to reach for. This lesson is that decision guide, and your graduation.
The decision guide: goal → technique
Start from the constraint, not the technique. Map your goal to the lesson that solves it:
GOAL REACH FOR
------------------------------------------- --------------------------------
Make a model do a specific task/format SFT (Tracks 1-2)
"Better"/tone/helpfulness, not one answer DPO / ORPO (4.5)
Match a big model's quality, but small Distillation (4.2-4.4)
Fit a device / cut serving cost Quantization (4.6, 4.8)
One model, several related tasks Multi-task (4.7)
The model needs facts it should look up RAG / reroute (Track 3.9)
Faster / cheaper endpoint Serving (vLLM) (4.8)
Catch quality decay in production Drift checks (4.9)
Don't know if a tweak helps A/B on a gold set (everywhere)They compose — a worked example
Goal: a fast, on-device assistant that matches a frontier model's tone on your domain. No single technique gets there; you stack them:
- Distill the frontier teacher into a small student (4.2–4.4) — size down at high quality retained.
- Preference-tune the student with ORPO (4.5) — align the tone.
- Quantize to GGUF Q4_K_M (4.6) — fit the device.
- Serve efficiently and watch for drift (4.8–4.9) — keep it cheap and honest.
- At every step, evaluate against a gold set and only keep changes that clear the gate.
The one rule under all of them
Every technique in this Academy is judged the same way: measure against a baseline on a held-out gold set, and keep only what clears the gate. That discipline — not any single method — is what separates a model that works from one that merely ran.
What you can do now
You started not knowing what a model was. You can now: explain next-token prediction and the Transformer; build, mask, and tokenize an SFT dataset; write a LoRA training loop by hand and read its curves; drive the entire BrewSLM lifecycle with full understanding of each stage; and choose among distillation, preference tuning, quantization, multi-task, RAG, and serving for a real constraint. That's the full arc from zero to shipping.
Graduate
That completes the BrewSLM Academy. Mark this lesson done, and if you've finished every track, head to your completion certificate to claim it. From here, the best teacher is a real project — pick a task you care about and ship it.
Key idea
Advanced ML is technique selection, then composition: start from the constraint, pick the tool, stack tools when one isn't enough, and judge every step against a gold set. You have the whole toolkit now — go build something and claim your certificate.
Key terms
- technique selection
- Choosing the method that matches a goal/constraint rather than defaulting to one.
- composing techniques
- Stacking methods (distill → preference-tune → quantize → serve) when one isn't enough.
- decision guide
- A goal→technique mapping across the Academy's methods.
- the gold-set rule
- Judge every change against a baseline on a held-out gold set; keep only what clears the gate.
Check yourself
Answers are saved to this browser.