Engineering blog

Engineering notes on small language model fine-tuning infrastructure.

These posts support the product and the Academy: dataset ingestion, task-aware evaluation, and design decisions behind local, auditable small language model workflows. No content mill, just the implementation details that matter.

All posts

2026 · Build · Structured extraction

Narrative in, JSON out: a 1.5B accident-report extractor that knows when to say null

NTSB aviation accident narratives to a 13-field JSON record. On 1,234 held-out reports a fine-tuned Qwen2.5-1.5B gets 0.959 of fields exactly right (base: 0.494) and writes null for 0.976 of the fields the text doesn't state. The rule written first, all three seeds, where it still misses, and the model on the Hugging Face Hub.

Read

2026 · Part 2 · Training a passage reader

Teaching a 1.5B model to read Pub 17: two misses, one pass

The sequel. Fine-tuned to answer from retrieved passages, a 1.5B model beat the same base model reading the same passages: 0.688 vs 0.570 on 157 questions fixed before any scoring, all three seeds better. The two attempts that missed, the fixes they forced, the caveats, and the model on the Hugging Face Hub.

Read

2026 · Walkthrough · Retrieval vs fine-tuning

The check said don't fine-tune. We did anyway. It was right.

IRS Publication 17 as a Q&A assistant. Before training, BrewSLM scored the base model with and without the publication's passages on 41 held-out questions: 0.17 vs 0.73, and stopped. The fine-tune we ran anyway scored 0.02 while its token F1 doubled. Every number, the reroute to retrieval, and what comes next.

Read

2026 · Walkthrough · Span extraction

Build a sensitive-data detector from zero, with no labelled data

PCI, GDPR, HIPAA and your own ids, tagged with exact character offsets — from an empty project and a local Ollama to a Hugging Face model. Every step in BrewSLM, every number real, including the second-opinion key that took the score from 0.88 to 0.74, the label fixes that brought it to 0.85, and multi-page documents with headers and footers.

Read

2026 · Engineering · Dataset pipeline

Why the schema introspector beats hand-written converters

We shipped a 175-line BIO-to-spans converter for the Kaggle PII dataset. Then we threw it away and built a column sniffer. Here's why the sniffer wins.

Read

2026 · Engineering · Evaluation

Inside the task-aware eval handler dispatcher

One generic "pass rate" metric flatters classification and lies about NER. The story of a real eval bug, how the dispatcher fixes it, and what each task handler does that the others can't.

Read

2026 · Product · Design

Gamifying a dev tool without making it a toy

Most gamified developer tools feel infantilizing. We tried it anyway. The Lab Journal progression layer, the design rules we kept, what we refused to ship — and a postscript on why we removed it in October 2026.

Read

Want to write one?

Pitch a post via GitHub Issues

If you've used BrewSLM in production and found something interesting — a workflow insight, a metric you wish existed, a bug story worth retelling — open an issue on the repo with the title and a paragraph. We'll either point you at the right place to publish or run it here with attribution.

$ open https://github.com/TensorGreed/__SLM__/issues/new