Clean and prepare: the manifest is the source of truth
After this lesson you can run the optional clean stage (Data → Prepare), understand what the Split step produces, and explain why the manifest — not raw files — is the single source of truth for training and evaluation.
In Track 2 you tokenized examples, built the loss mask, and split train/val by hand. BrewSLM folds all of that into two stages whose output is one authoritative artifact: the manifest.
05 Clean (optional)
If your rows are raw documents rather than tidy pairs, the clean stage processes them: it emits cleaned text, chunk JSONL, and per-document metadata — PII findings, a quality score, and a text hash. In the workspace this is Data → Prepare (/project/<id>/data?step=cleaning): set the chunk size, tick redact PII & secrets, press Clean All Documents; the quality and safety scans sit in the step's Details drawer. It runs as a background task with task_id polling so a big cleaning pass never blocks past the proxy timeout, and it has explicit failure modes: cleaning_pii_block and cleaning_outlier_threshold_exceeded. For already-clean classification pairs like our sentiment data, you can skip it.
06 Prepare — build the manifest
Prepare is the Data → Split step (/project/<id>/data?step=dataprep, the Train / Val / Test Split card). It takes your cleaned + synthetic rows and the project's split adapter and produces:
prepared/
manifest.json # the source of truth: counts, schema, task_profile,
# output_schema.scoring_mode, paths, hashes
train.jsonl # the training split
val.jsonl # the validation split
test.jsonl # the held-out test split ("test examples" in the UI)
versions/<v>/ # a snapshot of each split runThat train.jsonl / val.jsonl / test.jsonl split is the platform doing your train_test_split from Track 2 — and the held-out test.jsonl is what the pass/fail rules will score against later (Results → Verdict). Rows are deduplicated as they are split; a later re-split can keep the current test examples so new data never moves the answer key into training; and the dataset versions in the step's Details drawer let you make an earlier split active again.
From Track 2: where did tokenization and the loss mask go?
They didn't disappear — they moved into the task handler at train time. The manifest records the task_profile and scoring_mode; the handler for that task knows how to apply the chat template, build input_ids/labels, and mask the prompt — exactly the mechanics you wrote in lesson 2.4. You declared the what; the handler does the how.
The manifest is the contract
The single most important rule downstream: nothing reads from disk paths directly — everything reads the manifest. Train, Evaluate, and Export all consume manifest.json. That's why a BrewSLM run is reproducible in a way your script wasn't: the manifest pins exactly which rows, which schema, which scoring mode, and which hashes went into the run. Change the data and you get a new manifest, not a silently different result from the same code.
Key idea
The Split step turns reviewed rows into a manifest plus train / val / test splits. The manifest carries the task profile and scoring mode and is the source of truth for every later stage — so the tokenization and masking you did by hand become a declaration the handler executes, reproducibly.
Key terms
- Clean stage
- Optional processing of raw documents into cleaned text + chunks + PII/quality metadata (Data → Prepare).
- manifest
- prepared/manifest.json — the authoritative record of rows, schema, task_profile, scoring_mode, paths, and hashes.
- train/val/test split
- train.jsonl, val.jsonl and test.jsonl produced by Data → Split; the platform's train_test_split. The test split is the UI's "test examples".
- task_profile
- The declared task type recorded in the manifest, used to pick the handler.
- scoring_mode
- How outputs are scored, recorded in the manifest's output_schema.
Check yourself
Answers are saved to this browser.