Clawdemy Lessons

Brief for the lesson on BERT's two derivatives: DistilBERT shrinks BERT via distillation; RoBERTa retrains it without NSP for better quality.

Brief for the attention efficiency lesson: prerequisites, learning outcomes, and how sliding window attention and the MHA, MQA, GQA progression fit Phase 2.

How supervised fine-tuning (SFT) bridges a base model to an assistant: what it changes (response shape) versus keeps (knowledge), plus LoRA and its limit.

Brief for the transformer block lesson: what you'll learn, where it fits in Phase 2, prerequisites, and the read and practice time.

Brief for the multi-head attention lesson: why one head must choose, the split-run-concatenate pattern, the 12-head 768-dim flow, plus MQA, GQA, MLA variants.

Brief for the text generation lesson: what it covers (autoregressive prediction loop, decoding strategies, KV caching), prerequisites, outcomes, and timing.

Overview of the embeddings lesson: how token IDs become meaning-carrying vectors, where it fits in Phase 1, prerequisites, and read and practice time.

Overview of the tokenization lesson: why models read tokens not words or letters, what byte-pair encoding does, where it fits in Track 5, and learning goals.

Overview of the self-attention lesson: what it covers, where it fits in Track 5 Phase 2, the prerequisites, and the learning outcomes before you start.

Overview of the human delta lesson: what you'll learn, why AI amplifies rather than replaces capability, prerequisites, and the time it takes.