Clawdemy Lessons

Brief for the BERT training lesson: why bidirectionality forced MLM and NSP, the two-stage train-then-fine-tune workflow, and the CLS vs per-token heads.

Brief for BERT's architecture lesson: the encoder-only branch, bidirectional self-attention, the CLS and SEP tokens, and three additive input embeddings.

Brief for the tool-use failures lesson: the three buckets, nine named sub-failures, where it fits in Phase 7, prerequisites, and time and difficulty.

A brief on benchmark literacy: the major benchmark categories, what each measures, and the structural reasons scores can rise faster than real capability.

Overview of the AI safety recap lesson: the per-phase threads it covers, where it fits in the foundations track, prerequisites, and learning outcomes.

Brief for the lesson on transformer adaptations beyond text: Vision Transformers (ViT) for images and Mixture-of-Experts (MoE) for sparse parameter scaling.

Brief for the RLHF and DPO lesson: what you'll learn, where it fits in Phase 4, prerequisites, and the learning outcomes on PPO, DPO, and reward hacking.

Brief for the lesson on alternatives to autoregressive generation: what speculative decoding and diffusion LLMs are, how each works, and when each fits.

Lesson brief: in-context learning and few-shot prompting, with learning outcomes, prerequisites, where it fits in Phase 5, and time and difficulty.

Brief overview of the LLM-as-a-Judge lesson: what LaaJ is, pointwise vs pairwise, why it works, and the three biases plus prerequisites and time.

Brief for the reasoning-models lesson: what they are, how they differ from standard LLMs, the benchmarks (AIME, GSM8K, HumanEval, SWE-bench), and Pass@K.

Phase 1 brief on why models need word order added explicitly, plus the learned and sinusoidal position schemes the 2017 transformer paper proposed.

A brief on function calling: how it gives an LLM structured data and actions, the three-stage mechanism, what the model sees, and how it is trained.

Course brief for the agent loops lesson: scope, prerequisites, learning outcomes, and how it fits as the Phase 6 closer in AI Foundations.

Brief for the Phase 5 closer on chain-of-thought prompting: what you'll learn, the prerequisites, learning outcomes, and where CoT fits before Phase 6.

A brief for the reward-model lesson: how preference pairs fill the gap SFT leaves, where this fits in the RLHF arc, prerequisites, and learning outcomes.

Brief for the scaling-laws lesson: how the Kaplan and Chinchilla results give the 'pretraining works because of scale' claim its empirical foundation.

Brief for the Phase 3 closer on precision: what floating-point bits mean, why lower precision saves memory and compute, plus quantization and mixed precision.

Brief for the parallelism and Flash Attention lesson: prerequisites, learning outcomes, and how the four memory tricks fit into Phase 3 pretraining.

Brief for the pretraining lesson, Phase 3 opener: next-token prediction at internet scale, the data sources, one worked training step, and scale figures.

Brief for the lesson on how modern LLMs inject position into attention: the input-to-attention shift, T5 relative bias, ALiBi, and the RoPE intuition.

Brief for the normalization lesson: what LayerNorm does, why transformers prefer it to BatchNorm, the post-norm to pre-norm shift, and the move to RMSNorm.

Brief for the RAG lesson: what retrieval-augmented generation is, the pipeline, bi-encoder and cross-encoder retrieval, HyDE, and indirect prompt injection.

Brief for the prompting lesson: what a prompt is at the token level, what system prompts do, prompt injection, and where it sits in Phase 5 of Track 5.

Brief for the encoder-decoder and T5 lesson: the T5 family (T5, mT5, byT5), span corruption pretraining, and why decoder-only models eventually won out.