AI Post Transformers

This episode examines CacheBridge, a paper proposing targeted fixes to a training-free method for transferring KV caches between different transformer models in multi-model routing setups. It explains why caches can't simply be handed off — differing residual widths, GQA head counts, and RoPE position encoding make one model's cache unreadable to another — and how a prior affine-mapper approach (F...

This episode explores a Google DeepMind paper proposing that separately trained language models can exchange raw internal state — the key-value cache built during transformer inference — through a shared "global latent space," rather than communicating only through text. The hosts unpack why text is a lossy bottleneck for inter-model communication, and how lightweight, frozen-weight adapter pairs...

This episode examines "Mixture-of-Translators: Translating KV Caches Across Heterogeneous Large Language Models" by Jin-woo Lee and six co-authors from Chungnam National University and KISTI, which tackles the problem of transferring one model's KV cache — its layer-by-layer key/value memory of a processed document — to a completely different model architecture without re-running the original text...

This episode examines "Dream-RSI: Recursive Self-Improvement through Evolving Worlds," which proposes making the exploration strategy of a discovery system — not the underlying model — the target of recursive self-improvement. Rather than optimizing candidate solutions directly, the system optimizes the policy that decides how to search: which branches to expand, how to parallelize workers, and wh...

This episode examines Semantic Cache Distillation, a technique for reusing KV caches across producer and consumer transformers that share architecture but have different fine-tuned weights. The discussion covers why prefill-decode disaggregation splits compute-bound and memory-bandwidth-bound phases across separate machines, and how naively shipping raw or compressed KV caches between differently-...

This episode examines Programmatically Interpretable Reinforcement Learning (PIRL), a 2018 framework from Rice University, Google Brain, and DeepMind researchers that forces RL policies to be expressed as short, human-readable programs rather than opaque neural network weights. The discussion centers on why formal verification—proving properties like bounded steering output in a self-driving car—i...

This episode explores "Deep Drone Acrobatics," which trains a quadrotor to fly extreme maneuvers — a Power Loop, Barrel Roll, and Matty Flip — using only an onboard camera and IMU, with no external motion capture. The discussion centers on how the policy is trained entirely in simulation via DAgger imitation learning, where a privileged model-predictive controller with perfect ground-truth state a...

This episode examines "An Imitative Reinforcement Learning Framework for Pursuit-Lock-Launch Missions," which trains an unmanned combat aerial vehicle to complete a three-stage dogfighting task by blending TD3-style actor-critic reinforcement learning with a behavior-cloning term drawn from expert trajectories generated in the authors' own simulator. The discussion covers why sparse-reward, multis...

This episode examines Guided Policy Search, a 2013 method from Sergey Levine and Vladlen Koltun that lets flexible neural-network policies control robots without falling into the poor local optima that plague direct policy search over high-dimensional parameter spaces. The discussion traces the paper's teacher-student structure: differential dynamic programming (DDP), a model-based trajectory opti...

This episode examines MPC-Net, a 2019/2020 paper from ETH Zürich's Robotic Systems Lab that trains a fast neural policy to replace expensive model predictive control on the ANYmal quadruped, cutting per-step evaluation from 38 milliseconds to roughly 0.125 milliseconds using less than ten minutes of demonstration data. The discussion centers on why the method learns by minimizing the control Hamil...

This episode examines "Hierarchical Reinforcement Learning for Air Combat at DARPA's AlphaDogfight Trials," in which the PHANG-MAN agent swept a graduate of the USAF Weapons Instructor Course 5-0 in simulated dogfighting. The discussion traces the DARPA ACE program's rationale for building trust incrementally toward AI-assisted piloted aircraft, and contrasts this work with prior systems like Nick...

This episode examines "A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning" (Ross, Gordon, and Bagnell, AISTATS 2011), unpacking why naively training a policy via supervised learning on expert demonstrations produces errors that compound quadratically with task horizon rather than linearly. It traces this compounding-error problem through concrete analogues — a...

This episode examines a 2018 survey, "An Algorithmic Perspective on Imitation Learning," which frames robot skill acquisition as an alternative to brittle manual programming or fragile reward engineering. It contrasts two core approaches: behavioral cloning, which treats the problem as supervised learning but suffers from compounding errors when the policy drifts into states the expert never demon...

This episode examines Piper, a training system from Oak Ridge National Laboratory designed to fix catastrophic GPU underutilization in large-scale Mixture-of-Experts training, where the leading framework X-MoE hits only about 5% utilization on a 545-billion-parameter model. The discussion traces MoE's evolution from GShard and Switch Transformer's coarse-grained experts to DeepSeek-MoE's fine-grai...

This episode explores MoM (Mixture-of-Memories), a linear sequence modeling architecture from researchers at Shanghai AI Laboratory and collaborating universities that tackles a core weakness in efficient Transformer alternatives: their tendency to forget information from earlier in a sequence. The discussion traces the trade-off at the heart of the field — Transformers preserve every token via a...

This episode examines a Lightmatter-authored paper claiming photonic interconnects can cut inference prefill latency by up to 8.5x for long-context, Mixture-of-Experts workloads. The hosts unpack why prefill has become a dominant cost center as agentic coding pushes median prompt lengths toward 96K tokens, and explain the technical distinction between compute-bound prefill and memory-bandwidth-bou...

This episode revisits "The Case for Learned Index Structures" by Tim Kraska and coauthors from MIT and Google, which proposes replacing classic data structures like B-Trees, hash maps, and Bloom filters with small trained neural networks. The discussion covers how a B-Tree traversal is mathematically equivalent to estimating a cumulative distribution function, and how a tiny two-layer model can pr...

This episode examines "Post-Training Language Models for Gold-Medal Performance in Coding Competitions" from NVIDIA, whose Ultra-CC system scored 535.4 out of 600 at the 2026 International Olympiad in Informatics — beating not just the gold cutoff of 361.12 but the top human competitor's 498.27, live and under real contest conditions. The discussion breaks down why IOI-style problems are a harder...

This episode surveys "Recursive Self-Improvement in AI," a paper by Mingguang Chen and colleagues that classifies 1,250 papers on how AI systems attempt to improve themselves. The discussion establishes precise definitions distinguishing agents, harnesses, and evaluators, then draws a critical line between bounded self-refinement (improvement against a fixed external evaluator) and open-ended recu...

This episode examines "AI Coaching for Accelerating Human Skill Development with Reinforcement Learning," a University of Pennsylvania and Johns Hopkins paper that challenges the assumption that AI assistance always benefits learners, arguing that copilots optimized for immediate task success can quietly prevent people from ever mastering a skill independently. The discussion traces the tension be...

This episode examines MegaTrain, a method for full-precision training of 100-billion-parameter-plus language models on a single H200 GPU paired with 1.5 terabytes of host RAM, from a Notre Dame and Lehigh University team. The discussion centers on why memory, not compute, is the real bottleneck for most researchers, citing a survey showing only two of 167 surveyed U.S. universities average more th...

This episode examines when it actually pays to split LLM inference hardware into four specialized pools rather than the now-standard two-way prefill/decode split, based on a paper proposing PDAF (prefill-attention, prefill-FFN, decode-attention, decode-FFN). It traces the reasoning from why agentic workloads — which can hit context sizes of 100,000+ tokens through repeated tool calls — strain hard...

This episode examines "Language Models Can Control Their Own Attention" from KAIST AI, which tackles the memory bottleneck of long-context inference: at a million tokens, generating each token requires hauling roughly 15 gigabytes of key-value cache through memory, comparable to reloading the entire model's active parameters. The discussion traces prior fixes—StreamingLLM's attention-sink heuristi...

This episode examines a comparative study of three KV cache management strategies for LLM inference — vLLM's PagedAttention memory management, H2O's static sparsification, and InfiniGen's dynamic CPU-offload selection — tested side by side on identical hardware for the first time. The standout finding: both H2O and InfiniGen hit out-of-memory errors around 10,000 tokens, less than 10% of the 128K...

This episode examines TreeWY, a proposed method for speculative decoding verification in hybrid language models that mix standard attention with Gated DeltaNet linear-attention layers. The discussion explains why current systems like vLLM and SGLang must snapshot the full recurrent state at every draft position before verification, since GDN's decay-and-overwrite state update can't be partially ro...