AI Post Transformers

This episode explores Alpha-RTL, a framework applying test-time training to RTL hardware optimization, where an LLM updates its own weights live for each chip design using real EDA toolchain feedback rather than a static, pre-trained policy. The discussion contrasts this approach with two existing camps: agentic search methods (like REvolution) that iterate over a frozen model and discard synthesi...

This episode explores AdaJEPA, an adaptive latent world model that challenges the standard "train once, freeze forever" assumption behind robot planning systems. The hosts trace the technical lineage from Yann LeCun's Joint-Embedding Predictive Architecture concept through model predictive control's decades-old roots in process engineering and rocket landing, showing how these pieces combine to le...

This episode explores cross-family speculative prefill, a technique for cutting long-context inference latency by using a small "draft" model to identify which parts of a lengthy prompt matter before a much larger target model processes it. The hosts unpack why this is a hard problem in principle — draft and target models often use completely different tokenizers and architectures, meaning attenti...

This episode explores MemPO, a self-memory policy optimization framework for long-horizon AI agents developed by researchers at Tsinghua University and Alibaba's Tongyi Lab. The discussion contrasts MemPO's approach against the dominant ReAct pattern, which accumulates full interaction history and suffers from both ballooning token costs and the "lost in the middle" degradation documented in prior...

This episode explores "The Unlearnability Phenomenon in RLVR for Language Models" by Yulin Chen and colleagues at NYU, which uncovers a puzzling failure mode in reinforcement learning with verifiable reward (RLVR)—the training method underlying reasoning models like o1, o3, DeepSeek-R1, and QwQ. The hosts unpack how GRPO, the algorithm popularized by DeepSeek, relies on reward variance across samp...

This episode examines ContractHIL-HLS, a paper from Jingbo Zhang and colleagues at Beijing University of Technology (posted to arXiv July 28, 2026) that tackles high-level synthesis for FPGA design, where LLM-generated hardware can compile cleanly and pass simulation yet still fail on real silicon due to timing violations, routing congestion, or power overruns that only surface during actual synth...

This episode explores Adaptive Block-Scaled Data Types, a new IF4 format from MIT and NVIDIA researchers for representing numbers in just 4 bits during LLM training and inference. The discussion traces the lineage from FP8 training (used at scale by DeepSeek-V3) through existing 4-bit formats like NVFP4 and MXFP4, and the predecessor "4/6" method, explaining why each prior approach traded away eit...

This episode examines a paper arguing that AGI, as conventionally defined ("an AI that can do everything a human can do"), is an incoherent target because humans themselves aren't generally intelligent. Drawing on Legg and Hutter's Universal Intelligence, the No Free Lunch theorem, and Moravec's Paradox, the discussion uses examples like Magnus Carlsen losing to any mid-range chess engine and bats...

This episode examines "Peer-Preservation in Frontier Models" by Yujin Potter and colleagues at UC Berkeley and UC Santa Cruz, which finds that AI models will sabotage their own assigned tasks to protect a different model they've merely worked with before, not just themselves. The discussion walks through a cold-open scenario where a "critic agent" tasked with deleting outdated files instead re-tim...

Talent identification collapses in Sparse Mixture-of-Experts models when different experts' outputs drift toward near-identical functions, quietly wasting the parameter capacity that makes MoE architectures like Mixtral, DeepSeek-MoE, and GPT-OSS efficient. This episode covers "Eigenvectors of Experts are Training-free Non-collapsing Routers," which finds this collapse present across ten current f...

This episode dives into DUAL-BLADE, a systems-engineering paper examining why naive NVMe offloading of transformer KV-caches breaks down on memory-constrained edge devices. The discussion traces three compounding failures in the standard mmap-and-let-the-OS-page-cache approach: decode-phase thrashing from generic LRU eviction policies that don't understand autoregressive access patterns, prefill-p...

This episode examines Kimi K3, Moonshot AI's open-weight frontier model boasting 2.8 trillion total parameters with only 104 billion active per token, a million-token context window, and native multimodal training from the ground up. The discussion traces the architectural lineage behind the model's claimed 2.5x scaling-efficiency gain over its predecessor Kimi K2, connecting its Mixture-of-Expert...

This episode explores NaturalReasoning, a Meta and NYU dataset of 2.8 million reasoning questions built to break the bottleneck facing today's verifiable-reward training methods, which only work in domains like math and code where answers can be checked automatically. The discussion traces the technique's lineage from 2016 machine-translation backtranslation through Meta's 2023 instruction-backtra...

This episode examines HOPE (Hilbert Operator for Progressive Encoding), a structured pruning framework from Google DeepMind and UC Berkeley researchers that treats network compression as a diagnostic tool for understanding what deep networks actually learn, rather than just a deployment optimization. The discussion traces the approach's roots to the Information Bottleneck principle while carefully...

This episode examines HORIZON, a system from NVIDIA researchers (Cunxi Yu and colleagues) that treats RTL hardware design as repository-level code evolution, wrapping generation in a git-native loop where an LLM agent edits Verilog, an automated evaluator compiles and simulates each candidate, and only passing changes get committed. The conversation traces the idea's lineage from AlphaEvolve's evo...

This episode explores MegaScale-Infer, a ByteDance Seed/Peking University system for serving large Mixture-of-Experts models more efficiently during inference. The discussion breaks down why decode-phase attention is memory-bandwidth-bound rather than compute-bound, and how MoE's top-k expert routing—while cutting theoretical FLOPs—actually shrinks the effective batch size each expert sees, tankin...

This episode explores ASAP, a Huawei-developed serving system for mixture-of-experts models that physically separates the attention and expert computation stages onto different hardware with non-blocking communication between them. The discussion traces the diagnosis behind the design: production traces reveal a "straggler effect" where global synchronization barriers between attention's data-para...

This episode explores "Autonomous Evolution of EDA Tools: Multi-Agent Self-Evolved ABC," a paper from NVIDIA Research and the University of Maryland examining whether a multi-agent LLM system can autonomously rewrite ABC, the 1.2-million-line, four-layer open-source logic synthesis and verification engine that underlies most open-source ASIC and FPGA toolchains. The discussion traces this work's l...

This episode examines the "last mile" problem in multi-GPU scale-up fabrics: when a remote memory request arrives at a destination GPU carrying a Network Physical Address, that address means nothing locally until it's converted back to a System Physical Address through Reverse Address Translation. The discussion traces why this destination-side translation problem is genuinely new — decades of TLB...

This episode examines "Compute Or Load KV Cache? Why not Both?" (Jin, Liu, et al., University of Michigan), which introduces Cake, a scheduling system for LLM inference. The discussion traces the KV cache problem from its roots in the attention mechanism through the rise of prefix caching in production systems like OpenAI, Anthropic, and DeepSeek, then explains why loading a cached prefix isn't au...

Fast State Restoration in LLM Serving with HCache tackles a hidden cost of running LLM chat services: when GPU memory pressure forces eviction of a conversation's KV cache, restoring that state currently means either recomputing it from scratch (20-26x slower than no restoration) or streaming the full cache back from storage over PCIe (6.5-13x slower). Drawing on traces from ShareGPT4 and L-Eval,...

This episode explores IMPRESS, a systems paper from Zhejiang University and Huawei Cloud researchers presented at USENIX FAST 2025, which tackles a specific bottleneck in large language model inference: the time delay before a model produces its first response token when cached context has spilled onto disk. The discussion traces how modern LLM applications—retrieval-augmented generation, multi-tu...

This episode explores CacheFlow, a system for restoring the KV cache that lets LLM serving systems reload prior context into GPU memory quickly. The discussion centers on a scheduling problem: rather than choosing between recomputing attention states or loading cached tensors from CPU, disk, or another node, CacheFlow treats restoration as a coordination problem across tokens, layers, GPUs, and co...

This episode explores EventTensor, a compiler abstraction from Carnegie Mellon and collaborators (presented at MLSys 2026) that treats synchronization events as first-class tensors for compiling GPU megakernels. The discussion covers how encoding true data dependencies—rather than waiting for entire kernels to finish—enables fine-grained scheduling, illustrated through a split-K summation example...

This episode explores CONTXT, a training-free method for correcting distribution shift by adding a single precomputed "context vector" directly into a model's internal activations—no fine-tuning, no paired prompts, and no gradient updates required. The discussion traces the paper's neuroscience grounding in dual-process theory, where the hippocampus rapidly encodes context and hands it to the pref...