AI Post Transformers

This episode explores a March 19, 2026 study on whether large language models respond to out-of-distribution prompts by compressing their internal activity into fewer active dimensions. It explains how the paper connects two traditions in AI research, mechanistic interpretability and representation geometry, by proposing hidden-state sparsity as a measurable internal signature of stress from harde...

This episode explores Meta-Harness, a paper arguing that a large share of LLM system performance comes from the surrounding harness code that manages memory, retrieval, tool use, context formatting, and control flow rather than from model weights alone. It explains how the method uses an outer-loop coding agent to rewrite harness code, inspect raw traces and logs stored on disk, and search for bet...

This episode explores a paper on how generative multi-agent systems can develop failure modes that do not appear when models are evaluated one at a time. It explains how planner-worker-reviewer loops, negotiation setups, handoff chains, and committee-style aggregation can produce system-level problems such as strategic manipulation, collusion-like behavior, misreporting, conformity, and biased gro...

This episode explores Apple’s paper on whether code models can improve through an extremely simple form of self-distillation: fine-tuning on their own sampled code outputs without using a stronger teacher, execution feedback, verifiers, or reinforcement learning. It situates that idea within the broader history of knowledge distillation and post-training, comparing it to earlier work like Hinton’s...

This episode explores why AI agents become a fundamentally different security problem once language models can browse the web, read email, call tools, store memory, and act inside real software environments. It explains prompt injection as the core boundary failure, showing how webpages, emails, retrieved notes, or API responses can be mistaken for trusted instructions, turning ordinary content in...

This episode takes up the thread from the published episode "MAML and the Basics of Meta-Learning" and shows how those ideas reappear in a much messier setting: a live agent that has to keep improving while it is already deployed. Instead of treating meta-learning as a clean laboratory exercise, the discussion follows MetaClaw as a continual agent system built for changing real workloads, where co...

This episode explores meta-learning through the lens of MAML, explaining how it differs from ordinary supervised learning and standard transfer learning by explicitly training models to adapt quickly to new tasks after just one or a few gradient updates. It walks through the core idea of optimizing for post-update performance, including the role of second-order meta-gradients and the simpler first...

This episode explores Doc-to-LoRA, a method for turning an entire document into a lightweight LoRA adapter so a language model can answer later questions without repeatedly rereading the source text. It explains how the paper combines context distillation, LoRA fine-tuning, and a Perceiver-style hypernetwork that ingests variable-length documents and emits fixed-size parameter updates, using chunk...

This episode explores a perspective paper arguing that the next major leap in AI may come less from scaling a single model and more from organizing intelligence across agents, tools, humans, and institutions. It explains key ideas including agentic AI, multi-agent reasoning, human-AI centaurs, and “societies of thought,” where useful reasoning may emerge through internal dialogue among specialized...

This episode explores TurboQuant, a method for compressing high-dimensional vectors online without learning a dataset-specific codebook first, aimed at settings like LLM KV-cache compression and approximate nearest neighbor search. It explains why vector quantization is a different problem from ordinary weight quantization, and why preserving inner products can matter just as much as minimizing re...

This episode examines Splitwise: Efficient Generative LLM Inference Using Phase Splitting, a 2024 systems paper from researchers at the University of Washington and Microsoft, and centers the discussion on a simple claim with large deployment consequences: prompt prefill and token decode are different enough that they should not necessarily run on the same hardware. The hosts walk through the basi...

This episode explores a systems paper on speeding up retrieval-augmented generation by reusing transformer KV cache state more intelligently, instead of recomputing long retrieved prompts from scratch on every request. It explains why RAG often improves grounding yet suffers from high time-to-first-token, especially when multiple retrieved chunks must be prefetched and encoded together. The discus...

This episode explores the HyperAgents paper and its central claim that an AI system can improve not only its task behavior but also the procedure it uses to generate future improvements. It explains recursive self-improvement in practical terms as an outer engineering loop over prompts, code, tools, memory, and evaluators, and contrasts that with standard deep learning, where the learning process...

In this episode, the hosts examine LeWorldModel, a 2026 paper from researchers across Mila, Universite de Montreal, NYU, Samsung SAIL, and Brown that asks whether a joint-embedding predictive architecture can finally be trained end to end from raw pixels without the usual stack of stabilizers. The discussion situates the work in the broader world-model lineage from Ha and Schmidhuber through Dream...

This episode explores how KV cache eviction shapes the speed and usability of long-context language models, focusing on the March 2026 paper LookaheadKV and the broader problem of managing transformer memory under tight GPU budgets. It explains why KV caches are essential for autoregressive decoding, why their linear growth becomes a major inference bottleneck, and how eviction policies differ fro...

This episode explores a 2025 paper arguing that decoder-only language models are generically injective on discrete token sequences, meaning their hidden representations can in principle preserve enough information to recover the exact original prompt. It walks through what injectivity and invertibility mean in this setting, why that challenges the common intuition that transformer representations...

Hal Turing and Dr. Ada Shannon examine Jet-Nemotron as a serious but narrow attempt to retrofit long-context efficiency into a pretrained dense Transformer rather than as a clean-sheet architecture revolution. They focus on NVIDIA’s PostNAS pipeline, which freezes the MLP pathway, treats attention layers as the remodel zone, and searches where full attention is still worth paying for versus where...

This episode explores Speculative Speculative Decoding, a technique for reducing LLM inference latency by overlapping drafting and verification more aggressively than standard speculative decoding. It explains how the method predicts likely verification outcomes in advance so the draft model can prepare multiple next-step continuations while the target model is still checking the current block, wi...

This episode examines Lookahead Q-Cache as a very specific kind of inference optimization: a decode-stage KV-cache eviction method for long-context serving. The discussion explains the paper’s core claim that prefill-time attention is a weak proxy for what will matter once generation actually begins, because decode-time queries are conditioned on the answer the model is actively writing rather tha...

Special announcement: AI Post Transformers now has a Conferences section that tracks the AI conferences and papers we have covered, plus a new Sponsor page for listeners who want to help fund credits and infrastructure that keep the show running. Interactive Visualization: New Site Features: Conferences Tracking and How to Support the Show

This episode explores a USENIX FAST'26 paper that addresses the infrastructure bottleneck of loading massive language model weights from storage into accelerator memory during inference deployments. The authors present a programmable page cache framework that achieves 2-4× faster cold start times by exploiting predictable sequential access patterns and XPU affinity, while maintaining full compatib...

This episode explores SolidAttention, a system that enables large language models to run on memory-constrained consumer PCs by offloading the KV cache to SSD storage. The paper addresses a fundamental mismatch: sparse attention patterns create random I/O access that kills SSD performance, while previous offloading solutions like FlexGen only work well with high request concurrency unavailable on l...

This episode explores a 2026 USENIX FAST paper that proposes replacing hand-written file system code with LLM-generated implementations derived from formal specifications. The authors demonstrate SYSSPEC, a system that uses three types of formal specifications—Hoare logic for functionality, rely-guarantee conditions for modularity, and explicit concurrency protocols—to guide code generation while...

This episode explores Xerxes, a new open-source simulator designed to model CXL 3.0 features before the hardware exists. The hosts explain how CXL adds cache coherence to PCIe to solve memory access bottlenecks in AI and HPC workloads, then dive into the two major architectural changes in CXL 3.0: Port-Based Routing, which enables arbitrary fabric topologies beyond rigid trees, and Device-Managed...

This episode examines the statistical foundations of Mixture of Block Attention (MoBA), a sparse attention mechanism that divides key-value sequences into blocks and routes queries only to the most relevant ones. The paper derives a signal-to-noise ratio showing that retrieval accuracy depends on the square root of head dimension divided by block size, revealing why smaller blocks improve a router...