AI Post Transformers

This episode explores the HyperAgents paper and its central claim that an AI system can improve not only its task behavior but also the procedure it uses to generate future improvements. It explains recursive self-improvement in practical terms as an outer engineering loop over prompts, code, tools, memory, and evaluators, and contrasts that with standard deep learning, where the learning process...

In this episode, the hosts examine LeWorldModel, a 2026 paper from researchers across Mila, Universite de Montreal, NYU, Samsung SAIL, and Brown that asks whether a joint-embedding predictive architecture can finally be trained end to end from raw pixels without the usual stack of stabilizers. The discussion situates the work in the broader world-model lineage from Ha and Schmidhuber through Dream...

This episode explores how KV cache eviction shapes the speed and usability of long-context language models, focusing on the March 2026 paper LookaheadKV and the broader problem of managing transformer memory under tight GPU budgets. It explains why KV caches are essential for autoregressive decoding, why their linear growth becomes a major inference bottleneck, and how eviction policies differ fro...

This episode explores a 2025 paper arguing that decoder-only language models are generically injective on discrete token sequences, meaning their hidden representations can in principle preserve enough information to recover the exact original prompt. It walks through what injectivity and invertibility mean in this setting, why that challenges the common intuition that transformer representations...

Hal Turing and Dr. Ada Shannon examine Jet-Nemotron as a serious but narrow attempt to retrofit long-context efficiency into a pretrained dense Transformer rather than as a clean-sheet architecture revolution. They focus on NVIDIA’s PostNAS pipeline, which freezes the MLP pathway, treats attention layers as the remodel zone, and searches where full attention is still worth paying for versus where...

This episode explores Speculative Speculative Decoding, a technique for reducing LLM inference latency by overlapping drafting and verification more aggressively than standard speculative decoding. It explains how the method predicts likely verification outcomes in advance so the draft model can prepare multiple next-step continuations while the target model is still checking the current block, wi...

This episode examines Lookahead Q-Cache as a very specific kind of inference optimization: a decode-stage KV-cache eviction method for long-context serving. The discussion explains the paper’s core claim that prefill-time attention is a weak proxy for what will matter once generation actually begins, because decode-time queries are conditioned on the answer the model is actively writing rather tha...

Special announcement: AI Post Transformers now has a Conferences section that tracks the AI conferences and papers we have covered, plus a new Sponsor page for listeners who want to help fund credits and infrastructure that keep the show running. Interactive Visualization: New Site Features: Conferences Tracking and How to Support the Show

This episode explores a USENIX FAST'26 paper that addresses the infrastructure bottleneck of loading massive language model weights from storage into accelerator memory during inference deployments. The authors present a programmable page cache framework that achieves 2-4× faster cold start times by exploiting predictable sequential access patterns and XPU affinity, while maintaining full compatib...

This episode explores SolidAttention, a system that enables large language models to run on memory-constrained consumer PCs by offloading the KV cache to SSD storage. The paper addresses a fundamental mismatch: sparse attention patterns create random I/O access that kills SSD performance, while previous offloading solutions like FlexGen only work well with high request concurrency unavailable on l...

This episode explores a 2026 USENIX FAST paper that proposes replacing hand-written file system code with LLM-generated implementations derived from formal specifications. The authors demonstrate SYSSPEC, a system that uses three types of formal specifications—Hoare logic for functionality, rely-guarantee conditions for modularity, and explicit concurrency protocols—to guide code generation while...

This episode explores Xerxes, a new open-source simulator designed to model CXL 3.0 features before the hardware exists. The hosts explain how CXL adds cache coherence to PCIe to solve memory access bottlenecks in AI and HPC workloads, then dive into the two major architectural changes in CXL 3.0: Port-Based Routing, which enables arbitrary fabric topologies beyond rigid trees, and Device-Managed...

This episode examines the statistical foundations of Mixture of Block Attention (MoBA), a sparse attention mechanism that divides key-value sequences into blocks and routes queries only to the most relevant ones. The paper derives a signal-to-noise ratio showing that retrieval accuracy depends on the square root of head dimension divided by block size, revealing why smaller blocks improve a router...

This episode examines the MATT (Model-Aware Tokenizer Transfer) paper from AGH University of Krakow, which proposes a fundamentally different approach to extending language models to underserved languages. Using Georgian as the central case study, the episode explains tokenizer fertility — how tokenizers optimized for high-resource languages fragment Georgian words into six to eight subword pieces...

This episode explores the xLLM Technical Report from JD.com, which describes a production system for running large language model inference at enterprise scale. The core technical challenge is co-locating online chatbot workloads with offline batch jobs on the same GPU cluster to maximize hardware utilization during traffic valleys, while maintaining strict latency guarantees during peak hours. Th...

This episode examines a comprehensive survey paper that proposes a new framework for understanding memory in AI agent systems. The authors challenge traditional cognitive psychology categories (like short-term versus long-term memory) and instead organize agent memory along three dimensions: form (how memory is stored—as tokens, parameters, or latent vectors), function (what memory represents—fact...

This episode explores Qwen3Guard, a safety guardrail system for large language models that introduces two key architectural innovations. The paper presents a three-way classification scheme—safe, controversial, and unsafe—allowing organizations to customize content moderation policies rather than relying on rigid binary thresholds, plus a streaming-compatible variant that evaluates safety token-by...

This episode explores a new system called Bidaw that dramatically improves the performance of long, multi-turn AI chatbot conversations by solving a critical caching problem. The paper reveals that existing approaches waste over 93% of computation redundantly recalculating conversation history, and that naive two-tier storage systems (using both RAM and SSD) increase latency by 3.8x because the GP...

We ran out of ElevenLabs credits. This episode introduces our new open-source voices powered by Kokoro, an 82-million parameter text-to-speech model built on StyleTTS 2. We explain the Docker container saga of running Python 3.12 dependencies on a 3.13 host, rave about CPU-only inference speed, tease a future deep-dive on the papers behind lightweight neural TTS, demo Spanish multilingual support,...

This episode examines "Attention Residuals," a March 2026 paper from Moonshot AI's Kimi team that challenges a foundational element of transformer architecture. The paper proposes replacing the fixed, uniform residual connections inherited from ResNet with learned attention mechanisms for depth-wise information aggregation. While attention replaced recurrent neural networks for sequence modeling o...

This episode explores a recent paper that extends neural scaling laws to predict real-world task performance rather than just training loss, while accounting for context length as a first-order variable. The episode discuss how traditional scaling laws from Kaplan (2020) and Chinchilla (2022) successfully predicted pretraining metrics but failed to address downstream task accuracy or the impact of...

This episode examines CacheSlide from USENIX FAST26, a system that enables LLMs to reuse cached key-value pairs across shifting prompt positions in agentic workflows. The paper introduces chunked contextual position encoding and priority-based eviction to solve the position mismatch problem that prevents KV cache reuse when prompt segments shift in multi-turn agent conversations. Interactive Visua...

This episode examines a Meta-led paper that develops the first systematic scaling laws for reinforcement learning in large language models, based on over 400,000 GPU-hours of experiments. The researchers propose a sigmoid framework to predict RL performance at large compute budgets from early training runs, addressing a critical gap in the field—while pre-training has well-established power-law re...

This episode of AI Post Transformers examines "Agentic Code Reasoning" by Shubham Ugare and Satish Chandra from Meta, which introduces semi-formal reasoning certificates as an inference-time scaffold for LLM agents analyzing code without executing it. Rather than letting a model produce free-form chain-of-thought verdicts, the certificate framework requires the agent to state explicit premises, tr...

This episode explores the challenge of getting self-interested AI agents to cooperate without hardcoding cooperative behavior, examining a 2026 Google paper on multi-agent cooperation through in-context co-player inference. The hosts build up the technical foundations carefully, explaining why standard reinforcement learning breaks down in multi-agent settings due to non-stationarity, and how soci...