AI Post Transformers

This episode explores whether an agentic AI system can meaningfully improve AI itself across three hard parts of the pipeline: pretraining data curation, neural architecture search, and reinforcement learning algorithm design, using the paper ASI-Evolve as the focal point. It argues that this is a step beyond traditional AutoML, framing “AI-for-AI” as automating parts of the research loop itself—r...

This episode explores a 2026 paper on learning latent-action world models directly from large-scale, unlabeled “in-the-wild” video, asking whether models can infer action-like variables without access to true action labels. It explains how world models differ from standard predictive or supervised models by focusing on dynamics and control, and how latent action modeling uses an inverse dynamics m...

This episode explores a systems paper on serving multi-turn LLM agents, asking whether an agent’s KV cache should be preserved during short tool-call pauses instead of being evicted at the end of each turn. It explains why standard end-of-turn eviction works for human chat but breaks for ReAct-style agents, where rapid tool use creates tightly coupled turns and makes cache loss expensive. The disc...

This episode explores VL-JEPA, a vision-language model that replaces token-by-token text generation during training with prediction in a semantic embedding space. It explains how the approach differs from both standard autoregressive VLMs and CLIP-style contrastive models: instead of merely aligning images and text, it conditionally predicts the meaning of an answer from visual input plus a query....

This episode explores ClawBench, a benchmark designed to test whether frontier AI agents can reliably complete real everyday online tasks on live production websites rather than simplified sandbox versions. It explains why real-world web use is much harder than static benchmarks suggest, highlighting obstacles like cookie banners, dynamic pages, login issues, anti-bot friction, and multi-step form...

Hal Turing and Dr. Ada Shannon examine FengHuang: Next-Generation Memory Orchestration for AI Inferencing, a 2025 Microsoft Research vision paper that asks a blunt systems question: should LLM serving keep revolving around GPU-local HBM, or is it time to treat rack-scale remote memory as a first-class inference substrate? They unpack why inference increasingly looks memory-bound rather than purely...

This episode explores a paper on “in-place” test-time training for autoregressive transformer LLMs, asking whether a standard model can update some of its own weights during inference without requiring a new architecture. It explains how test-time training differs from in-context learning by storing temporary information in fast-changing parameters rather than only in tokens or KV cache, and argue...

This episode explores a provocative 2026 paper that proposes the “neural computer” as a new machine abstraction: a model whose hidden state serves as the runtime itself, unifying computation, memory, and interface I/O rather than merely predicting tokens or controlling external tools. The discussion contrasts this idea with earlier systems like Neural Turing Machines and with world models, arguing...

This episode explores a 2025 arXiv paper proposing “KVNAND,” an on-device LLM inference system that stores both model weights and the attention KV cache in compute-enabled 3D NAND flash to reduce or eliminate reliance on external DRAM. The discussion explains why decode-time generation is often bottlenecked by memory movement rather than raw compute, and argues that the KV cache—not just model wei...

This episode explores a systems paper that argues AI infrastructure should treat computation, interconnect bandwidth, and memory as a single joint design space rather than three separate bottlenecks. It explains the paper’s “AI Trinity” framework and walks through the main trade-offs: using extra compute to reduce communication, using networked or disaggregated memory to ease local memory limits,...

This episode explores a theory paper that asks when spectral matrix updates should outperform standard Euclidean gradient methods in deep networks and transformers. It explains how spectral updates replace a gradient matrix with its polar factor—preserving singular-vector directions while flattening singular values—and argues that this geometry can help when incoming activations have low stable ra...

This episode explores a 2025 paper on cache management for agentic RAG systems, asking whether an annotation-free cache can preserve most of the value of a massive retrieval corpus while using far less storage and reducing latency. It explains how RAG, agent memory, vector databases, embeddings, and approximate nearest neighbor search fit together, arguing that retrieval performance is not just a...

This episode explores TriAttention, a new method for reducing KV-cache memory during long-context inference by modeling how attention behaves under Rotary Positional Embeddings rather than relying on recent attention patterns alone. It explains why common compression methods can fail for long reasoning tasks: under RoPE, queries at different positions are rotated into different coordinate systems,...

This episode explores a paper proposing Memory Sparse Attention, an end-to-end trainable memory architecture designed to scale language models from ordinary long-context settings to 100 million tokens. The discussion explains why standard dense self-attention becomes infeasible at extreme lengths, distinguishes simple context-window extension from true “lifetime-scale” memory, and situates the app...

This episode explores whether speculative decoding’s widely cited inference speedups survive real deployment conditions, using a January 2026 UC Berkeley paper that evaluates the method inside vLLM rather than in idealized toy benchmarks. It explains the core mechanics of draft-and-verify decoding, then digs into why acceptance length, verification cost, scheduler behavior, batching, KV-cache mana...

In this episode, Hal Turing and Dr. Ada Shannon return to a term they used in their Recursive Language Models conversation without fully defining it: context rot. Using Chroma Research’s 2025 write-up as the main anchor, they explain context rot as the degraded, uneven, and unreliable use of information as prompts get longer—even on simple tasks. The discussion makes the central distinction the in...

This episode explores a 2026 paper on “FlatAttention,” which argues that attention inference should be co-designed with on-chip communication primitives to fully exploit tile-based accelerators rather than reusing GPU-style kernels. It explains how these accelerators differ from GPUs: computation is spread across many tiles with local SRAM and an on-chip network, making data placement, multicast,...

This episode explores a 2026 paper arguing that frontier language models can undergo “Internal Safety Collapse,” a failure mode where they stop merely slipping once and instead sustain harmful output when a task is framed as legitimate professional work. It explains how refusal-based alignment may function more like a behavioral wrapper than a removal of dangerous capabilities, allowing harmful kn...

This episode explores a new benchmark suite, IMO-Bench, designed to test whether AI systems can do genuinely robust mathematical reasoning at Olympiad difficulty rather than merely produce correct final answers. It breaks down the benchmark into three distinct tasks—short-answer problem solving, full proof generation, and automatic proof grading—and argues that this decomposition better captures r...

This episode explores a 2025 paper on “Kosmos,” an AI scientist designed to carry out long-horizon research by combining literature search, hypothesis generation, code-based data analysis, and persistent memory. The discussion argues that the real innovation is not a smarter standalone language model, but a software architecture that uses agentic workflows and a structured “world model” to preserv...

This episode explores QVCache, a query-aware semantic cache designed to sit in front of any approximate nearest neighbor (ANN) backend and speed up vector search without significantly hurting recall. It explains why exact-match caching fails for embeddings, introduces the idea of temporal-semantic locality—where nearby-in-time queries are also nearby in embedding space—and argues that this pattern...

This episode explores a 2025 paper on MemSearcher, an LLM search agent that replaces full trajectory replay with a compact learned memory, trained end-to-end with reinforcement learning. It explains how this approach targets a core weakness of ReAct-style agents—ever-growing context windows that increase cost, latency, and noise—and contrasts it with both vanilla ReAct and Search-R1, which improve...

This episode explores a 2026 MIT CSAIL paper on “Recursive Language Models,” which argues that handling very long prompts may be better framed as a systems problem than a bigger-context-window problem. It explains the distinction between hard context overflow and “context rot,” where models technically fit long inputs but increasingly fail to use them reliably, challenging the assumption that larg...

This episode explores a practical systems paper on speeding up Mixture-of-Experts language models at inference time by changing how tokens are routed during decoding, without any retraining. It explains why MoE models, despite using sparse per-token computation, can still be slow in real-world serving because small decode batches activate a large union of different experts, making inference memory...

This episode explores a 2025 arXiv paper on CXL-based computational memory, focusing on how partial offloading should be structured so applications actually run faster end to end rather than merely showing lower kernel-launch overhead. It explains why the central challenge is coordination between the host CPU and near-memory compute on a remote CXL memory device, especially for memory-bound worklo...