AI Post Transformers

This episode examines "More Agents Is All You Need," a Tencent paper showing that sampling a single LLM multiple times and voting on the outputs can match the performance of a model roughly five times larger — no fine-tuning, no multi-agent debate, no specialized prompting required. The discussion traces this "Agent Forest" method back to classical ensemble techniques like Random Forests and conne...

This episode examines BitWhisper, a 2015 paper from Ben-Gurion University claiming that two air-gapped computers can exchange data using only CPU heat, with the receiving PC reading its own thermal sensors. It places the work against earlier covert channels (FM radio, ultrasound, electromagnetic and optical leakage), most of which can only leak data out, whereas this one claims half-duplex two-way...

This episode looks at RelayS2S, a dual-path design for real-time voice dialogue. A small, fast speech model speaks the first five words of a reply while a stronger text-based pipeline writes the rest. The discussion sets the target at roughly 200 milliseconds, the average gap between turns in human conversation. It contrasts slow but smart ASR→LLM→TTS cascades with fast but weaker full-duplex mode...

This episode examines "You Only Cache Once," a decoder-decoder architecture from Microsoft Research and Tsinghua University that computes one global key-value cache and reuses it across every upper layer. The hosts explain why the KV cache becomes a deployment bottleneck at long context. They cite a 65B model needing about 86 GB of cache at 512K tokens even with grouped-query attention and 8-bit q...

This episode examines DeepSeek-AI's mHC: Manifold-Constrained Hyper-Connections, which tries to improve the transformer's residual connection, a piece of the architecture that has barely changed in a decade. It first explains why the plain skip path x + F(x) has been so hard to displace. It then covers ByteDance's Hyper-Connections, which widen the residual stream into four parallel streams with l...

This episode examines the Qwen team's design-rationale report for Qwen3.8-Flash-Next, a 125B-total, 6B-activated base model with 51B of n-gram embeddings held in host memory. It walks through the five components: a hybrid of Gated DeltaNet layers with periodic full attention, Qwen's sparse attention variant (QSA) as a response to the quadratic cost of DeepSeek-style indexers, a four-branch gated r...

This episode examines Hyper-Connections (ICLR 2025, ByteDance Seed), which replaces the fixed residual connection in transformers with learned connection strengths. It traces the "seesaw" between Pre-Norm and Post-Norm: Pre-Norm gives stable gradients but suffers representation collapse in deep layers, while Post-Norm does the reverse. The method carries n parallel residual streams and learns dept...

This episode explores "Next-Latent Prediction Transformers Learn Compact World Models," a Microsoft Research paper proposing a small auxiliary loss that pushes an ordinary transformer to compress its history into a compact belief state, with no architecture changes. It covers why transformers lack the built-in compression that recurrent networks get from a fixed-size state, using the Manhattan tax...

This episode explores "Subliminal Learning," a paper showing that a language model can pass behavioral traits to a student model through data that looks unrelated to the trait. In the headline experiment, a teacher prompted to love owls generates only number sequences. After filtering, a fresh copy of the base model is finetuned on those numbers, and its share of "owl" answers to a favorite-animal...

This episode dissects the BASED architecture from Arora, Eyuboglu, Zhang et al. (Stanford/Buffalo), examining how it tries to resolve the tradeoff between recall accuracy and inference throughput in sequence models. The discussion traces the lineage of alternatives to standard softmax attention — state space models like Mamba, gated-convolution approaches like H3 and Hyena, linear attention, and s...

This episode unpacks ALEX, a 2020 paper from MIT, Microsoft Research, Arizona State, and Georgia Tech that reworks the "learned index" idea for real-world databases. It traces the lineage from Kraska et al.'s 2018 Learned Index, which used a hierarchy of regression models to predict a key's position in a sorted array but only worked on static, read-only data since any insert would break the model'...

This episode examines SALI, a 2024 SIGMOD systems paper diagnosing why learned database indexes like ALEX and LIPP see throughput drop as thread counts rise, rather than scale with additional concurrency. The discussion traces the lineage from Google's 2018 Recursive Model Index through the buffer-based and model-based approaches that emerged to handle inserts, focusing on the shift-versus-chain t...

This episode examines APEX, a persistent-memory learned index from researchers at CUHK, MIT, Microsoft Research, and Simon Fraser University, presented at VLDB 2022. It traces the collision of two research threads — Intel's Optane persistent memory, which sits on the DDR bus but requires manual cache-line flushing and fencing to guarantee crash durability, and "learned indexes," which replace B-tr...

This episode explores Structural Language Models of Code, a 2020 paper from Technion, Tel Aviv University and Facebook AI Research that tackles "any-code completion": predicting a missing piece of a program with no restriction on vocabulary or structure. It traces the history from 1969-era automatic programming through domain-specific systems like FlashFill and DeepCoder, and prior general-languag...

This episode examines "Skip a Layer or Loop it? Test-Time Depth Adaptation of Pretrained LLMs," which proposes Chain-of-Layers (CoLa). CoLa takes a frozen pretrained model and, for each input, skips some layers, repeats others, and reorders them, with no finetuning. The hosts explain why skipping works, citing residual connections, ResNets behaving like ensembles of shallow paths, and static pruni...

This episode examines "Skip a Layer or Loop It? Learning Program-of-Layers in LLMs" (ICML 2026), which asks whether a frozen pretrained LLM can answer better if each input gets its own sequence of skipped and repeated layers, with no retraining of the base model. The hosts place it against prior work: layer-dropping and early-exit methods built to save compute, recurrent-depth models trained to lo...

This episode explores Simula, a system from EPFL and Google DeepMind researchers that generates entire specialized training datasets from scratch, with no seed examples required. It breaks down the reasoning-driven, multi-role agentic pipeline — where separate model roles propose, critique, sample, generate, and filter data — contrasting it with older approaches like Self-Instruct's hand-written t...

This episode examines RewardingDoubt, a reinforcement-learning method for training large language models to express calibrated confidence rather than defaulting to near-maximal certainty regardless of correctness. The discussion covers why miscalibrated confidence is dangerous in high-stakes domains like medicine, legal consultation, and customer service, where deferring to a human reviewer only w...

This episode breaks down DeepSeek-V4.1-Flash, a 552-billion-parameter Mixture-of-Experts model whose KV cache footprint has shrunk roughly 437-fold since DeepSeek's original V1, including a 4x drop in this single generational jump from V4-Flash. The discussion explains why long-context, tool-using agents make the KV cache the real serving bottleneck, and how the model's architecture attacks it fro...

This episode examines "The Unreasonable Ineffectiveness of the Deeper Layers," a 2025 study showing that up to half the layers of a 70-billion-parameter model can be removed with almost no drop in standard QA benchmark performance. The discussion covers the residual stream architecture that makes such pruning possible, why later transformer layers often contribute diminishing changes to accumulate...

This episode examines "Transformer Layers as Painters," which asks whether a frozen, ordinarily trained Llama 2 can tolerate having its layers skipped, swapped, reordered, or run in parallel without any retraining. The discussion uses a painter-on-an-assembly-line analogy: the residual stream serves as a shared canvas, so layers read and write the same space. It places the paper alongside related...

This episode examines ShortGPT, a layer-pruning method that ranks the layers of a pre-norm LLM by Block Influence (one minus the average cosine similarity between a layer's input and output hidden states) and deletes the lowest-scoring ones with no gradients or retraining. It sets the method against unstructured pruning and the structured baselines LLM-Pruner, SliceGPT, and LaCo. It also covers th...

This episode examines ProgramBench, a new benchmark testing whether frontier language models can rebuild working software from just a compiled binary and its documentation, with no source code, scaffolding, or prescribed architecture to work from. Across nine frontier models and two hundred tasks spanning CLI tools up to FFmpeg, SQLite, and the PHP interpreter, zero tasks were fully resolved, expo...

This episode examines the NVQLink architecture, a NVIDIA-led collaboration with nine national labs and research institutions that tightly couples high-performance computing with quantum processors. The discussion centers on why reaction time—not just throughput—is critical for quantum error correction, since qubits decohere while waiting for classical correction signals to arrive. It breaks down t...

This episode examines "Learning to Solve Hard Problems in RL for LLMs by Never Giving Up," which finds that RL post-training on math, code, and agentic coding benchmarks disproportionately improves performance on problems models already handle reasonably well, while barely moving the needle on the hardest tasks — a pattern the authors dub the Matthew Effect, after Robert Merton's sociology of scie...