LLMs Research Podcast

Your 70-Billion-Parameter Model Might Be 40% Wasted Three papers from February 1–6, 2026 converge on a question the field has been avoiding since 2016: what if most transformer layers aren't doing compositional reasoning at all, but just averaging noise? This video traces a decade of evidence, from Veit et al.'s original ensemble observation in ResNets through ShortGPT's layer pruning results and...

Fixing Reasoning from Three Directions at Once LLMs Research Podcast | Episode: Feb 1–6, 2026 DeepSeek-R1 made RL the default approach for reasoning. This episode covers the 30 papers from the first week of February that are debugging what that approach got wrong, across training geometry, pipeline design, reward signals, memory architecture, and inference efficiency. Timestamps [00:00] Opening an...

State space models like Mamba promised linear scaling and constant memory. They delivered on efficiency, but researchers kept hitting the same wall: ask Mamba to recall something specific from early in a long context, and performance drops. Three papers at ICLR 2026 independently attacked this limitation. That convergence tells you how fundamental the problem is. This podcast breaks down: - Why Ma...

Episode Title: What ICLR 2026 Taught Us About Multi-Agent Failures Episode Summary: We scanned ICLR 2026 accepted papers and found 14 that address real problems when building multi-agent systems: slow pipelines, expensive token bills, cascading errors, brittle topologies, and opaque agent coordination. This episode walks through five production problems and the research that provides concrete solu...

The podcast discusses the technical shift in large language models from a standard 512-token context window to modern architectures capable of processing millions of tokens. Initial growth was constrained by the quadratic complexity of self-attention, which dictated that memory and computational needs increased by the square of the sequence length. To address this bottleneck, researchers developed...

Jan 17–23, 2026: The Rise of the Action LayerHow multimodal AI is shifting from passive perception to active controlThe current landscape of research reflects a shift toward Multimodal Agentic Intelligence , where multimodal capabilities are no longer treated as simple perception but as actionable interfaces for control and interaction. This trend involves the integration of visual representations...

Over the past several years, we have moved from the machine learning era through the large language model era and into what researchers now call the agent era. But we did not arrive here overnight. A series of research contributions, each building on the last, have made agents progressively more capable. This podcast traces that timeline through the papers that shaped today's autonomous systems. G...

The focus of LLM research is undergoing a significant shift from simply increasing model size to making existing capabilities practically usable for deployment . This week’s papers highlight four emerging themes: Reasoning & Agents, Generation & Synthesis, LLM Decisions, and Structured Episodic Memory . The overarching goal is to solve the real-world bottlenecks of agentic systems, such as high in...

Key takeaway from todays podcast: Robust Reasoning: The Holonomic Network achieves perfect fidelity extrapolation 100x beyond training lengths, demonstrating a new universality class for logical reasoning with topological stability. Efficient Inference: RelayLLM cuts inference cost by 98.2% invoking LLM only 1.07% of tokens, while GlimpRouter reduces latency by 25.9% and boosts accuracy by 10.7% v...

* Bayesian & Cognitive Advances: Transformers achieve ultra-precise Bayesian inference with 10⁻³ to 10⁻⁴ bit accuracy, while CREST boosts reasoning accuracy by 17.5% and cuts token usage by 37.6%. * Model Efficiency & Scaling: TG reduces data needs by up to 8% and parameters by 42% compared to GPT-2, and Recursive Language Models handle 100x longer inputs at similar or lower inference costs. * Sta...