Neural intel Pod

Neural Intel Podcast Episode MoE Giants: Decoding the 670 Billion Parameter Showdown Between DeepSeek V3 and Mistral Large This week on Neural Intel, we dive deep into the architectural blueprints of two colossal Mixture-of-Experts (MoE) models:  DeepSeek V3  (673B/671B) and  Mistral 3 Large  (675B/673B). We explore the configurations that define these massive language models, noting their shared...

In this episode of the Neural Intel podcast, we go under the hood of GLM-4.7 , the newest native agentic LLM from Z.AI. Released on December 22, 2025, this model represents a massive 41% reasoning improvement over its predecessor, GLM-4.6.We discuss the strategic decision to release an incremental 4.7 update rather than jumping to version 5.0, focusing on how Z.AI has optimized tool orchestration...

In this deep-dive episode, Neural Intel goes behind the data of the Medmarks v0.1 benchmark suite , led by Sophont and the MedARC community. While previous benchmarks like MultiMedQA have "saturated," Medmarks introduces MedXpertQA , a reasoning-heavy task that currently pushes even the strongest frontier models to their limits. We examine the technical nuances of the study:• Thinking vs...

In this episode of Neural Intel, we break down Andrej Karpathy’s "2025 LLM Year in Review," exploring the massive paradigm shifts that redefined artificial intelligence over the last year. From the technical evolution of the training stack to the cultural phenomenon of "vibe coding," 2025 marked the transition from simple chatbots to "summoned ghosts" and autonomous a...

This episode dives into neural_net_checklist , the indispensable PyTorch toolkit designed to automate the crucial diagnostic process for training complex neural networks. Inspired by Andrei Karpathy's seminal blog post, "A Recipe for Training Neural Networks," this repository transforms a manual debugging guide into a set of programmatic assertions, saving developers significant time...

This episode of Neural Intel dives deep into the NVIDIA Nemotron 3 Nano (30B A3B) , the foundational model of the new Nemotron 3 family engineered specifically for scalable, trustworthy agentic AI systems. We break down the breakthrough Hybrid Mamba-Transformer Mixture-of-Experts (MoE) architecture , a design that strategically decouples the model's total capacity of 31.6B parameters from its...

Join us for a deep technical discussion on Olmo 3 , the latest family of state-of-the-art, fully open language models developed by the Olmo Team at the Allen Institute for AI (Ai2). Targeting the specialized audience of ML insiders, this episode dissects the entire model flow —a commitment to releasing the full lifecycle, including every stage, checkpoint, datapoint, and dependency used to build t...

This episode breaks down OpenAI's urgent launch of GPT-5.2 on December 11, 2025, a release explicitly labeled an internal " code red " response to the competitive lead established by Google’s Gemini 3 model. We examine the unprecedented acceleration of model velocity, as the jump from GPT-5.1 to 5.2 occurred in less than a month. The episode delves into the key technical advancement:...

Join us for a deep dive into Fara-7B , Microsoft Research's first agentic Small Language Model (SLM) designed specifically for computer use . This open-weight, ultra-compact model is pushing the frontiers of computer-use agents, optimized for real-world web tasks. As ML insiders, discover how Fara-7B achieves state-of-the-art performance within its size class (only 7 billion parameters ) and i...

Dive deep into the extraordinary journey of DeepMind and its relentless pursuit of Artificial General Intelligence (AGI) . This episode draws on the recollections of founders and early scientists, detailing the ambition to create a "general learning machine" capable of cognitive breadth and flexibility akin to human intelligence. Key Topics for the ML Community: • Reinforcement Learning...

Dive into the technical architecture and training pipeline behind INTELLECT-3 , a 106B-parameter Mixture-of-Experts model (12B active) that achieves state-of-the-art performance for its size across math, code, science, and reasoning benchmarks, outperforming many larger frontier models. This episode provides an insider look into the large-scale reinforcement learning (RL) infrastructure stack deve...

Key topics covered include: •  K2 Model Development:  Yang Zhilin details the technical breakthroughs in K2, emphasizing the focus on  Token Efficiency  (getting more intelligence from the same amount of data) using non-Adam optimization techniques like the  MOG optimizer . •  Agentic LLMs:  The shift from "Brain in a Vat" models (pure reasoning) to  Agentic LLMs  that interact with the...

Ilya Sutskever, a leading figure in AI and CEO of SSI, declared that the  "age of scaling"  is ending, marking a return to the  "age of research" . He outlines the most fundamental bottleneck facing modern AI: the severe lack of  generalization  compared to human learning. Sutskever explores the paradox of today's models that  "seem smarter than their economic impact w...

The assessment of an Intel Technology YouTube video provides an overview of  neuromorphic computing , a field inspired by the architecture and efficiency of the  biological brain . Narrated by Intel's Mike Davies, the text explains that early computer pioneers were influenced by the brain, and today's research aims to replicate the brain's features, like its  incredible speed and low p...

Google has officially ushered in "A new era of intelligence with Gemini 3," releasing what it describes as its most intelligent model yet, designed to help users bring any idea to life. The launch of Gemini 3 Pro (available in preview) on November 18, 2025, represents a significant step on the path toward AGI

The episode provides a technical overview of  DeepSeek-OCR , a new end-to-end Vision-Language Model (VLM) designed specifically for Optical Character Recognition (OCR) tasks, emphasizing  vision-text compression . The core innovation is the  DeepEncoder  architecture, which minimizes vision tokens and activation memory for high-resolution images by serially connecting a local attention component (...

This source is an academic paper that investigates whether large language models ( LLMs ) can develop behavioral patterns analogous to human gambling addiction. The researchers conducted experiments on four different LLMs using a negative expected value slot machine task, finding that models consistently displayed core cognitive biases like  loss chasing  and the  illusion of control  when given t...

The provided text is an excerpt from the pre-print service  arXiv , promoting its support for  Open Access Week  while presenting information about a new paper submission. The paper, titled  "Glyph: Scaling Context Windows via Visual-Text Compression,"  proposes a novel framework called  Glyph  that addresses the computational challenges of large language models (LLMs) with extensive con...

This research paper proposes a novel approach to address  catastrophic forgetting  in large language models (LLMs) during continual learning, introducing  sparse memory finetuning . This method utilizes  memory layer models , which are designed for sparse updates, by selectively training only the memory slots that are highly activated by new knowledge relative to existing information, using a  TF-...

On today's episode we cover Dwarkesh Patel's recent interview with  Andrej Karpathy , discussing his views on the future of  Large Language Models (LLMs)  and  AI agents . Karpathy argues that the full realization of competent AI agents will take a  decade , primarily due to current models'  cognitive deficits , lack of continual learning, and insufficient multimodality. He contrasts t...

Today we provide an overview of the escalating  legal conflicts between Elon Musk's entities (xAI and X Corp.) and OpenAI , a company Musk co-founded. The core dispute involves  two major lawsuits : one filed by xAI alleging that OpenAI engaged in  systematic trade secret theft  by unlawfully poaching employees with knowledge of xAI’s Grok chatbot and business plans, and a second antitrust cla...

This episode offers a comprehensive overview of  IBM's newly released Granite 4.0 family of open-source language models , highlighting their  innovative hybrid Mamba-2/transformer architecture . This new design is consistently emphasized for its  hyper-efficiency , leading to significantly lower memory requirements and faster inference speeds, particularly crucial for long-context and enterpri...

The provided sources announce and review the launch of  Anthropic's Claude Sonnet 4.5  large language model, positioning it as the company's most advanced tool, particularly for  coding and complex agentic workflows . Multiple articles and a Reddit discussion highlight its superior performance on  coding benchmarks  like SWE-Bench Verified, claiming it often surpasses the flagship Opus mod...

The provided sources offer an extensive overview of OpenAI's recent release,  GPT-5-Codex , a specialized agentic model designed for software engineering tasks. The articles and discussions highlight the model's key differentiating feature,  "variable grit,"  which allows it to dynamically adjust its reasoning time, tackling simple tasks quickly while persistently working on comp...

These sources provide an extensive overview of  xAI’s Grok 4 Fast  model, positioning it as a speed-optimized variant of Grok 4 that prioritizes low latency and cost-efficiency for high-volume, quick interactions, particularly in  coding and developer workflows . The texts explain that Grok 4 Fast achieves performance comparable to the flagship Grok 4 on key benchmarks while using  40% fewer &quot...