Mixture of Experts

Live from Paris, Tim Hwang is at the AI Action Summit 2025. In episode 42 of Mixture of Experts, we welcome Anastasia Stasenko, CEO and Co-Founder of pleias along with our veteran experts Marina Danilevsky and Chris Hay. Last week, we touched on some potential conversations at the Paris AI Summit, this week we recap what actually happened. Is AI safety improving Globally? Next, for our paper of th...

What does Sam Altman have up his sleeve? In episode 41 of Mixture of Experts, join host Tim Hwang along with experts Nathalie Baracaldo, Marina Danilevsky and Chris Hay. Last week, we covered all things DeepSeek, and this week OpenAI has some new releases to share. Today, the experts dissect deep research and o3-mini. Next, our host Tim Hwang is travelling to AI Action Summit, he asks our experts...

Let’s bust some early myths about DeepSeek. In episode 40 of Mixture of Experts, join host Tim Hwang along with experts Aaron Baughman, Chris Hay and Kate Soule. Last week, we covered the release of DeepSeek-R1; now that the entire world is up to speed, let’s separate the facts from the hype. Next, what is model distillation and why does it matter for competition in AI? Finally, Sam Altman among o...

What does the future hold for DeepSeek? In episode 39 of Mixture of Experts, join host Tim Hwang along with experts Abraham Daniels, Kaoutar El Maghraoui and Skyler Speakman to discuss the release of DeepSeek-R1. Next, Mistral indicates going IPO. Then, FrontierMath’s new benchmark is particularly difficult, the experts debrief. Finally, IDC released a report on code assistants, what do we need to...

What would you do with $2 billion? In episode 38 of Mixture of Experts, join host Tim Hwang along with experts Chris Hay, Kaoutar El Maghraoui and Vyoma Gajjar to discuss the Anthropic valuation rumors. Next, Microsoft CEO Nadella created a new CoreAI group to build and run apps for customers. Then, NotebookLM upgraded some of its features, including podcast intervention. Finally, AI agents are ma...

What’s the most exciting CES AI announcement? In episode 37 of Mixture of Experts, host Tim Hwang is joined by Skyler Speakman, Volkmar Uhlig and Shobhit Varshney to debrief CES 2025. Specifically, the experts dive into NVIDIA’S Project DIGITS, among other announcements from the AI hardware giant. Next, a new enterprise AI development survey came out that detailing how developers really feel about...

Is deep learning hitting a wall? It’s 2025 and Mixture of Experts is back and better than ever. In episode 36, host Tim Hwang is joined by Chris Hay, Kate Soule and Kush Varshney to debrief one of the biggest releases of 2024, OpenAI o3. Next, DeepSeek-V3 is here! Finally, will AI exist in 2027? The experts dissect the AI bet between Miles Brundage and Gary Marcus. All that and more on the first M...

Will 2025 be the year of AI agents? In Episode 35 of Mixture of Experts, host Tim Hwang is joined by some show veterans to debrief 2024 in AI. This week, we review AI models, agents, hardware and product releases with some of the top industry experts. What was the best model of 2024? Is NVIDIA king? What are some of the AI trends in 2025? All that and more on this special edition of Mixture of Exp...

Is pre-training a thing of the past? In Episode 34 of Mixture of Experts, host Tim Hwang is joined by Abraham Daniels, Vagner Santana and Volkmar Uhlig to debrief this week in AI. First, OpenAI cofounder Ilya Sutskever said that “peak data” was achieved, does this mean there is no longer a need to model pre-training? Next, IBM released Granite 3.1 with a slew of features, we cover them all. Then,...

Is o1 Pro worth the cost? In Episode 33 of Mixture of Experts, host Tim Hwang is joined by Marina Danilevsky, Kate Soule and Vyoma Gajjar. First, the experts debrief the 12 Days of OpenAI. Next, we review some of the top papers in NeurIPS, how are the experts keeping up with all these research papers? Then, we are back with another benchmark, can ARC Prize make AGI more tractable? Finally, Meta an...

What’s the mystery behind the name ChatGPT refuses to discuss? In Episode 32 of Mixture of Experts host Tim Hwang dives into the hottest topics shaping the AI landscape with an all-star panel: Aaron Baughman, Vagner Figueredo de Santana, and Shobhit Varshney. First, they disect the biggest announcements and takeaways from AWS re:Invent 2024, Amazon’s premier AI event. Next, they talk about overcom...

How much future learning will be done with an AI assistant? In Episode 31 of Mixture of Experts, host Tim Hwang is joined by Phaedra Boinodiris, Marina Danilevsky and Skyler Speakman for the AI in education special episode. First, the experts give an update on the state of AI within education. Next, we cover concerns around AI safety and literacy, what do students and teachers need to be aware of?...

Should your AI assistant remember everything about you? In Episode 30 of Mixture of Experts, host Tim Hwang is joined by Vagner Santana, Vyoma Gajjar and Shobhit Varshney. First, the experts breakdown claims of “near-infinite memory” within AI models. Next, Shobhit is fresh off the plane from Microsoft Ignite, he shares some of the exciting new announcements following the event. Then, a new benchm...

Is 2024 the year scaling AI officially breaks? In Episode 29 of Mixture of Experts, host Tim Hwang is joined by Anthony Annunziata, Kate Soule and Naveen Rao. First, the experts discuss whether we are living in a post scale world. Next, we can’t have an episode without chatting AI agents, but what does the future hold for this technology? Finally, is AGI here to stay? Tune-in to this week’s Mixtur...

Could AI wipe out software engineers? In Episode 28 of Mixture of Experts, host Tim Hwang is joined by Chris Hay, Kaoutar El Maghraoui, and Shobhit Varshney. First, the experts discuss GitHub reporting a rise of developers driven by AI code assistant tools. Next, Big Sleep finds a vulnerability in SQLite, what is the future for these kinds of AI agents? Finally, OpenAI released SearchGPT, what is...

Agents, agents, and more agents! In Episode 27 of Mixture of Experts, host Tim Hwang is joined by Volkmar Uhlig and Vyoma Gajjar. First, the experts chat about Mark Benioff’s spicy tweet, and what this means for the future of AI agents. Next, how much energy is needed to power AI models, and should we be concerned? Then, the experts debrief Anthropic’s release of computer use. Finally, Google is i...

Can chat replace search? In Episode 26 of Mixture of Experts, host Tim Hwang is joined by Kate Soule, Kush Varshney and Petros Zerfos for IBM TechXchange week! First, the experts describe how the team created the Granite 3.0 models. Next, NVIDIA enters the open source model game, what does this mean for the competition? Finally, Perplexity AI is seeking over double their valuation in new funding r...

Can AI solve infectious disease? In Episode 25 of Mixture of Experts, host Tim Hwang is joined by Kaoutar El Maghraoui, Maya Murad, and Ruben Boonen. Today we analyze some papers. First, the experts dissect Machines of Loving Grace, a 15,000 word essay written by Anthropic’s CEO making some major AI predictions. Then, Apple generated a new benchmark based of GSM8K in a recent paper, the findings w...

Could AI win a Nobel Prize in the future? In Episode 24 of Mixture of Experts, host Tim Hwang is joined by Chris Hay and Edward Calvesbert. First, the experts debrief the ‘Godfather of AI’ sharing a Nobel Prize. Next, we talk AI platforms and the hype around DGX B200. Finally, unstructured data is becoming usable for LLMs, why are companies like NVIDIA so interested in this data? Tune-in today to...

Will DeepDive replace the Mixture of Experts podcast? In Episode 23, host Tim Hwang is joined by IBM Researchers Marina Danilevsky, Nathalie Baracaldo and Vagner Santana to dissect this week’s AI news. First, the experts talk about the hype around Google’s NotebookLM, specifically regarding the DeepDive podcast feature. Next, OpenAI DevDay sparks some interesting conversation around vision fine-tu...

Meta releases Llama 3.2! In Episode 22 of Mixture of Experts, host Tim Hwang is joined by Maryam Ashoori, Skyler Speakman, and Shobhit Varshney to debrief an exciting week of AI news. First, Meta is back with the release of Llama 3.2, and lightweight (1B/3B) models. Next, it’s Climate Week NYC, we chat the use of gen AI in achieving sustainable development goals. Specifically, IBM and NASA’s AI mo...

Meta releases Llama 3.2! In Episode 22 of Mixture of Experts, host Tim Hwang is joined by Maryam Ashoori, Skyler Speakman, and Shobhit Varshney to debrief an exciting week of AI news. First, Meta is back with the release of Llama 3.2, and lightweight (1B/3B) models. Next, it’s Climate Week NYC, we chat the use of gen AI in achieving sustainable development goals. Specifically, IBM and NASA’s AI mo...

Can Apple Intelligence compete with the AI market offerings? In Episode 20 of Mixture of Experts, host Tim Hwang is joined by Marina Danilevsky, Kate Soule and Maya Murad. Today, the experts chat Apple Intelligence, the performance of Reflection’s 70B, and a new paper released on LLMs generating novel research ideas. Additionally, IBM soft launched the Bee Agent Framework to help build agentic wor...

Will prompt engineering ever die? In Episode 19 of Mixture of Experts, host Tim Hwang is joined by Kaoutar El Maghraoui, Kate Soule and Shobhit Varshney. Today, the experts chat the future of prompt engineering, a new paper released about The AI Scientist, NEO 1X’s humanoid robot, and OpenAI’s in-house AI chips. Will AI takeover scientific discovery? Will everyone have at home AI assistants? Why i...

Is search less trustworthy? In Episode 18 of Mixture of Experts, host Tim Hwang is joined by the IBM Fellows—Aaron Baughman, Kush Varshney, and Trent Gray-Donald. Today, the experts chat how AI is being integrated at the US Open. Next, the Perplexity is introducing ads in Q4, what is the affect on search? Finally, what's all the hype with Cursor? Tune-in to today’s episode for all this and more. 0...