The AutoML Podcast

AutoML is dead an LLMs have killed it? MLGym is a benchmark and framework testing this theory. Roberta Raileanu and Deepak Nathani discuss how well current LLMs are doing at solving ML tasks, what the biggest roadblocks are, and what that means for AutoML generally. Check out the paper: https://arxiv.org/pdf/2502.14499 More on Roberta: https://rraileanu.github.io/ More on Deepak: https://dnathani....

Where and how can we use foundation models in AutoML? Richard Song, researcher at Google DeepMind, has some answers. Starting off from his position paper on leveraging foundation models for optimization, we chat about what makes foundation models valuable for AutoML, how the next steps could look like, but also why the community is not currently embracing the topic as much as it could. Paper Link:...

Oscar Beijbom is talking about what it's like to run an AutoML startup: Nyckel. Beyond that, we chat about the differences between academia and industry, what truly matters in application and more. Check out Nyckel at: https://www.nyckel.com/

Colin White, head of research at Abacus AI, takes us on a tour of Neural Architecture Search: its origins, important paradigms and the future of NAS in the age of LLMs. If you're looking for a broad overview of NAS, this is the podcast for you!

There are so many great foundation models in many different domains - but how do you choose one for your specific problem? And how can you best finetune it? Sebastian Pineda has an answer: Quicktune can help select the best model and tune it for specific use cases. Listen to find out when this will be a Huggingface feature and if hyperparameter optimization is even important in finetuning models (...

Designing algorithms by hand is hard, so Chris Lu and Matthew Jackson talk about how to meta-learn them for reinforcement learning.  Many of the concepts in this episode are interesting to meta-learning approaches as a whole, though: "how expressive can we be and still perform well?", "how can we get the necessary data to generalize?" and "how do we make the resulting algo...

AutoML can be a tool for good, but there are pitfalls along the way. Rahul Sharma and David Selby tell us about how AutoML systems can be used to give us false impressions about explainability metrics of ML systems - maliciously, but also on accident. While this episode isn't talking about a new exciting AutoML method, it can tell us a lot about what can go wrong in applying AutoML and what w...

In today's episode, we're introducing the very special Theresa Eimer to the show. Theresa will be taking over the hosting of many of the future episodes. Theresa has already recorded multiple episodes and we are stoked to air those shortly. We also spend a few moments explaining my relative absence in the last few months (since the war in the middle east erupted) and what I'm up to...

Today we're talking with Nick Erickson from AutoGluon. We discuss AutoGluon's fascinating origin story, its unique point of view, the science and engineering behind some of its unique contributions, Amazon's Machine Learning University, AutoGluon's multi-layer stack ensembler in all its detail, their feature preprocessing pipeline, their feature type inference, their adaptive a...

Today we're talking with Joseph Giovanelli about his work on integrating logic and argumentation into AutoML systems. Joseph is a PhD student at the University of Bologna. He was more recently in Hannover working on ethics and fairness with Marius’ team. The paper he published presents his framework, HAMLET, which stands for Human-centric AutoML via Logic and Argumentation. It allows a user t...

Today we're talking with Caitlin Owen, a post-doc at the University of Otago about her work on error decomposition. She recently published a paper titled "Towards Explainable AutoML Using Error Decomposition" about how a more granular view of the components of error can lead the construction of better AutoML systems. Read her paper here: https://link.springer.com/chapter/10.1007/978...

Today we're talking with Gaurav Rao, the EVP & GM of Machine Learning and AI at AtScale, a company centered around the semantic layer. For some time now, I've been feeling that there is a deep connection between a formal articulation of business context and the realization of the dream of AutoML, so I searched for people in the space who can help shine light on this direction. Gaurav...

Today Ankush Garg is speaking with Rishi Bommasani, PhD student at Stanford and one of the originator of the term Foundation Models. They’re talking about the origins of the term Foundation Model, which he and his group advanced, in the paper "On the Opportunities and Risks of Foundation Models". They’ll talk about self-supervision, issues of scale, the motivation behind the terminology,...

Today we're talking with Raymond Peck, a senior engineer and director in the AutoML space. He spent time at H2O, dotData, Alteryx and many other places. This is a fascinating conversation about the business, engineering, and science of machine learning automation in production. Learning about his experience is crucial for understanding the biography of the space. We discuss the early motivati...

Today we’re talking to Noah Hollmann and Samuel Muller about their paper on TabPFN - which is an incredible spin on AutoML based on Bayesian inference and transformers. [Quick note on audio quality]: Some of the tracks have not recorded perfectly but I felt that the content there was too important not to release. Sorry for any ear-strain! In the episode, we spend some time discussing posterior pre...

Today we’re talking to Sean Sexton, the Director of Modeling and Analytics Consulting at KPMG, about the role of models in financial institutions and how the risks associated with them is managed. This turned out to be an incredibly deep and interesting topic, and we really only scratched the surface of it. Sean has a unique ability to summarize developments in an entire space. If you're inte...

Today we’re talking to Matt Levine. Matt is a PhD student in computing and mathematical sciences at Caltech, and he focuses on improving the prediction and inference of physical systems by blending together both mechanistic modeling and  machine learning. This episode is one of my favorites: we go pretty deep into dynamical systems, and into Matt's new framework for solving them by blending t...

Today we’re chatting with Junhong Shen, a PhD student at Carnegie Mellon. Junhong and her team are working on the generalizability of NAS algorithms across a diverse set of tasks. Today we'll be talking about DASH, a NAS algorithm that takes diversity of tasks at its center. In order to implement DASH, Junhong and her team implemented three clever ideas that she'll share with us. Efficie...

Today we're speaking with Marius Lindauer and it is certainly one of my favorite episodes! As you’ll hear, Marius is full of ideas for where AutoML systems can and should go. These ideas are crystallized in a blog-post, published here: https://www.automl.org/rethinking-automl-advancing-from-a-machine-centered-to-human-centered-paradigm/ If you’re searching for research directions, this conver...

Today Ankush Garg is talking to Mehdi Bahrami about his recent project: BERT-Sort. BERT-Sort is an example of how large language models can add useful context to tabular datasets, and to AutoML systems. Mehdi is a Member of Research Staff at Fujitsu and, as he describes, he began using AutoML systems for his research, yet he came across some crucial limitations of existing solutions. The modificat...

In today's episode, we’re talking to Lars Kothoff about the fascinating origin story of AutoML (as he sees it), and how it emerged from the SAT community. While talking to many of you, it became clear that this origin story is one that a lot of people have some vague sense about, but not a very concrete knowledge of so hopefully this episode can help to flesh out the narrative with greater cl...

Today we’re talking to Cedric Kulbach about online learning, the challenges of doing it properly, why it is so promising, how it’s connected to evolutionary strategies, and recent advances in the field that can help to unlock these promises. We then discuss the close connection between online learning and AutoML systems, and we explore a recent framework that he recently published, called EvoAutoM...

This short episode is a narration of Richard Sutton's The Bitter Lesson. Richard Sutton is a distinguished research scientist at DeepMind and a professor of computing science at the University of Alberta. He is considered one of the founders of modern computational reinforcement learning, having several significant contributions to the field, including temporal difference learning and policy...

More drama in the contest between traditional machine learning models and deep learning models when it comes to tabular data. We have on the show Vadim Borisov, a research fellow at the University of Tubingen as well as Kathrin Sessler, a PhD student from the same university. This episode will be led by Ankush Garg, exploring Vadim and Kathrin's recent paper “Deep Neural Networks and Tabular...

Today we’re speaking with Jeff Clune about a new path towards general artificial intelligence, that he calls AI Generating Algorithms (AI-GAs). Jeff is a Professor of Computer Science at the University of British Columbia. He was previously a Senior Research Scientist at Uber AI, and more recently a Research Team Leader at OpenAI. He's currently a Faculty Member at the Vector Institute. We’re...