Benchtalks

For our third Benchtalks, the series dedicated to the researchers building the measurement toolkits that frontier labs hill-climb on, Snorkel AI co-founder Vincent Sunn Chen sat down with Parth Asawa , a UC Berkeley PhD student advised by Matei Zaharia and Joey Gonzalez , and the creator of Continual Learning Bench , the first standardized benchmark for measuring whether AI systems actually learn...

For our second Benchtalks , the series dedicated to the researchers building the measurement toolkits that frontier labs hill-climb on, Snorkel AI co-founder Vincent Sunn Chen sat down with John Yang , a Stanford PhD student and creator of the SWE-bench franchise, SWE-smith, CodeClash, and most recently ProgramBench . This interview covers:  Why every frontier model scored 0% at launch — until GPT...

In this inaugural episode of Benchtalks, Snorkel AI co-founder Vincent Chen sits down with Alex Shaw, MTS at Laude Institute and co-creator of Terminal-Bench, to unpack what the rapid hill-climbing on TB2 reveals about the state of AI agent evaluation — and where the field needs to go. This interview covers:  Why TB2 went from 20–30% during development to 75–80% at the frontier today The bet on th...