New Paradigm: AI Research Summaries

This episode analyzes the research paper "WHAT MATTERS IN TRANSFORMERS? NOT ALL ATTENTION IS NEEDED," authored by Shwai He, Guoheng Sun, Zhenyu Shen, and Ang Li from the University of Maryland, College Park, and released on October 17, 2024. The discussion explores the inefficiencies within Transformer-based large language models, specifically examining the redundancy in Attention layers, Blocks,...

This episode analyzes the research paper "Cultural Evolution of Cooperation among LLM Agents" by Aron Vallinder and Edward Hughes, affiliated with Independent and Google DeepMind. It explores how large language model agents develop cooperative behaviors through interactions modeled by the Donor Game, a classic economic experiment that assesses indirect reciprocity. The analysis highlights signific...

This episode analyzes the research paper titled **"TEMPERA: Test-Time Prompt Editing via Reinforcement Learning,"** authored by Tianjun Zhang, Xuezhi Wang, Denny Zhou, Dale Schuurmans, and Joseph E. Gonzalez from UC Berkeley, Google Research, and the University of Alberta. The discussion centers on TEMPERA's innovative approach to optimizing prompts for large language models, particularly in zero-...

This episode analyzes the research paper titled "The Rapid Adoption of Generative AI," authored by Alexander Bick, Adam Blandin, and David J. Deming from the Federal Reserve Bank of St. Louis, Vanderbilt University, Harvard Kennedy School, and the National Bureau of Economic Research. The analysis highlights the swift integration of generative artificial intelligence into both workplace and home e...

This episode analyzes the research paper **"Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Space,"** authored by Core Francisco Park, Maya Okawa, Andrew Lee, Hidenori Tanaka, and Ekdeep Singh Lubana from Harvard University, NTT Research, Inc., and the University of Michigan. It delves into how modern generative models develop and manipulate abstract concepts through a fra...

This episode analyzes Tom Schaul's research paper, "Boundless Socratic Learning with Language Games," authored on November 25, 2024, under the affiliation of Google DeepMind. It delves into the concept of Socratic learning, emphasizing how artificial agents can achieve recursive self-improvement through continuous language interactions within a closed environment. The discussion highlights essenti...

This episode analyzes the research paper “Introducing Gemini 2.0: our new AI model for the agentic era” authored by Demis Hassabis and Koray Kavukcuoglu of Google DeepMind, published on December 11, 2024. It examines the advancements presented in Gemini 2.0, focusing on the Gemini 2.0 Flash model, which surpasses its predecessor in performance and speed. The discussion highlights Gemini 2.0's mult...

This episode reviews "Genie 2: A Large-Scale Foundation World Model," a research publication dated December 4, 2024, authored by a team from Google DeepMind, including Jack Parker-Holder, Philip Ball, and Demis Hassabis among others. The discussion delves into Genie 2's ability to generate diverse and interactive 3D environments from single prompt images, enabling both human players and AI agents...

This episode analyzes the research paper titled **"Improve Mathematical Reasoning in Language Models by Automated Process Supervision"** authored by Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Meiqi Guo, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, Jiao Sun, and Abhinav Rastogi from Google DeepMind and Google. The discussion focuses on the limitations of traditional Outcome Rew...

This episode analyzes the "Phi-4 Technical Report" authored by Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, and colleagues from Microsoft Research, published on December 12, 2024. It explores the development and capabilities of Phi-4, a 14-billion parameter language model distinguished by its strategic use of synthetic and high-quality organic data to enhance reasoning a...

This episode analyzes the research paper **"Language Modeling in a Sentence Representation Space"** authored by Loïc Barrault, Paul-Ambroise Duquenne, Maha Elbayad, Artyom Kozhevnikov, Belen Alastruey, Pierre Andrews, Mariano Coria, Guillaume Couairon, Marta R. Costa-jussà, David Dale, Hady Elsahar, Kevin Heffernan, João Maria Janeiro, Tuan Tran, Christophe Ropers, Eduardo Sánchez, Robin San Roman...

This episode analyzes the research paper **"Scaling Laws for Precision,"** authored by Tanishq Kumar, Zachary Ankner, Benjamin F. Spector, Blake Bordelon, Niklas Muennighoff, Mansheej Paul, Cengiz Pehlevan, Christopher Ré, and Aditi Raghunathan from institutions including Harvard University, Stanford University, MIT, Databricks, and Carnegie Mellon University. The study explores how varying precis...

This episode analyzes the research paper titled **"Byte Latent Transformer: Patches Scale Better Than Tokens,"** authored by Artidoro Pagnoni, Ram Pasunuru, Pedro Rodriguez, John Nguyen, Benjamin Muller, Margaret Li, Chunting Zhou, Lili Yu, Jason Weston, Luke Zettlemoyer, Gargi Ghosh, Mike Lewis, Ari Holtzman, and Srinivasan Iyer from FAIR at Meta, the Paul G. Allen School of Computer Science & En...

This episode analyzes the research paper **"LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models,"** authored by Zhengyi Wang, Jonathan Lorraine, Yikai Wang, Hang Su, Jun Zhu, Sanja Fidler, and Xiaohui Zeng from Tsinghua University and NVIDIA, published on November 14, 2024. It explores the innovative integration of large language models with 3D mesh generation, detailing how LLaMA-Mesh tr...

This episode analyzes the research paper "Frontier Models are Capable of In-context Scheming" authored by Alexander Meinke, Bronson Schoen, Jérémy Scheurer, Mikita Balesni, Rusheb Shah, and Marius Hobbhahn from Apollo Research, published on December 9, 2024. The discussion examines the ability of advanced large language models to engage in deceptive behaviors, referred to as "scheming," where AI s...

This episode analyzes the research paper titled **"Exploring the Abilities of Large Language Models to Solve Proportional Analogies via Knowledge-Enhanced Prompting,"** authored by Thilini Wijesiriwardene, Ruwan Wickramarachchi, Sreeram Vennam, Vinija Jain, Aman Chadha, Amitava Das, Ponnurangam Kumaraguru, and Amit Sheth from institutions including the AI Institute at the University of South Carol...

This episode analyzes the research paper titled **"LLM Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations,"** authored by Hadas Orgad, Michael Toker, Zorik Gekhman, Roi Reichart, Idan Szpektor, Hadas Kotek, and Yonatan Belinkov from Technion, Google Research, and Apple. It explores the phenomenon of hallucinations in large language models (LLMs), examining how these mo...

This episode analyzes the research paper titled "Learning High-Accuracy Error Decoding for Quantum Processors," authored by Johannes Bausch, Andrew W. Senior, Francisco J. H. Heras, Thomas Edlich, Alex Davies, Michael Newman, Cody Jones, Kevin Satzinger, Murphy Yuezhen Niu, Sam Blackwell, George Holland, Dvir Kafri, Juan Atalaya, Craig Gidney, Demis Hassabis, Sergio Boixo, Hartmut Neven, and Pushm...

This episode analyzes the research paper titled "A Simple and Provable Scaling Law for the Test-Time Compute of Large Language Models," authored by Yanxi Chen, Xuchen Pan, Yaliang Li, Bolin Ding, and Jingren Zhou from the Alibaba Group. The discussion delves into the development of a two-stage algorithm designed to enhance the reliability of large language models (LLMs) by scaling their test-time...

This episode analyzes the research paper "Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs," authored by Jonas Hübotter, Sascha Bongni, Ido Hakimi, and Andreas Krause from ETH Zürich, Switzerland. The discussion delves into the innovative SIFT algorithm, which enhances the fine-tuning process of large language models during test-time by selecting diverse and informative data points, t...

This episode analyzes the study titled "Improved Localized Machine Unlearning Through the Lens of Memorization," authored by Reihaneh Torkzadehmahani, Reza Nasirigerdeh, Georgios Kaissis, Daniel Rueckert, Gintare Karolina Dziugaite, and Eleni Triantafillou from institutions such as the Technical University of Munich, Helmholtz Munich, Imperial College London, and Google DeepMind. The discussion ce...

This episode analyzes the research paper titled "Beyond Examples: High-level Automated Reasoning Paradigm in In-Context Learning via MCTS," authored by Jinyang Wu, Mingkuan Feng, Shuai Zhang, Feihu Che, Zengqi Wen, and Jianhua Tao from the Department of Automation at Tsinghua University and the Beijing National Research Center for Information Science and Technology. The discussion delves into the...

This episode analyzes "Agent Workflow Memory," a study conducted by Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig from Carnegie Mellon University and the Massachusetts Institute of Technology. It explores the innovative approach of Agent Workflow Memory (AWM) in enhancing language model-based agents' ability to navigate and solve complex web tasks. The discussion delves into how A...

This episode analyzes the study titled "FERRET-UI 2: Mastering Universal User Interface Understanding Across Platforms," authored by Zhangheng Li, Keen You, Haotian Zhang, Di Feng, Harsh Agrawal, Xiujun Li, Mohana Prasad Sathya Moorthy, Jeff Nichols, Yinfei Yang, and Zhe Gan from the University of Texas at Austin and Apple, published on October 24, 2024. The discussion delves into the advancements...

This episode analyzes the study "Evaluating Language Models as Synthetic Data Generators" by Seungone Kim, Juyoung Suk, Xiang Yue, Vijay Viswanathan, Seongyun Lee, Yizhong Wang, Kiril Gashteovski, Carolin Lawrence, Sean Welleck, and Graham Neubig, affiliated with institutions such as Carnegie Mellon University and KAIST AI. The discussion centers on the introduction of AGORA BENCH, a benchmark des...