Enoch H. Kang

Best AI papers explained

Cut through the noise. We curate and break down the most important AI papers so you don’t have to.

Vizitează neapărat site-ul podcastului și susține-i creatorul: podcasters.spotify.com

Autor

Enoch H. Kang

Categorie

Technology

Site-ul podcastului

podcasters.spotify.com

Cel mai nou episod

19 nov. 2026

Unde asculți?

Podcasturi în aplicație Replaio Radio În curând

Podcasturile ajung în curând în aplicație. Instaleaz-o acum și fii primul care descoperă o abordare complet nouă a podcasturilor

Descarcă din Google Play Instalează gratuit Android 10 mil.+ descărcări · nota 4,7+ iOS în curând

Episoade

Language models are injective and hence invertible 30.10.2025

The academic paper argues that decoder-only Transformer language models, such as GPTs, are almost surely injective, meaning that distinct input prompts map to distinct internal hidden states, preserving input information without loss. This contrasts with the common assumption that non-linear components make models lossy. The authors mathematically prove that this injectivity is a structural proper...

ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory 29.10.2025

This paper introduces **ReasoningBank**, a novel memory framework designed to enhance Large Language Model (LLM) agents by distilling and structuring reasoning patterns from both successful and failed task trajectories. Traditional memory systems typically overlook failure experiences and lack the ability to abstract high-level reasoning, a limitation ReasoningBank addresses by creating **structur...

RLAD: Training LLMs to Discover Abstractions 29.10.2025

This paper introduces a novel two-player reinforcement learning (RL) framework, RLAD, designed to enhance the reasoning capabilities of large language models (LLMs). This framework jointly trains an **abstraction generator** and an **abstraction-conditioned solution generator** to propose and utilize **concise natural language descriptions of procedural and factual knowledge** called "reasoning ab...

How to Train Your Advisor: Steering Black-Box LLMs with ADVISOR MODELS 29.10.2025

The academic paper introduces **ADVISOR MODELS**, a novel framework for dynamically steering the behavior of rigid, **black-box Large Language Models (LLMs)** that are only accessible via an API. Unlike static prompting methods, this approach employs a second, lightweight model, the "advisor," which is trained using **reinforcement learning (RL)** to generate instance-specific, natural language ad...

Self-improving LLM agents at Test-Time 27.10.2025

This is research paper introduces and evaluates a novel framework called Test-Time Self-Improvement (TT-SI) for large language model (LLM) agents. This approach focuses on improving model performance efficiently during inference by adapting to challenging examples on the fly. The method involves three key steps: Self-Awareness (identifying uncertain test inputs), Self-Data Augmentation (generating...

KL-Regularized Reinforcement Learning is designed to Mode Collapse 27.10.2025

The academic paper investigates the common belief that Kullback-Leibler (KL) regularized reinforcement learning (RL) objectives, particularly when used for post-training large language models (LLMs), inherently promote or inhibit output diversity based on the choice between reverse and forward KL divergence. The authors challenge this intuition, demonstrating both mathematically and empirically th...

How do LLMs use their depth? 27.10.2025

The research paper explores how Large Language Models (LLMs) utilize their depth during inference, proposing a "Guess-then-Refine" framework to explain layer-wise prediction dynamics. The authors use the TunedLens method to trace intermediate representations, revealing that early layers function as "statistical guessers" by promoting high-frequency tokens as initial predictions...

Thought Communication in Multiagent Collaboration 27.10.2025

The academic paper proposes "thought communication," a new paradigm for multi-agent collaboration that allows large language models (LLMs) to exchange latent thoughts directly, akin to telepathy, instead of relying on lossy natural language. The authors formalize this process using a latent variable model where agent states are generated from underlying thoughts, proving that both shared...

Reasoning with Sampling: Base Models Outperform RL 26.10.2025

The research paper titled "Reasoning with Sampling: Your Base Model is Smarter Than You Think" by Harvard researchers introduces a novel, training-free iterative sampling algorithm inspired by Markov Chain Monte Carlo (MCMC) techniques to enhance the reasoning capabilities of large language models (LLMs) at inference time. This method, termed "Power Sampling," leverages the bas...

Continual Learning via Sparse Memory Finetuning 26.10.2025

The paper by Meta and Berkeley proposes a novel approach to address catastrophic forgetting in large language models (LLMs) during continual learning, introducing sparse memory finetuning. This method utilizes memory layer models, which are designed for sparse parameter updates, to selectively update only the memory slots that are highly activated by new knowledge relative to existing, pre-trainin...

Direct Preference Optimization with Unobserved Preference Heterogeneity: The Necessity of Ternary Preferences 24.10.2025

The academic paper claims that pairwise-comparison-based RLHF is incapable of learning heterogeneous preferences, whereas tenary comparisons can. They propose **Expectation-Maximization Direct Preference Optimization (EM-DPO)**, a clustering algorithm that discovers latent user preference groups and trains an ensemble of specialized LLMs for each group. Crucially, the authors establish a theoretic...

The Coverage Principle: How Pre-Training Enables Post-Training 24.10.2025

This paper provides a theoretical analysis of next-token prediction in language models, introducing the concept of the coverage profile ($\text{Cov}_N$) as a superior metric to cross-entropy for predicting downstream performance with Best-of-N (BoN) sampling. The authors establish a "coverage principle," demonstrating that maximum likelihood, or next-token prediction, implicitly optimize...

The Era of Real-World Human Interaction: RL from User Conversations 24.10.2025

This paper introduces Reinforcement Learning from Human Interaction (RLHI), a new method for aligning large language models by learning directly from in-the-wild user conversations rather than expert-annotated data. This paradigm is built on two complementary approaches: User-Guided Rewrites, which leverage users' natural language follow-ups to revise unsatisfactory model outputs, and User-Bas...

Agent Learning via Early Experience 24.10.2025

This paper discusses the "early experience" paradigm as a method for training autonomous language agents, aiming to bridge the gap between reward-free Imitation Learning (IL) and reward-dependent Reinforcement Learning (RL). This novel approach allows agents to learn from their own generated interactions, or "experience," without needing explicit external rewards, addressing a...

Demystifying the Mechanisms Behind Emergent Exploration in Goal-conditioned RL 22.10.2025

This paper examines emergent exploration in reinforcement learning, specifically using a goal-conditioned contrastive learning algorithm called SGCRL. The authors employ methodologies inspired by cognitive science, such as rational analysis and controlled intervention experiments, to analyze the implicit drivers of agent behavior in this reward-free setting. They demonstrate both theoretically and...

Rewriting History: A Recipe for Interventional Analyses to Study Data Effects on Model Behavior 22.10.2025

This paper introduces an experimental recipe for interventional analyses designed to study how training data specifically affects the behavior of language models (LMs). This methodology, termed "Rewriting History," involves a three-stage process: selecting target evaluation items, matching relevant pretraining documents to those items, and then modifying those documents before retraining...

A Definition of AGI 22.10.2025

This paper attempts to provide a comprehensive framework grounded in the Cattell-Horn-Carroll (CHC) theory of cognitive abilities, which breaks down general intelligence into ten core, equally weighted cognitive components, such as General Knowledge (K), Mathematical Ability (M), and various forms of Memory (WM, MS, MR). The paper details specific, measurable tasks and examples—often drawn from hu...

Provably Learning from Language Feedback 21.10.2025

This paper introduces a new formal framework called Learning from Language Feedback (LLF), which addresses the challenge of training AI agents, particularly large language models (LLMs), using rich natural language critiques and guidance instead of traditional scalar rewards. The authors formalize the LLF problem and introduce the transfer eluder dimension as a complexity measure to quantify how e...

In-Context Learning for Pure Exploration 21.10.2025

This paper introduces In-Context Pure Exploration (ICPE), a Transformer-based architecture designed to efficiently solve active sequential hypothesis testing problems, also known as pure exploration. ICPE meta-trains a model to map observation histories to actions and predicted hypotheses, enabling in-context learning to actively gather data and infer the correct hypothesis on new tasks without re...

On the Role of Preference Variance in Preference Optimization 20.10.2025

This academic paper investigates the concept of Preference Variance (PVar) as a metric for improving the efficiency of Direct Preference Optimization (DPO), a method for aligning large language models (LLMs) with human feedback. The authors establish a theoretical foundation demonstrating that the magnitude of the DPO training gradient is bounded by the PVar of a given prompt, meaning prompts with...

Training LLM Agents to Empower Humans 20.10.2025

This rsearch paper by Sergey Levine's group introduces a self-supervised method for fine-tuning Large Language Model (LLM) agents to be more effective and aligned assistants, particularly in code generation. The core idea is to train agents to maximize the human's empowerment, defined as the user's ability to effect desired changes in the environment, rather than relying on costly expl...

Richard Sutton Declares LLMs a Dead End 20.10.2025

Once again, instead of introducing new research, we discuss what Richard Sutton, a pioneer of reinforcement learning (RL), controversially argues that large language models (LLMs) represent a "dead end" for achieving general intelligence. Sutton contends that LLMs, which rely on imitation learning from vast datasets of human text, lack the ability to learn continuously "from experie...

Demystifying Reinforcement Learning in Agentic Reasoning 19.10.2025

The research paper systematically investigates how reinforcement learning (RL) can enhance the agentic reasoning capabilities of Large Language Models (LLMs), particularly in tool-integrated environments. The authors conduct a comprehensive empirical study across three main dimensions: data curation, algorithmic design, and reasoning mode to demystify optimal practices for agentic RL. Key findings...

Emergent coordination in multi-agent language models 19.10.2025

This paper introduces an **information-theoretic framework** designed to determine when multi-agent Large Language Model (LLM) systems transition from simple aggregates to integrated, synergistic collectives. The research utilizes a **group guessing game without direct communication** to experimentally test how different prompt designs—specifically, a control condition, assigning agent **personas*...

Learning-to-measure: in-context active feature acquisition 19.10.2025

This paper introduces Learning-to-Measure (L2M) to address the challenges of meta-Active Feature Acquisition (meta-AFA), a sequential decision-making problem. Traditional AFA methods often struggle with scalability because they are designed for a single task and fail when trained on retrospective data containing systematic missingness in features. L2M overcomes these limitations by formalizing the...

Ascultă podcastul Best AI papers explained în Replaio

Radio și podcasturi într-o singură aplicație - gratuit și fără înregistrare. Instaleaz-o azi și nu rata lansarea

Descarcă din Google Play

Replaio nu este editorul podcasturilor; numele emisiunilor, coperțile și materialul audio aparțin autorilor lor și sunt distribuite prin fluxuri RSS publice