Enoch H. Kang
Best AI papers explained
Cut through the noise. We curate and break down the most important AI papers so you don’t have to.
Vizitează neapărat site-ul podcastului și susține-i creatorul: podcasters.spotify.com
Autor
Enoch H. Kang
Categorie
Site-ul podcastului
Cel mai nou episod
19 nov. 2026
Unde asculți?
Podcasturi în aplicație Replaio Radio În curândPodcasturile ajung în curând în aplicație. Instaleaz-o acum și fii primul care descoperă o abordare complet nouă a podcasturilor
Episoade
Inverse Reinforcement Learning Meets Large Language Model Post-Training: Basics, Advances, and Opportunities 22.07.2025 26:20
The source **comprehensively reviews** the **integration of Inverse Reinforcement Learning (IRL) with Large Language Model (LLM) post-training**, primarily focusing on **alignment challenges and opportunities**. It explains how LLM generation can be framed within a **Markov Decision Process (MDP) framework**, despite the inherent difficulty of defining explicit reward functions, and highlights the...
The Invisible Leash: Why RLVR May Not Escape Its Origin 20.07.2025 16:09
This paper explores the limitations of Reinforcement Learning with Verifiable Rewards (RLVR) in expanding the reasoning capabilities of large language models (LLMs). It argues that RLVR primarily functions as a conservative reweighting mechanism , enhancing the precision of existing solutions rather than discovering entirely new ones. The text introduces a theoretical perspective, validated empiri...
Language Model Personalization via Reward Factorization 20.07.2025 10:03
This paper discusses Personalization via Reward Factorization (PReF) , a novel framework designed to enhance Large Language Models (LLMs) by personalizing responses to individual user preferences. Unlike traditional Reinforcement Learning from Human Feedback (RLHF) which assumes universal preferences, PReF models user-specific rewards as a linear combination of "base reward functions" an...
Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions 18.07.2025 13:41
This academic paper explores masked diffusion models (MDMs) , a promising approach for generative modeling in discrete domains. It investigates the trade-off between training complexity and inference flexibility in MDMs compared to autoregressive models (ARMs) . The authors demonstrate that MDMs are trained on computationally challenging subproblems, leading to performance imbalances. However, the...
Do We Need to Verify Step by Step? Rethinking Process Supervision from a Theoretical Perspective 17.07.2025 12:56
This research examines two fundamental paradigms in reinforcement learning: process supervision and outcome supervision . Process supervision offers fine-grained, step-by-step reward feedback , while outcome supervision provides only a cumulative reward at the end of a task . The paper challenges the conventional belief that outcome supervision is inherently more difficult, demonstrating that, und...
Soft Best-of-n Sampling for Model Alignment 16.07.2025 14:27
This paper introduces Soft Best-of-n (BoN) sampling , an advancement over traditional BoN sampling for aligning large language model (LLM) outputs with human preferences . While standard BoN samples multiple responses and picks the highest-reward one, Soft BoN incorporates a temperature parameter (λ) , enabling a smoother trade-off between maximizing reward and maintaining similarity to the origin...
On Temporal Credit Assignment and Data-Efficient Reinforcement Learning 15.07.2025 16:56
This paper introduces a novel performance measure for evaluating Reinforcement Learning (RL) algorithms, specifically addressing the temporal credit assignment problem . The authors argue that existing measures for generalization and exploration do not adequately capture an algorithm's ability to attribute outcomes to past actions and states . They propose "misallocation" (MALLOC) ,...
Bradley–Terry and Multi-Objective Reward Modeling Are Complementary 15.07.2025 16:39
This research introduces SMORM, a novel framework designed to enhance reward models for Large Language Models (LLMs) by addressing the persistent issue of "reward hacking," particularly in out-of-distribution (OOD) settings . The paper highlights that current state-of-the-art methods struggle when training and testing data distributions differ . SMORM uniquely combines Bradley-Terry sing...
Probing Foundation Models for World Models 15.07.2025 11:47
This paper investigates whether foundation models truly acquire a deeper understanding of underlying "world models" beyond mere accurate sequence prediction. Researchers introduce an "inductive bias probe" to evaluate how these models adapt to new tasks based on postulated world models, such as Newtonian mechanics for orbital trajectories or game rules for Othello. The findings...
GenAI-Powered Statistical Inference (with Unstructured Data) 14.07.2025 20:10
This paper introduces GenAI-Powered Inference (GPI) , a novel statistical framework for both causal and predictive analysis of unstructured data , such as images and text. GPI utilizes open-source Generative AI models to extract low-dimensional representations from high-dimensional unstructured data, which are then used in conjunction with machine learning techniques to quantify causal and predict...
Interpretable Reward Modeling with Active Concept Bottlenecks 14.07.2025 11:38
This academic paper introduces Concept Bottleneck Reward Models (CB-RM) , a novel framework designed to enhance the interpretability of reward functions used in Reinforcement Learning from Human Feedback (RLHF) . Unlike traditional opaque models, CB-RM decomposes reward prediction into human-understandable concepts, such as helpfulness or correctness. To address the high cost of data annotation, t...
PrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applications 14.07.2025 13:54
The research introduces PrefillOnly , a novel inference engine specifically designed for Large Language Models (LLMs) used in discriminative tasks, where only a single output token is generated. Unlike traditional LLM engines optimized for variable-length outputs, PrefillOnly significantly reduces GPU memory consumption by only storing the Key-Value (KV) cache of the last computed layer and by usi...
A Collectivist, Economic Perspective on AI 14.07.2025 21:19
We discuss the paper "A Collectivist, Economic Perspective on AI," which critiques the prevailing individualistic and cognitive focus in artificial intelligence development. It argues for a collectivist, economic, and inferential approach to designing AI systems, emphasizing that human intelligence is inherently social and that technology's societal impact should be a primary concern...
Textual Bayes: Quantifying Uncertainty in LLM-Based Systems 12.07.2025 9:03
This paper titled "Textual Bayes: Quantifying Uncertainty in LLM-Based Systems," available on arXiv. This paper addresses the critical challenge of quantifying uncertainty in large language model (LLM)-based systems , which is crucial for their application in high-stakes environments. The authors propose a novel Bayesian approach where prompts are treated as textual parameters within...
The Winner's Curse in Data-Driven Decisions 11.07.2025 30:06
The document outlines how data-driven decision-making, particularly in marketing, is susceptible to the "winner's curse," a phenomenon where selected optimal policies are overvalued due to estimation errors. It explains that this upward bias occurs because algorithms tend to pick options that appear best in available data, even if their true performance is lower. The authors demonstr...
SPIRAL: Self-Play for Reasoning Through Zero-Sum Games 11.07.2025 17:19
This paper introduces SPIRAL , a novel self-play framework designed to enhance the reasoning capabilities of large language models (LLMs) without relying on human supervision or pre-curated datasets. By engaging in multi-turn, zero-sum games like TicTacToe, Kuhn Poker, and Simple Negotiation, LLMs learn to develop transferable cognitive patterns such as systematic decomposition, expected value cal...
Beyond Statistical Learning: Exact Learning Is Essential for General Intelligence 11.07.2025 21:42
This paper argues that Artificial General Intelligence (AGI) , particularly for tasks requiring deductive reasoning , demands a fundamental shift from statistical learning to exact learning . Current AI systems, based on statistical methods, excel on average but consistently fail on straightforward deductive tasks due to their inherent design, which optimizes for statistical performance over distr...
Aligning Learning and Endogenous Decision-Making 11.07.2025 16:25
This academic paper introduces a novel end-to-end framework for solving contextual stochastic optimization problems where decisions directly influence outcomes , unlike traditional approaches. The authors propose a robust optimization variant that accounts for machine learning model uncertainty by constructing uncertainty sets to optimize actions against worst-case predictions , provin...
Reliable Statistical Inference with Synthetic Data from Large Language Models 11.07.2025 14:12
This paper introduces a novel framework for conducting reliable statistical inference using synthetic data generated by large language models (LLMs) , particularly in social science research. The authors propose a Generalized Method of Moments (GMM) estimator that effectively integrates both real human-annotated data and LLM-generated synthetic samples . This method aims to improve stat...
Multi-Turn Reinforcement Learning from Human Preference Feedback 10.07.2025 17:06
This academic paper introduces Multi-turn Preference Optimization (MTPO) , a novel approach to Reinforcement Learning from Human Feedback (RLHF) for Large Language Models (LLMs) . Unlike existing RLHF methods that evaluate single conversational turns, MTPO focuses on multi-turn interactions , where feedback is provided for entire conversations to capture long-term goals and planning. The pa...
Provably Learning from Language Feedback 09.07.2025 17:12
This research introduces a formal framework called Learning from Language Feedback (LLF), where AI agents learn from natural language interactions instead of numerical rewards . The authors propose "transfer eluder dimension" to measure the complexity and efficiency of learning in LLF problems, demonstrating that rich language feedback can lead to exponentially faster learning than tradi...
Markets with Heterogeneous Agents: Dynamics and Survival of Bayesian vs. No-Regret Learners 05.07.2025 21:22
This paper examines the performance of Bayesian learners and no-regret learners in competitive asset markets, identifying conditions for their survival or vanishing. It contrasts the economic focus on Bayesian learning with the computer science emphasis on no-regret learning, highlighting that low regret doesn't always guarantee market survival against a perfect Bayesian, while Bayesi...
Why Neural Network Can Discover Symbolic Structures with Gradient-based Training: An Algebraic and Geometric Foundation 05.07.2025 13:30
This academic paper introduces a theoretical framework explaining how discrete symbolic structures can naturally emerge in neural networks through continuous gradient-based training . The authors model neural network optimization as a Wasserstein gradient flow in a measure space , demonstrating that under geometric constraints like group invariance , the network's parameters under...
Causal Abstraction with Lossy Representations 04.07.2025 25:36
This academic paper introduces projected abstractions , a novel framework designed to enhance causal inference in artificial intelligence systems by accommodating lossy representations . Traditional causal abstraction methods, which simplify complex "low-level" causal models into more manageable "high-level" ones, often fail when multiple low-level interventions map to the sa...
The Winner's Curse in Data-Driven Decisions 04.07.2025 23:00
This academic paper addresses the "winner's curse" in data-driven decision-making, a phenomenon where selecting optimal policies based on estimated effects leads to overly optimistic evaluations of actual policy value . The authors theoretically demonstrate the existence of this curse and empirically illustrate its presence across various marketing applications like A/B testin...
Podcasturi asemănătoare
Replaio nu este editorul podcasturilor; numele emisiunilor, coperțile și materialul audio aparțin autorilor lor și sunt distribuite prin fluxuri RSS publice