Enoch H. Kang
Best AI papers explained
Cut through the noise. We curate and break down the most important AI papers so you don’t have to.
Be sure to visit the podcast's website and support the creator: podcasters.spotify.com
Author
Enoch H. Kang
Category
Podcast website
Latest episode
Nov 19, 2026
Where to listen?
Podcasts in the app Replaio Radio Coming soonPodcasts are coming to the app soon. Install now and be the first to see a whole new take on podcasts
Episodes
Trial-Error-Explain In-Context Learning for Personalized Text Generation 27.05.2025 12:20
This paper introduce Trial-Error-Explain In-Context Learning (TICL) , a method for personalizing large language models (LLMs) to match individual user writing styles without requiring model fine-tuning. TICL expands the in-context learning prompt by adding model-generated negative examples and explanations that highlight discrepancies from the target style. Evaluations demonstrate that TICL signif...
Reinforcement Learning for Reasoning in Large Language Models with One Training Example 27.05.2025 12:40
This research investigates how little training data is needed for Reinforcement Learning with Verifiable Reward ( RLVR ) to significantly boost the mathematical reasoning abilities of large language models (LLMs) . Surprisingly, the authors demonstrate that training on even just one carefully chosen example can achieve performance comparable to using datasets containing thousands, resulting in...
Test-Time Reinforcement Learning (TTRL) 27.05.2025 17:49
This paper introduces Test-Time Reinforcement Learning (TTRL) , a novel method enabling Large Language Models (LLMs) to improve performance on unlabeled test data using Reinforcement Learning (RL). TTRL overcomes the lack of ground-truth labels by employing majority voting on multiple model outputs to estimate rewards , essentially allowing models to self-supervise their training. The research d...
Interpreting Emergent Planning in Model-Free Reinforcement Learning 26.05.2025 14:56
This paper presents research exploring whether a model-free reinforcement learning agent, specifically a DRC agent playing the game Sokoban , learns to plan. Through a concept-based interpretability methodology involving probing for planning-relevant concepts like future agent and box movements, investigating how plans are formed internally, and verifying the causal link between internal represent...
Agentic Reward Modeling_Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systems 26.05.2025 13:04
This paper proposes a new reward system for large language models (LLMs) called agentic reward modeling, which aims to create more reliable rewards by integrating human preferences with verifiable correctness signals . An empirical implementation, named REWARDAGENT , is presented, which combines human preference rewards with signals related to factuality and instruction following . Extensive exper...
Beyond Reward Hacking: Causal Rewards for Large LanguageModel Alignment 26.05.2025 12:34
This research introduces a novel method for aligning large language models (LLMs) with human preferences while avoiding common pitfalls like reward hacking and spurious correlations . The authors propose a causal reward modeling approach that integrates causal inference and counterfactual invariance to ensure that reward predictions are based on true relationships rather than irrelevant data patte...
Learning How Hard to Think: Input-Adaptive Allocation of LM Computation 26.05.2025 19:31
This paper introduces an approach to optimize the computational resources used by language models (LMs) when responding to different queries. Instead of applying the same level of processing to every request, the method learns to predict how much a query would benefit from more intensive computation and then allocates resources adaptively . This is achieved by training a model to estimate the pote...
Highlighting What Matters: Promptable Embeddings for Attribute-Focused Image Retrieval 26.05.2025 15:39
This paper introduces COCO-FACET , a new benchmark dataset designed to evaluate text-to-image retrieval models on attribute-focused queries , which differ from traditional general image caption queries. The researchers demonstrate that existing models, including CLIP-like and MLLM-based models, struggle with these specific attributes , especially those less prominent in images or less explored in...
UFT: Unifying Supervised and Reinforcement Fine-Tuning 26.05.2025 14:20
This paper introduces Unified Fine-Tuning (UFT) , a novel method for enhancing the reasoning capabilities of large language models (LLMs) by integrating supervised fine-tuning (SFT) and reinforcement fine-tuning (RFT) . The authors argue that traditional SFT and RFT have limitations, with SFT potentially overfitting and RFT being constrained by the base model's initial capacity. UFT addresses...
Understanding High-Dimensional Bayesian Optimization 26.05.2025 19:39
This paper investigates challenges in high-dimensional Bayesian optimization (HDBO) , particularly focusing on vanishing gradients that hinder the effectiveness of standard methods in high dimensions. It highlights how initialization schemes and acquisition function optimization contribute to these issues, and explains why recent methods succeed by promoting local search behaviors and mitigating v...
Inference time alignment in continuous space 25.05.2025 15:55
This academic paper introduces Simple Energy Adaptation (SEA) , a novel method for aligning large language models (LLMs) with human preferences during the inference phase. Unlike traditional methods that rely on discrete searches within a limited set of responses from the base model, SEA formulates alignment as an iterative optimization process in a continuous latent space. By applying gradient-ba...
Efficient Test-Time Scaling via Self-Calibration 25.05.2025 24:17
This academic paper explores methods to improve the efficiency and accuracy of Large Language Models (LLMs) during the final step of generating responses, known as test-time scaling. The authors propose Self-Calibration , a technique to teach LLMs to reliably estimate their own confidence in an answer with a single pass. By incorporating these calibrated confidence scores, they develop efficient t...
Conformal Prediction via Bayesian Quadrature 25.05.2025 22:58
This paper explores a novel perspective on conformal prediction , a method for providing performance guarantees for machine learning models without assuming a specific data distribution. The authors propose viewing conformal prediction through a Bayesian lens , specifically utilizing Bayesian quadrature , a technique for estimating integrals with uncertainty. They argue that this approach addresse...
Predicting from Strings: Language Model Embeddings for Bayesian Optimization 25.05.2025 27:18
This paper, a research paper from Google DeepMind, introduces a novel approach called Embed-then-Regress for Bayesian Optimization . This method leverages the ability of language models to embed string representations of various types of inputs, including synthetic, combinatorial, and hyperparameter configurations, into fixed-length vectors. These vectors then serve as features for a Transformer-b...
Self-Evolving Curriculum for LLM Reasoning 25.05.2025 14:55
This document presents Self-Evolving Curriculum (SEC) , a novel method for reinforcement learning (RL) fine-tuning of large language models (LLMs) to enhance their reasoning capabilities. SEC frames curriculum selection as a non-stationary Multi-Armed Bandit (MAB) problem , where problem categories represent individual "arms". It learns a curriculum policy concurrently with LLM training...
Online Decision-Focused Learning in Dynamic Environments 25.05.2025 20:48
This paper introduces online decision-focused learning (DFL) , a framework for training predictive models used in dynamic, sequential decision-making tasks where data distributions and objectives change over time. The authors propose a new algorithm, Decision-Focused Online Gradient Descent (DF-OGD) , which handles the non-differentiable and non-convex nature of the decision objective by regulariz...
FisherSFT: Data-Efficient Supervised Fine-Tuning of Language Models Using Information Gain 25.05.2025 14:11
This paper introduces FisherSFT , a method for making supervised fine-tuning (SFT) of large language models (LLMs) more data-efficient by selecting the most informative training examples . The key concept is to choose examples that maximize information gain , which is approximated by evaluating the Hessian of the LLM's log-likelihood . This approach uses a computationally efficient approximati...
Reward Shaping from Confounded Offline Data 25.05.2025 20:33
This academic paper explores a novel technique for automatic reward shaping in reinforcement learning , specifically addressing the challenge of learning from offline data that may contain unobserved confounding factors . The authors propose using causal state value upper bounds , derived from this confounded data, as potential functions for Potential-Based Reward Shaping (PBRS). They demonstrate...
Trajectory Bellman Residual Minimization: A Simple Value-Based Method for LLM Reasoning 25.05.2025 17:45
This academic paper introduces Trajectory Bellman Residual Minimization (TBRM) , a novel value-based reinforcement learning algorithm designed to enhance the reasoning capabilities of large language models (LLMs), particularly in mathematical problem-solving. Unlike prevailing policy-based methods like PPO and GRPO, TBRM streamlines the training process by eliminating the need for critics, importa...
Understanding Best-of-N Language Model Alignment 25.05.2025 14:13
This paper critically examines the best-of-n policy , a common method for aligning generative language models by selecting the highest-reward sample from $n$ options drawn from a reference policy . It disproves a widely-used analytical formula for the KL divergence between the best-of-n policy and the reference, proving that the formula is only an upper bound . The authors analyze the conditions u...
Maximizing Acquisition Functions for Bayesian Optimization - and its relation to Gradient Descent 24.05.2025 19:17
This academic paper explores methods to improve Bayesian optimization (BO) , a process for finding optimal settings for complex, costly functions. The authors address the challenge of maximizing acquisition functions , which are heuristics guiding BO's search and are often difficult to optimize, especially when evaluating multiple points simultaneously. They demonstrate that Monte Carlo integr...
Bayesian Prompt Ensembles: Model Uncertainty Estimation for Black-Box Large Language Models 24.05.2025 17:15
This paper introduces Bayesian Prompt Ensembles (BayesPE) , a novel method for quantifying uncertainty in black-box large language models (LLMs) without requiring access to their internal parameters or retraining. BayesPE achieves this by ensembling the outputs of an LLM prompted with various semantically equivalent instructions, learning the optimal weighting for each prompt through approximate B...
Prompting Strategies for Enabling Large Language Models to Infer Causation from Correlation 24.05.2025 12:05
This academic paper introduces PC-SUBQ , a prompting strategy designed to improve the ability of Large Language Models (LLMs) to infer causal relationships from correlations . The strategy breaks down the complex task into sequential sub-questions that mirror the steps of the PC algorithm, a formal causal discovery method . Evaluating PC-SUBQ on the CORR2CAUSE benchmark , which provides correlatio...
The Parallel Knowledge Gradient Method for Batch Bayesian Optimization 24.05.2025 14:57
This academic paper presents the parallel knowledge gradient method (q-KG) , a novel approach for batch Bayesian optimization designed to efficiently find the global optimum of costly, derivative-free functions when multiple evaluations can be performed concurrently. Unlike previous methods that build batches greedily, q-KG uses a decision-theoretic analysis to select a set of points that is Bayes...
FunBO: Discovering Acquisition Functions for Bayesian Optimization with FunSearch 24.05.2025 16:09
This paper introduces FunBO , a novel method utilizing Large Language Models (LLMs) to discover and refine acquisition functions (AFs) for Bayesian Optimization (BO) . The core idea is to treat the discovery of effective AFs as an algorithm discovery problem , leveraging FunSearch, an LLM-based approach for mathematical sciences. FunBO iteratively generates and evaluates candidate AFs written in c...
Similar podcasts
Replaio is not a podcast publisher; show names, artwork and audio belong to their authors and are distributed through public RSS feeds.