Neural intel Pod

This pod introduces  Confidence-Reward driven Preference Optimization (CRPO) , a novel method for  improving machine translation  by more effectively selecting training data for large language models (LLMs). The paper highlights challenges in applying LLMs to translation due to pretraining on English-centric data and the complexity of traditional reinforcement learning from human feedback. While ...

This academic paper introduces  MiCRo , a two-stage framework designed to improve how Large Language Models (LLMs) learn and adapt to  diverse human preferences , moving beyond the traditional assumption of a single universal preference.   Reward modeling , a key component in aligning LLMs with human feedback, typically uses a single model that struggles with varied preferences. The authors demons...

This document introduces  Prolonged Reinforcement Learning (ProRL) , a new training method designed to significantly  enhance the reasoning abilities of large language models . By implementing  KL divergence control  and  reference policy resetting , ProRL maintains training stability over extended periods, allowing models to discover novel reasoning strategies and  outperform base models  across...

This academic paper introduces  PROXYTHINKER , a novel inference-time method designed to enhance the visual reasoning abilities of  large vision-language models (LVLMs) . Unlike computationally expensive fine-tuning approaches like  reinforcement fine-tuning (RFT) , PROXYTHINKER allows larger models to inherit reasoning skills from smaller, pre-trained reasoning models. It achieves this by adjusti...

This academic paper presents  Open CaptchaWorld , a  novel benchmark dataset  designed to assess the ability of  multimodal AI agents  to solve  complex, multi-step CAPTCHAs  encountered in real-world online environments. Unlike existing benchmarks that focus on static, single-turn tasks, Open CaptchaWorld emphasizes the  interactive and dynamic nature  of modern human verification puzzles. Throug...

This document presents  DexMachina , a novel  curriculum-based reinforcement learning algorithm  for  functional retargeting  in  bimanual dexterous manipulation . The method focuses on teaching robot hands to  replicate human object manipulation trajectories  from demonstrations, particularly for  articulated objects and complex, long-horizon tasks . By employing  virtual object controllers  that...

This work introduces a  novel approach  and a  new benchmark  for advancing  embodied AI agents  operating in  3D environments . The proposed model,  3DLLM-MEM , is designed with a dual-memory system, combining a limited working memory with a long-term episodic memory using  dense 3D representations  to handle complex tasks requiring  spatial-temporal reasoning  and interaction with objects across...

This podcast offers a comprehensive overview of  fine-tuning large language models (LLMs) , exploring both foundational principles and advanced techniques. It details a  seven-stage pipeline  for fine-tuning, covering everything from initial  data preparation  and  model initialization  to  training setup ,  evaluation ,  deployment , and ongoing  monitoring and maintenance . The text also discuss...

This document presents RENT, a novel method for  improving the reasoning abilities of language models using unsupervised reinforcement learning.  Instead of relying on external feedback or ground-truth answers, RENT utilizes the  model's own confidence , specifically the negative entropy of its token distributions, as a reward signal. Experiments on various reasoning benchmarks and models demo...

This work examines the  critical points of random neural networks , particularly as network  depth increases in the infinite-width limit . The authors provide  asymptotic formulas for the expected number of critical points , categorized by their index or when exceeding a threshold. Their analysis reveals  three distinct scaling regimes for the expected number of critical points  based on a specifi...

This source introduces  BAGEL , a large multimodal model designed for  unified image understanding and generation . It discusses the model's  Mixture-of-Transformer-Experts (MoT) architecture , highlighting its  bottleneck-free design which enables better long-context interaction and scaling. The document details the  diverse training data , including text, image-text pairs, and interleaved vi...

This document introduces  R1-Searcher++ , a novel framework for  Large Language Models (LLMs)  designed to improve their ability to handle  factual questions  by strategically utilizing both their  internal knowledge  and  external search  capabilities. Unlike traditional methods that often over-rely on one source, R1-Searcher++ uses a two-stage training approach involving supervised fine-tuning f...

This source  evaluates and compares two reinforcement learning algorithms ,  GRPO and DPO , for their effectiveness in generating images from text descriptions. The research investigates how  different reward models , designed to assess image quality and human preferences, influence the performance and generalization capabilities of these algorithms. Additionally, the study examines the  impact of...

This document presents a research paper on a novel framework,  Let Androids Dream (LAD) , designed to enhance AI's ability to understand the implied meanings and metaphors in images, a significant challenge for current multimodal models. Inspired by human cognition, LAD employs a three-stage process:  Perception  to convert visuals to text,  Search to incorporate external knowledge, and  Reaso...

This source introduces  SmolVLM , a collection of small-scale multimodal models designed for efficiency on devices with limited computing power. The authors  experiment with different architectural choices, image processing techniques, and training data strategies  to create models that perform well on image and video tasks while using significantly less memory than larger models. They demonstrate...

This academic survey provides a  comprehensive overview of Federated Learning (FL) , a distributed machine learning approach allowing  collaborative model training without centralizing sensitive data . It details FL's  architecture and communication protocols , highlighting key stages like local training and model aggregation. The text emphasizes  technical challenges  such as handling diverse...

This document details research into improving  Federated Learning (FL)  efficiency in  autonomous mobile networks  by incorporating  tiny language models (TLMs)  for predicting network performance features. It focuses on the challenge of  communication overhead  in FL due to frequent neural network data exchanges. The paper proposes and evaluates the use of  NNCodec , an implementation of the  ISO...

This technical report introduces  Mobile-MMLU , a new benchmark designed to  evaluate large language models (LLMs) specifically for mobile devices , addressing the limitations of existing benchmarks which focus on desktop or server environments. Mobile-MMLU and its challenging subset, Mobile-MMLU-Pro, consist of  thousands of multiple-choice questions across 80 mobile-relevant domains , emphasizin...

This document presents  AI-RAN , a paradigm shift integrating  Radio Access Network (RAN)  and  Artificial Intelligence (AI)  workloads onto a unified platform. It outlines the  evolution of RAN  and categorizes AI-RAN into three forms:  AI-for-RAN, AI-on-RAN, and AI-and-RAN . The paper identifies key  requirements and enablers  for this convergence, including accelerated computing and cloud-nativ...

These sources collectively explain that  fine-tuning  is a process of retraining a pre-trained  Large Language Model  on a specialized dataset to enhance its performance on particular tasks or domains. While it can significantly improve a model's responses, it's not ideal for injecting entirely new factual knowledge. Several methods exist for  evaluating fine-tuned models , including gener...

This document describes the development of a  Large Language Model (LLM)  specifically tailored for explaining  VHDL code  within a high-performance processor design environment. Recognizing the unique requirements of such settings, including data security and leveraging existing design knowledge, the researchers employed  extended pretraining (EPT)  and  instruction tuning  on a base LLM using pr...

This document introduces and analyzes  AWNN (Adaptively Weighted Nearest Neighbors) , a novel matrix completion method. Traditional Nearest Neighbor (NN) methods struggle with selecting the appropriate number of neighbors and their weights, often relying on computationally expensive techniques like cross-validation. AWNN addresses this by formulating weight selection as a convex optimization probl...

This academic paper explores the  training dynamics of neural networks , specifically focusing on  gradient flow  for fully connected feedforward networks with various smooth activation functions. The authors establish a  dichotomy , showing that gradient flow either converges to a critical point or diverges to infinity while the loss approaches a generalized critical value. Utilizing the mathemat...

This academic paper introduces  WavReward , a novel evaluation system for  end-to-end spoken dialogue models , which process speech input and output directly, unlike older systems that rely on text. Recognizing the limitations of existing evaluations that primarily focus on text, WavReward leverages  audio language models  to assess both the content and acoustic aspects of spoken interactions, inc...

This academic paper introduces  BLIP3-o , a suite of cutting-edge multimodal models designed for both  understanding and generating images . The research investigates various architectural choices and training techniques, finding that  CLIP image features and flow matching are effective for image generation , while a  sequential training strategy —starting with understanding before generation—yiel...