Neural intel Pod

This work introduces a  novel approach  and a  new benchmark  for advancing  embodied AI agents  operating in  3D environments . The proposed model,  3DLLM-MEM , is designed with a dual-memory system, combining a limited working memory with a long-term episodic memory using  dense 3D representations  to handle complex tasks requiring  spatial-temporal reasoning  and interaction with objects across...

This podcast offers a comprehensive overview of  fine-tuning large language models (LLMs) , exploring both foundational principles and advanced techniques. It details a  seven-stage pipeline  for fine-tuning, covering everything from initial  data preparation  and  model initialization  to  training setup ,  evaluation ,  deployment , and ongoing  monitoring and maintenance . The text also discuss...

This document presents RENT, a novel method for  improving the reasoning abilities of language models using unsupervised reinforcement learning.  Instead of relying on external feedback or ground-truth answers, RENT utilizes the  model's own confidence , specifically the negative entropy of its token distributions, as a reward signal. Experiments on various reasoning benchmarks and models demo...

This work examines the  critical points of random neural networks , particularly as network  depth increases in the infinite-width limit . The authors provide  asymptotic formulas for the expected number of critical points , categorized by their index or when exceeding a threshold. Their analysis reveals  three distinct scaling regimes for the expected number of critical points  based on a specifi...

This source introduces  BAGEL , a large multimodal model designed for  unified image understanding and generation . It discusses the model's  Mixture-of-Transformer-Experts (MoT) architecture , highlighting its  bottleneck-free design which enables better long-context interaction and scaling. The document details the  diverse training data , including text, image-text pairs, and interleaved vi...

This document introduces  R1-Searcher++ , a novel framework for  Large Language Models (LLMs)  designed to improve their ability to handle  factual questions  by strategically utilizing both their  internal knowledge  and  external search  capabilities. Unlike traditional methods that often over-rely on one source, R1-Searcher++ uses a two-stage training approach involving supervised fine-tuning f...

This source  evaluates and compares two reinforcement learning algorithms ,  GRPO and DPO , for their effectiveness in generating images from text descriptions. The research investigates how  different reward models , designed to assess image quality and human preferences, influence the performance and generalization capabilities of these algorithms. Additionally, the study examines the  impact of...

This document presents a research paper on a novel framework,  Let Androids Dream (LAD) , designed to enhance AI's ability to understand the implied meanings and metaphors in images, a significant challenge for current multimodal models. Inspired by human cognition, LAD employs a three-stage process:  Perception  to convert visuals to text,  Search to incorporate external knowledge, and  Reaso...

This source introduces  SmolVLM , a collection of small-scale multimodal models designed for efficiency on devices with limited computing power. The authors  experiment with different architectural choices, image processing techniques, and training data strategies  to create models that perform well on image and video tasks while using significantly less memory than larger models. They demonstrate...

This academic survey provides a  comprehensive overview of Federated Learning (FL) , a distributed machine learning approach allowing  collaborative model training without centralizing sensitive data . It details FL's  architecture and communication protocols , highlighting key stages like local training and model aggregation. The text emphasizes  technical challenges  such as handling diverse...

This document details research into improving  Federated Learning (FL)  efficiency in  autonomous mobile networks  by incorporating  tiny language models (TLMs)  for predicting network performance features. It focuses on the challenge of  communication overhead  in FL due to frequent neural network data exchanges. The paper proposes and evaluates the use of  NNCodec , an implementation of the  ISO...

This technical report introduces  Mobile-MMLU , a new benchmark designed to  evaluate large language models (LLMs) specifically for mobile devices , addressing the limitations of existing benchmarks which focus on desktop or server environments. Mobile-MMLU and its challenging subset, Mobile-MMLU-Pro, consist of  thousands of multiple-choice questions across 80 mobile-relevant domains , emphasizin...

This document presents  AI-RAN , a paradigm shift integrating  Radio Access Network (RAN)  and  Artificial Intelligence (AI)  workloads onto a unified platform. It outlines the  evolution of RAN  and categorizes AI-RAN into three forms:  AI-for-RAN, AI-on-RAN, and AI-and-RAN . The paper identifies key  requirements and enablers  for this convergence, including accelerated computing and cloud-nativ...

These sources collectively explain that  fine-tuning  is a process of retraining a pre-trained  Large Language Model  on a specialized dataset to enhance its performance on particular tasks or domains. While it can significantly improve a model's responses, it's not ideal for injecting entirely new factual knowledge. Several methods exist for  evaluating fine-tuned models , including gener...

This document describes the development of a  Large Language Model (LLM)  specifically tailored for explaining  VHDL code  within a high-performance processor design environment. Recognizing the unique requirements of such settings, including data security and leveraging existing design knowledge, the researchers employed  extended pretraining (EPT)  and  instruction tuning  on a base LLM using pr...

This document introduces and analyzes  AWNN (Adaptively Weighted Nearest Neighbors) , a novel matrix completion method. Traditional Nearest Neighbor (NN) methods struggle with selecting the appropriate number of neighbors and their weights, often relying on computationally expensive techniques like cross-validation. AWNN addresses this by formulating weight selection as a convex optimization probl...

This academic paper explores the  training dynamics of neural networks , specifically focusing on  gradient flow  for fully connected feedforward networks with various smooth activation functions. The authors establish a  dichotomy , showing that gradient flow either converges to a critical point or diverges to infinity while the loss approaches a generalized critical value. Utilizing the mathemat...

This academic paper introduces  WavReward , a novel evaluation system for  end-to-end spoken dialogue models , which process speech input and output directly, unlike older systems that rely on text. Recognizing the limitations of existing evaluations that primarily focus on text, WavReward leverages  audio language models  to assess both the content and acoustic aspects of spoken interactions, inc...

This academic paper introduces  BLIP3-o , a suite of cutting-edge multimodal models designed for both  understanding and generating images . The research investigates various architectural choices and training techniques, finding that  CLIP image features and flow matching are effective for image generation , while a  sequential training strategy —starting with understanding before generation—yiel...

This document introduces  CodePDE , a new framework for using large language models (LLMs) to generate  code that solves partial differential equations (PDEs) . The authors frame PDE solving as a code generation problem and demonstrate that, with techniques like  debugging and refinement , LLMs can create solvers that are competitive with, and sometimes surpass,  human-written solvers  on various...

This research investigates  online learning for feedforward neural networks  utilizing the sign activation function. The paper identifies a  margin condition  in the first hidden layer as crucial for learnability, demonstrating that the optimal error bound is closely tied to the  totally-separable packing number  of the input space, showing an exponential dependence on dimension in some cases. To...

This document introduces  CityAVOS , a  new benchmark dataset  designed for  Aerial Visual Object Search (AVOS) tasks using  Unmanned Aerial Vehicles (UAVs)  in realistic urban environments. The text describes the unique challenges of urban AVOS, such as complex semantics and difficult identification of similar objects, which differentiate it from previous navigation and object search tasks. It al...

This document introduces  BAT (Benchmark for Auto-bidding Task) , a new resource for researching  autobidding algorithms  in online advertising auctions. It provides a large-scale dataset from the Avito platform, including information on two common auction types,  VCG and FP , along with  campaign details, auction outcomes, and traffic patterns . The paper also presents several  baseline autobiddi...

This collection of texts from Amazon Science highlights the company's extensive research and development efforts across various scientific and technical domains, including  machine learning ,  artificial intelligence , and  robotics . A significant portion focuses on improving the training of large language models, particularly through a novel method called  SeRA which aims to reduce spurious...

This document introduces  T2I-R1 , a novel text-to-image generation model that uses  Reinforcement Learning (RL)  and a  bi-level Chain-of-Thought (CoT)  process to improve image generation. Unlike traditional methods, T2I-R1 leverages  semantic-level CoT  for high-level planning based on the text prompt and  token-level CoT  for detailed, patch-by-patch image generation. A key component is  BiCoT...