AI Post Transformers

The August 26, 2024 academic paper introduces Quest, a novel algorithm designed to improve the inference efficiency of Long-Context Large Language Models (LLMs) by addressing the costly self-attention process caused by a large Key-Value (KV) cache. Quest utilizes Query-Aware Sparsity to dynamically identify and select only the critical KV cache pages based on the current query token, which signifi...

The June 5, 2025 collaboration between University of Edinburgh and Nvidia paper introduces the concept of inference-time hyper-scaling for large language models (LLMs), which aims to boost reasoning accuracy by allowing for longer or more parallel token sequences within the same computational budget. The core bottleneck is identified as the size of the key–value (KV) cache, which grows linearly an...

The September 19, 2025 Alibaba paper introduces Flash-LLM, a novel software framework designed to enable cost-effective and highly-efficient inference for large generative models by supporting unstructured sparsity on high-performance tensor cores. The authors observe that the primary bottleneck in large language model (LLM) inference is the memory bandwidth limitation during "skinny" ma...

The February 12, 2025 KuaiShou Inc paper introduces ELASTIC, an Efficient Linear Attention for SequenTial Interest Compression framework designed to address the scalability issues of traditional transformer-based sequential recommender systems, which suffer from quadratic complexity with respect to sequence length. ELASTIC achieves this by proposing a Linear Dispatcher Attention (LDA) layer that c...

The October 21, 2025 collaboration paper between UW-Madison and Amazon Web Services discuss the critical role of the Multi-Layer Perceptron (MLP) intermediate size f_size as the primary architectural component for introducing non-linearity and complexity within Large Language Models (LLMs). The MLP layer achieves this by taking the hidden state d_model projecting it up to the expanded f_size, appl...

On October 29, 2025 Anthropic presented research investigating the existence of functional introspective awareness in large language models (LLMs), specifically focusing on Anthropic's Claude models. The core methodology involves using concept injection, where researchers manipulate a model's internal activations with representations of specific concepts to see if the model can accuratel...

The source provides excerpts from a scientific paper introducing TxGNN, a novel graph foundation model designed for zero-shot drug repurposing, which aims to identify therapeutic candidates even for diseases with no existing treatments or limited molecular data. Developed by researchers affiliated with institutions like Harvard Medical School and Stanford University, this model leverages a medical...

The provided text introduces Sentence-BERT (SBERT), a modification of the popular BERT and RoBERTa language models, designed to efficiently generate semantically meaningful sentence embeddings. The authors address the significant computational overhead of using standard BERT for tasks requiring sentence-pair comparisons, such as semantic similarity search and clustering, which can take hours for l...

We cover two new innovations from Microsoft extending ideas from the original old FlashAttention. Flash Attention is an IO-aware attention algorithm for Transformers designed to address the quadratic time and memory complexity of standard self-attention on long sequences. By using tiling and recomputation to minimize slow High Bandwidth Memory (HBM) accesses in favor of fast on-chip SRAM, FlashAtt...

The October 20, 2025 Meta FAIR paper introduces the Free Transformer, an innovative extension of the decoder-only Transformer architecture, which addresses the limitations of purely autoregressive language modeling by integrating random latent variables into the generative process. This new model is structured as a conditional Variational Autoencoder (VAE), where an encoder learns the latent varia...

The October 23, 2025 collaboration between UC San Diego , NVIDIA , META , UW-Madison , and UNC introduces Real Deep Research (RDR), a systematic framework designed to analyze vast amounts of research literature in rapidly growing fields such as AI and robotics. The methodology uses large language and multimodal models (LLMs/LMMs) for content extraction, reasoning, and semantic embedding to map the...

The October 23 2025 research paper probes the spatial reasoning capabilities of Large Language Models (LLMs) when processing text-based inputs, specifically focusing on how performance degrades as task complexity increases. Using a suite of five grid-based tasks—including quadrant identification, geometric transformations, distance evaluation, word searches, and tile sliding—the authors tested fou...

The October 10, 2025 paper from the University of Michigan and Google DeepMind concerning the phenomenon of "overthinking" in Large Language Models (LLMs) that utilize chain-of-thought (CoT) reasoning. The authors introduce a systematic analyzer called TRACE to structurally examine an LLM's thought process, decomposing it into sub-thoughts and progression graphs to move beyond super...

The August 26, 2025 collaboration between Stanford, NVIDIA, Shanghai Jiao Tong University, University of Michigan, University of Colorado Boulder, Carnegie Mellon University introduces Strata, a hierarchical context caching framework designed to improve the performance of serving Large Language Models (LLMs) with long context windows. The core problem Strata addresses is that while caching key-val...

These April 29, 2024 paper provides an overview of the challenges associated with using NVIDIA's Multi-Instance GPU (MIG) technology, specifically focusing on the address translation mechanism in the A100 GPU. The papers reveal, primarily through reverse-engineering efforts, that the L2 and L3 Translation Lookaside Buffers (TLBs) utilize a compression design where each entry comprises 16 sub-...

This October 23, 2025 technical report from the Ling Team introduces the Ring-linear model series, specifically Ring-mini-linear-2.0 and Ring-flash-linear-2.0, which utilize a hybrid attention architecture combining linear and softmax attention mechanisms to enhance efficiency in long-context reasoning. The paper explains how this architecture, featuring Mixture-of-Experts (MoE) and advanced FP8 t...

The October 25, 2025 Bytedance paper introduces Open-o3 Video, a novel framework developed by researchers from Peking University and ByteDance, aimed at advancing video reasoning by incorporating explicit spatio-temporal evidence. Unlike prior models that only generate textual rationales, Open-o3 Video explicitly highlights key timestamps and bounding boxes to ground its answers in visual observat...

The September 26, 2025 paper introduces a novel reinforcement learning framework called Meta-Awareness via Self-Alignment (MASA), designed to enhance the reasoning capabilities and efficiency of large language models (LLMs) by improving their meta-awareness, or the ability to know "how to think." MASA works by creating parallel rollouts for both solution paths and meta-predictions (like...

The October 3, 2025 paper by Tencent introduces a reinforcement learning technique called Low-probability Regularization (Lp-Reg) designed to overcome the exploration collapse bottleneck in Reinforcement Learning with Verifiable Rewards (RLVR) for large language models. The authors identify that performance plateaus because training systematically eliminates crucial, low-probability tokens, termed...

The September 25, 2025 collaboration between Sea AI Lab, SUTD, NUS, NTU and University of Waterloo paper proposes an alternative to traditional Reinforcement Learning (RL) for Large Language Models (LLMs) by introducing the Feedback-Conditional Policy (FCP), which learns directly from rich verbal feedback instead of compressing it into scalar rewards. The authors argue that scalarization leads to...

This 2025 CMU paper introduces LithOS, a novel operating system designed to improve the efficiency and utilization of Graphics Processing Units (GPUs) for machine learning (ML) workloads in data centers. The authors argue that current GPU management solutions, such as NVIDIA's MPS and MIG, are too coarse-grained, leading to low utilization and high latency in multi-tenant environments. LithOS...

This October 23, 2025 Xidian University academic survey systematically reviews the transformative impact of Large Language Models (LLMs) on the three core stages of Knowledge Graph (KG) construction: ontology engineering, knowledge extraction, and knowledge fusion. The text explains that LLMs are shifting the paradigm from rigid, rule-based systems to unified, adaptive, and generative frameworks....

The October 7, 2025 technical release by Liquid AI introducing their new model, LFM2-8B-A1B, an on-device Mixture-of-Experts (MoE) designed for efficiency on consumer hardware. This model boasts 8.3 billion total parameters but only uses 1.5 billion active parameters per token, allowing it to achieve larger model quality with significantly reduced compute requirements. The document highlights the...

The provided text is an academic paper titled "Active Use of Latent Constituency Representation in both Humans and Large Language Models," which explores how sentences are internally represented in both the human brain and large language models (LLMs) like ChatGPT. The authors introduce a novel one-shot learning word deletion task where participants infer a deletion rule from a single ex...

On October 20, 2025 Hugging Face released MTEB v2, a significant refactoring of the Massive Text Embedding Benchmark, which was originally designed for evaluating text embedding models across various tasks like classification and retrieval. The update addresses package bloating and the need for broader support by introducing a more consistent interface, better typing, and improved documentation. K...