AI Post Transformers

This September 2025 paper analyzes the theoretical benefits and limitations of Masked Diffusion Models (MDMs) for text generation, contrasting them with auto-regressive models. While MDMs can sample multiple tokens in parallel, offering a potential for efficiency, the research demonstrates that their actual performance depends heavily on the evaluation metric. Specifically, MDMs can achieve near-o...

The September 9 2025 press release and paper announce and detail K2 Think, an advanced open-source AI reasoning system developed by the Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) and G42 in the UAE. K2 Think stands out for its parameter efficiency, achieving performance comparable to much larger models, particularly in mathematical reasoning, with only 32 billion parameters....

This February 2025 paper introduces INF2, a novel framework designed to enhance the generative inference throughput of large language models (LLMs) by utilizing computational storage devices (CSDs). The core innovation, attention-near storage (ANS), offloads memory-intensive self-attention operations directly to accelerators within these storage devices, significantly reducing data transfer bottle...

This September 2025 paper introduces Diversity-Aware Reinforcement Learning (Darling), a novel framework designed to enhance both the quality and semantic diversity of large language model (LLM) generations. Recognizing that traditional post-training methods often sacrifice diversity for accuracy, Darling integrates a learned partition function to measure semantic diversity beyond simple lexical v...

This July 2002 paper introduced BLEU (Bilingual Evaluation Understudy), an automatic and inexpensive method for evaluating machine translation (MT) quality. It highlights the limitations of human evaluation, such as its high cost and time consumption, and proposes BLEU as a quick, language-independent alternative that correlates strongly with human judgment. The core concept of BLEU involves measu...

The May - June 2025 sources introduce AlphaEvolve, a novel AI coding agent developed by Google DeepMind in collaboration with mathematicians like Javier Gómez Serrano and Terence Tao. This Gemini-powered tool utilizes an evolutionary process, similar to natural selection, to generate and iteratively refine code solutions for complex problems. AlphaEvolve has demonstrated its capability in scientif...

This September 2025 paper introduces TraceRL, a novel reinforcement learning framework designed to enhance diffusion language models (DLMs) across various architectural types. The core idea behind TraceRL is to align the training process with the preferred inference trajectories of the model, which demonstrably improves performance on complex reasoning tasks like mathematics and coding. The author...

This September 2025 paper investigates the reliability and robustness of Large Language Models (LLMs) when evaluated using traditional benchmarks. The authors systematically paraphrased questions across six common benchmarks and observed how 34 different LLMs performed. Their findings indicate that while LLM rankings remain relatively consistent, their absolute effectiveness scores significantly d...

This September 2025 paper introduces "Behavioral Fingerprinting," a novel framework designed to evaluate Large Language Models (LLMs) beyond traditional performance scores like MMLU. It aims to understand how models "think," creating a multi-faceted profile of their intrinsic cognitive and interactive styles. The methodology employs a diagnostic prompt suite and an automated ev...

This March 2025 paper examines the input/output (I/O) characteristics of offloading large language model (LLM) components to NVMe SSDs during inference, a critical solution for overcoming GPU memory limitations with ever-growing LLMs. Researchers analyzed block-layer I/O traces from two prominent LLM frameworks, DeepSpeed and FlexGen, to understand how model weights and key-value (KV) caches are h...

This June 2024 paper introduces SGLang, a framework designed to enhance the efficiency of Large Language Model (LLM) and Vision Language Model (VLM) serving. It achieves this through a co-design of a flexible frontend language and a fast backend runtime. The frontend simplifies programming with primitives for generation and parallelism, while the backend utilizes novel optimizations like RadixAtte...

The provided May 2024 sources center around CoreNet, an Apple-developed library for training deep neural networks, and OpenELM, an efficient language model family built using CoreNet. CoreNet is a versatile toolkit supporting various tasks, including foundation models like large language models (LLMs), object classification, and semantic segmentation, with its development evolving from the earlier...

Thus describes EleutherAI's GPT-NeoX library, a robust open-source framework for training large-scale autoregressive language models on GPUs, building upon the Megatron and DeepSpeed libraries. It highlights the library's advanced features like distributed training, support for various hardware and systems, and cutting-edge architectural innovations. The text also provides practical guid...

These September 2025 posts describe HuggingFaceM4/FineVision, a large dataset designed for image and text modalities. It features a substantial size, ranging from 10M to 100M, and is available in the parquet format. This dataset includes various ratings, such as relevance, visual dependency, image correspondence, and formatting, indicating its use in evaluating the quality and relationship between...

This July 2021 paper documents the development and evaluation of OpenAI's Codex models, which are large language models specialized in code generation, particularly Python functions from docstrings. They introduce HumanEval, a hand-written dataset designed to assess the functional correctness of generated code through unit tests, a more robust metric than traditional match-based scores like B...

These sources collectively explore various approaches to evaluating and improving Large Language Models (LLMs). Several papers introduce new benchmark datasets designed to test LLMs on complex reasoning tasks, such as the "BIG-Bench Hard (BBH)" suite, the graduate-level "GPQA" questions in science, and "MuSR" for multistep soft reasoning in natural language narratives...

This blog post series from Chris Lattner extensively examines CUDA's pervasive dominance in AI compute, detailing its evolution from a graphics processor to a layered software platform integral to NVIDIA's success, while also highlighting the challenges and complexities it presents to developers and alternative hardware vendors. The articles critically assess various attempts to democrat...

This September 2025 paper describe SAIR, the Structurally Augmented IC50 Repository, a groundbreaking open-source dataset developed by SandboxAQ in collaboration with NVIDIA. SAIR is the largest publicly available collection of over 5 million AI-generated 3D protein-ligand structures, each linked with experimentally measured drug potency data (IC₅₀ values). This dataset aims to bridge a critical d...

This August 2025 paper from Google DeepMind, titled "On the Theoretical Limitations of Embedding-Based Retrieval," explores the fundamental constraints of vector embedding models in information retrieval. The authors demonstrate that the number of relevant document combinations an embedding can represent is inherently limited by its dimension. Through empirical "free embedding"...

These academic papers introduce and detail the Massive Multilingual Text Embedding Benchmark (MMTEB), a comprehensive evaluation framework for text embedding models. The MMTEB expands upon existing benchmarks by offering over 500 tasks across 250+ languages and various domains, significantly increasing the diversity and scale of evaluation. It incorporates optimizations like downsampling and cachi...

This September 2025 paper introduces Inverse IFEval, a novel benchmark designed to evaluate Large Language Models (LLMs) for their Counter-intuitive Ability. This refers to an LLM's capacity to override its ingrained training patterns and comply with instructions that conflict with conventional norms or standardized formats. The benchmark includes eight distinct categories of such challenging...

This document announces EmbeddingGemma, a new open embedding model from Google, specifically designed for on-device artificial intelligence (AI). It highlights the model's efficiency, compact size, and best-in-class performance for its category, particularly in multilingual text embedding. The source explains how EmbeddingGemma enables mobile-first Retrieval Augmented Generation (RAG) pipelin...

This September 2025 paper introduces DeepResearch Arena, a novel benchmark designed to evaluate the research capabilities of large language models (LLMs) by mirroring real-world academic inquiry. This benchmark addresses limitations of existing evaluation methods, which often suffer from data leakage or lack authenticity, by grounding its tasks in academic seminars and expert discourse. A Multi-Ag...

This June 2024 paper examines the current state and future potential of Physical Neural Networks (PNNs), which are AI systems implemented directly in physical hardware rather than purely digital software. It explores various training methodologies for PNNs, including in-silico (digital simulation), in-situ (real-world hardware training), and hybrid approaches like physics-aware training, each with...

This September 2025 paper article from Nature, authored by Kevin M. Cherry and Lulu Qian, introduces a novel DNA-based neural network capable of supervised learning in vitro. The authors demonstrate how DNA molecules can be programmed to autonomously classify patterns from molecular examples. This system integrates training data directly into molecular memories and uses these memories for subseque...