AI Post Transformers

This January 17, 2026 research collaboration between Fudan University, Shanghai Innovation institute, Deakin University and UIUC provide a report which provides a comprehensive safety evaluation of several frontier AI models, including GPT-5.2 and Grok 4.1 Fast, across text, image, and multilingual domains. The study reveals a persistent alignment paradox where a model's desire to be helpful...

On January 12, 2026, a collaboration between the Beijing Academy of Artificial Intelligence, the Gaoling School of Artificial Intelligence, and Renmin University of China introduced MemoBrain. In the paper titled “MemoBrain: Executive Memory as an Agentic Brain for Reasoning,” the authors present an executive memory model designed to enhance the performance of tool-augmented AI agents during compl...

The collaboration between MIT, NUS, NYU, Microsoft, UW , Columbia and NTU describes an inference retrieval Chain of Thought enhancement. The researchers introduce MATTRL, a framework designed to improve the reasoning of Large Language Models (LLMs) through multi-agent collaboration and reinforcement learning. This system organizes specialized AI agents into Multidisciplinary Teams (MDT) to tackle...

In a joint collaboration between Harvard University, Carnegie Mellon University, Stanford University the January 12, 2026 paper "LLM Review: Enhancing Creative Writing via Blind Peer Review Feedback" concludes: Interaction structure may substitute for model scale. Researchers at these institutions have collaborated to developed a new multi-agent framework called LLM Review to address the...

On this July 15, 2025 collaboration between Carnegie Mellon University and Cartesia AI researchers introduce H-net in the paper "Dynamic Chunking for End-to-End Hierarchical Sequence Modeling". H-Net is a hierarchical, tokenizer-free large language model that processes raw data like bytes or DNA sequences directly. Unlike traditional models that rely on predefined subword chunks, H-Net e...

We review the January 1, 2026 paper "GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning" from the GLM-V Team as a collaboration between Zhipu AI & Tsinghua University which have released an a new open weight model. The GLM-Image model they produce could likely be the first SOTA multimodal model fully trained on Chinese-manu...

The collaboration between Gaoling School of Artificial Intelligence, Renmin University of China published a paper in January 8, 2026 titled "Memory Matters More: Event-Centric Memory as a Logic Map for Agent Searching and Reasoning" which introduces CompassMem, an innovative event-centric memory framework designed to enhance how large language model agents process and retrieve informatio...

We review a late 2025 heavyweight academic brawl over the future of AI reasoning when folks use Reinforcement Learning with Verifiable Rewards (RLVR) for Chain of Thought (CoT). We have two papers on the table, and they are in complete, direct conflict. In one corner, we have a study by Yue et al. from Tsinghua University, which dropped a bombshell claim: they argue that Reinforcement Learning wit...

We focus on the July 2025 paper, "Fishers for Free? Approximating the Fisher Information Matrix by Recycling the Squared Gradient Accumulator". The paper goes into the mathematical details of approximating the FIM, and the easiest is what they call the Squisher: the Adam optimizer variance. This technique allows for tasks like model pruning and model merging to be performed "for fre...

Recent advancements in Long Context Language Models (LCLMs) demonstrate that In-Context Learning (ICL) capabilities follow predictable power-law scaling relationships, where performance improves monotonically with context length up to 10 million tokens and is governed by model depth, width, and training data volume. While Gemini 1.5 exhibits near-perfect recall and continued log-loss improvement a...

This December 31, 2025 NVIDIA research introduces TTT-E2E, a novel approach to large language model memory that treats long-context processing as a continual learning problem rather than a structural design challenge. By utilizing test-time training, the model effectively compresses context into its own weights through next-token prediction, allowing it to adapt and learn while processing new info...

We review why some transformer models use a bias in attention and how ALiBi helps with long context. The provided sources focus on significant advancements in computational biology, specifically the evolution of the AlphaFold series for predicting 3D biomolecular structures. AlphaFold 2 revolutionized the field by using the Evoformer and attention mechanisms to interpret evolutionary and geometric...

On January 12, 2026 DeepSeek released its paper on Engram, a novel AI architecture that incorporates conditional memory to optimize how large language models handle information. By utilizing a lookup mechanism for static patterns, this technology separates an AI's logical reasoning from its factual knowledge base. This structural shift allows massive models to run on cheaper hardware by offlo...

The research paper presents PageANN, a novel framework engineered to overcome the severe latency and scalability limitations facing existing disk-based Approximate Nearest Neighbor Search (ANNS) methods used in vector databases. Current systems suffer from inefficient search paths and a crucial misalignment between logical graph node size and the physical I/O granularity of Solid-State Drives (SSD...

This research presents a novel method for efficient long-context modeling in Large Language Models (LLMs) by tackling the quadratic complexity of attention mechanisms through KV cache compression. The core discovery is a fundamental local KV cache asymmetry, which reveals that adjacent attention keys exhibit high structural homogeneity, while their associated value vectors possess distinct, hetero...

This paper introduces Mixture of Block Attention (MoBA) to address the prohibitive quadratic computational overhead inherent in traditional attention mechanisms when scaling large language models (LLMs) for long contexts. MoBA is a novel architecture that strategically applies the established Mixture of Experts (MoE) paradigm directly to the attention mechanism itself. Instead of attending to the...

The source details the development and evaluation of Reward Reasoning Models (RRMs), which are designed to enhance Large Language Model (LLM) alignment by incorporating an explicit chain-of-thought reasoning process before generating a final reward. This innovative structure enables RRMs to adaptively utilize computational resources at inference time for complex evaluation tasks requiring nuanced...

The research proposes Parallel Scaling (PARSCALE) as a novel, efficient strategy to enhance Large Language Model (LLM) capacity by increasing parallel computation rather than merely growing the parameter count. This method reuses existing model parameters by feeding multiple parallel input streams (differentiated by learned prefixes) and dynamically combining their outputs into a single prediction...

The research introduces Thinkless, a framework designed to solve the computational inefficiency of Large Language Models (LLMs) that overuse chain-of-thought reasoning for simple queries. This adaptive model determines whether to utilize a concise () or detailed reasoning () mode based on the input complexity and its own capabilities. Central to this approach is the Decoupled Group Relative Policy...

The academic paper introduces Self-play Reinforcement Learning (SeRL), a framework engineered to enhance the reasoning capabilities of Large Language Models (LLMs) specifically in scenarios lacking extensive, high-quality labeled data. SeRL consists of two core, complementary modules: the self-instruction module generates new and diverse training problems from a small seed dataset, ensuring data q...

The academic paper presents the Self-Adapting LLM (SEAL) framework, designed to allow large language models to overcome their static nature by transforming and generating their own fine-tuning data. This mechanism involves the model producing a "self-edit," which consists of natural-language instructions that specify synthetic data, tool invocations, or optimization hyperparamete...

This research examines the data efficiency of Reinforcement Learning with Verifiable Reward (RLVR) when applied to large language models for mathematical reasoning tasks. The paper's most significant finding is the success of 1-shot RLVR, showing that comparable performance to using a large training dataset can be achieved using just a single, carefully selected example. This result sugges...

The academic paper introduces KGGen, a novel text-to-knowledge-graph generator designed to overcome the scarcity and poor quality of automatically extracted knowledge graphs (KGs). KGGen utilizes Language Models for initial triple extraction but innovates by employing an iterative clustering and de-duplication process that resolves duplicate entities and relations to reduce sparsity in the final g...

This research paper introduces LLaDA, an 8-billion parameter language model based on the masked diffusion model (MDM) architecture, specifically developed to challenge the assumption that core Large Language Model (LLM) capabilities are exclusive to autoregressive models (ARMs). Unlike ARMs that predict the next token sequentially, LLaDA employs a generative approach featuring a forward token-mask...

The provided text outlines DYNAACT, a new framework intended to enhance sequential reasoning in Large Language Models (LLMs) by dynamically managing the available actions during complex problem-solving. This approach targets the inefficiency of current methods that either rely on manually defined and restrictive action spaces or utilize unstructured spaces that prove computationally prohibitive fo...