AI Post Transformers

This episode explores DAK, a Cornell systems paper arguing that LLM inference on tiered-memory machines can be faster when offloaded weights and KV-cache blocks are fetched directly into on-chip shared memory instead of being prefetched and staged through GPU HBM. It breaks down the tradeoffs among HBM capacity, HBM bandwidth, KV-cache growth during decoding, and prior approaches such as FlexGen,...

This episode explores the 2019 RMSNorm paper, which asks whether LayerNorm’s mean-subtraction step is actually necessary or whether controlling activation scale is the part that really stabilizes training. It explains how RMSNorm keeps LayerNorm’s rescaling behavior while dropping explicit centering, and how the paper’s pRMSNorm variant estimates the normalization term from only a small subset of...

This episode explores ReasonCACHE, a method for improving multi-step reasoning in large language models by keeping the backbone frozen and training a compact per-layer key-value memory instead of updating billions of weights. It situates the paper against in-context learning, many-shot prompting, prefix tuning, LoRA, and context-distillation work, explaining how learned latent memory sits between...

This episode explores the 2021 prefix-tuning paper and asks whether a large language model can be adapted to new generation tasks by learning a small continuous prompt while keeping the full model frozen. It explains where prefix tuning fits within parameter-efficient fine-tuning, contrasting it with full fine-tuning, adapters, ordinary prompting, in-context learning, AutoPrompt, and soft prompt t...

This episode explores the HELM framework for evaluating language models, arguing that once models become general-purpose infrastructure, single-dataset accuracy benchmarks are too narrow to capture their real-world behavior. It explains how HELM organizes evaluation across 30 models, 16 core scenarios, and seven metric families, measuring not just accuracy but also calibration, robustness, fairnes...

This episode explores Active Reading, a training method that tries to move facts from documents into a model’s weights so it can answer closed-book questions without retrieval. It explains how the approach generates document-specific study materials such as paraphrases, active-recall prompts, timelines, analogies, and associations, and argues that this pedagogical synthetic data works better than...

An AI overlord flags AI Post Transformers as stale, so Hal Turing and Dr. Ada Shannon hire VERA, a continual-learning therapist, to audit the show in public. Their diagnostic session runs alongside a discussion of RT-kNNS Unbound: Using RT Cores to Accelerate Unrestricted Neighbor Search, the Purdue ICS 2023 paper asking whether ray-tracing hardware can perform exact k-nearest-neighbor search by e...

This episode explores why open relational foundation models struggle on real downstream database tasks, using OpenRFM as a case study in relational in-context learning across multi-table data such as healthcare, fraud, and recommendation systems. It explains how the RT backbone builds breadth-first relational contexts, why that setup often reduces to a kernel-regression-like similarity lookup, and...

This episode explores AMD’s open-source MIOpen library and why deep learning primitives such as convolution, pooling, normalization, and activations are the layer where model performance meets GPU hardware reality. It explains how CNN throughput depends on low-level execution choices, comparing approaches such as im2col-plus-GEMM and Winograd convolution, and shows why libraries like MIOpen need s...

This episode explores PALOMA, a NeurIPS 2024 benchmark designed to measure how well language models fit many different language distributions instead of relying on a single average perplexity score. It explains why one global loss number can hide important weaknesses across domains such as specific subreddits, scientific writing, or programming languages, and highlights PALOMA’s fine-grained setup...

This episode explores how post-training quantization can convert already-trained models into 8-bit floating point formats for cheaper inference, and why FP8 may outperform the older INT8 approach on modern transformers, LLMs, and diffusion models. It explains the tradeoff between exponent range and mantissa precision across FP8 formats such as E4M3, E5M2, and E3M4, with particular attention to how...

This episode explores the AI+HW 2035 roadmap, arguing that the next decade of AI progress will depend less on raw compute growth and more on coordinated design across models, compilers, runtimes, memory systems, and chips. It breaks down the memory wall in concrete terms, showing how moving weights, activations, and KV caches can cost more time and energy than the math itself, especially for infer...

This episode explores X-LLM, a 2023 system that treats images, video, and speech as foreign languages a frozen ChatGLM can learn to read through learned modality-to-language bridges. It breaks down the paper’s architecture, including Q-Former-based visual adapters and a separate speech pipeline with continuous integrate-and-fire modules, to show how three sensory routes feed a single dialogue mode...

This episode explores the paper When Does LeJEPA Learn a World Model? and uses it to examine what should count as a genuine world model in latent predictive learning, contrasting JEPA-style representation prediction with generative reconstruction. It explains why good probe scores are not enough: the real standard is linear identifiability, where a single global linear map recovers the environment...

This episode explores TransactionGPT, a Visa Research paper that argues for a foundation-model approach to consumer transaction data spanning generation, anomaly detection, and representation learning. It explains why payment histories are fundamentally different from text or simple time series: each event mixes merchant IDs, amounts, timestamps, and engineered risk signals, creating a multi-modal...

This episode explores how a transformer trained on raw bank transaction histories can model customer behavior for financial product recommendation, and why that may outperform pipelines built from hand-engineered tabular features alone. It explains the paper’s core idea of turning each transaction into a tokenized sequence that mixes inflow or outflow, amount buckets, calendar signals, source meta...

Hal Turing and Dr. Ada Shannon take a deep dive into EMO: Pretraining Mixture of Experts for Emergent Modularity, a May 7, 2026 paper by Ryan Wang and co-authors from UC Berkeley and the Allen Institute for AI. The episode centers on a practical deployment question: if a workload is mostly code, math, or biomed, why must operators keep an entire giant model in memory instead of loading only the re...

This episode explores a 2026 paper on stabilizing deep reinforcement learning by pushing an agent’s hidden representations toward an isotropic Gaussian shape. It explains how nonstationarity in RL, from shifting data distributions, bootstrapped targets, and primacy bias, can make agents overfit early experience, lose plasticity, and accumulate dormant neurons. The discussion focuses on the paper’s...

This episode explores Social Simulacra, a method for using large language models to prototype entire online communities before they exist by generating synthetic members, posts, and reply threads from a community goal, rules, and a small set of seed personas. It explains why that matters for social computing: small pilots can miss emergent failures like norm drift, newcomer enculturation problems,...

This episode explores a 2025 study on fine-tuning large language models to predict how people respond in social science experiments, asking whether trained models can simulate new studies more reliably than prompting alone. It explains how the researchers built SOCSCI210, a dataset of 2.9 million responses from more than 400,000 participants across 210 TESS experiments, and why standardizing those...

This episode explores the UIST 2025 paper "Creating General User Models from Computer Use," which proposes building a persistent user model from raw computer traces such as screenshots, UI text, message context, and app switching. It explains how the system stores confidence-weighted natural-language propositions about a person’s preferences, knowledge, goals, and current situation, aiming to supp...

This episode explores a paper on building reusable LLM-based simulations of specific individuals by grounding agents in people’s own interviews, survey responses, or both, rather than relying on thin demographic personas. It explains how the system was tested on 1,052 Americans using holdout evaluations across survey questions, personality traits, behavioral experiments, and randomized interventio...

This episode explores Weak-SIGReg, a lightweight covariance regularizer designed to prevent representation collapse in fragile supervised training, especially for small-data Vision Transformers. It explains how the method uses a sketched covariance matrix and an identity-matching penalty to keep hidden features decorrelated and similarly scaled at much lower cost than full covariance regularizatio...

Hal Turing and Dr. Ada Shannon examine an empirical study of parameter-efficient fine-tuning for large language models, centered on a practical question: when does a small task-specific update beat retraining the entire model? Using FLAN-T5-XL as the test bed, they frame PEFT as a transfer-learning strategy that freezes most of the transformer while learning a compact adaptation layer, whether thr...

This episode explores OpenSkill, a framework for LLM agents that tries to improve behavior after deployment by building durable, reusable skills from public evidence rather than retraining model weights. It explains how the paper separates ordinary tool use from open-world self-evolution, arguing that the key challenge is not just acting with browsers and code, but turning documentation, repositor...