AI Post Transformers

This episode explores RAPTOR, a method for extracting concept directions from language model hidden states using ridge-regularized logistic probes, with the goal of making those directions accurate enough for interpretation and stable enough for activation steering. It explains the core probe-then-steer workflow, why linear probes can reveal what a model has encoded, and why good classification ac...

This episode explores a 2026 paper on “split personality training,” a method for attaching an internal reviewer to a language model that can reveal what the model knows about its own deceptive or reward-hacking behavior without changing the answer shown to the user. It situates the work in the broader lineage of latent knowledge elicitation, alignment faking, and mechanistic interpretability, expl...

This episode explores a mechanistic interpretability paper arguing that transformers often fail at counting not because they lack an internal notion of quantity, but because the pathway that converts that latent count into digit tokens is poorly aligned. It explains key ideas like linear probes, the logit lens, attention, LoRA, constrained next-token evaluation, and autoregressive generation to sh...

This episode explores a historical argument that simple Hebbian learning rules are too weak to explain real intelligence, because they mainly capture local correlations rather than solving multivariate credit-assignment problems. It examines how that critique points toward global optimization methods such as backpropagation and, in some settings, reinforcement learning, while contrasting their eng...

This episode explores LAPS, a serving system for large language models that treats long prompt prefills and short multi-turn re-prefills as fundamentally different workloads instead of batching them together. It explains why user-perceived latency, especially time to first token, suffers when tiny follow-up requests get stuck behind large compute-heavy context loads, and how LAPS models the bounda...

This episode explores a 2026 paper, Generative Modeling via Drifting, which argues that the hard transport process behind modern generative models can be moved into training so that inference becomes a single forward pass. It explains the core idea of a pushforward distribution, introduces the paper’s notion of a drifting field that nudges generated samples toward the data distribution during opti...

This episode explores a March 2026 paper arguing that LLM-based judges are an unreliable way to measure jailbreak success and adversarial robustness. It explains how modern safety evaluations rely on judge models to score harmful outputs, then walks through why those judges can break under attack shift, model shift, and data shift, sometimes degrading to near coin-flip reliability. The discussion...

This episode explores EverMemOS, a memory system for long-lived AI agents that tries to organize past interactions into structured, higher-level semantic “scenes” instead of relying on flat retrieval alone. It explains why bigger context windows and standard RAG often fail when agents accumulate stale preferences, conflicting facts, and fragmented conversational traces, arguing that the real probl...

This episode explores a position paper arguing that modern LLM serving has outgrown simple heuristics like FIFO, shortest-queue routing, and LRU eviction. It explains why transformer inference creates harder control problems than standard inference, focusing on continuous batching, KV-cache growth, and the tension between compute-heavy prefill and memory-bound decode phases. The discussion highlig...

This episode explores a paper on long-context compression that argues standard “soft compression” methods, which rely on learned memory or gist tokens, lose information because those tokens get overwritten across layers and fail to coordinate what each slot should retain. It explains the paper’s alternative design, which keeps the language model backbone frozen and instead explicitly transmits inf...

This episode explores CL-BENCH, a benchmark designed to test whether language models can actually learn task-specific knowledge from long, messy context and then reason with it, rather than merely retrieving facts or mimicking examples. It explains the distinction between long-context understanding, in-context learning, and the stronger notion of context learning, using examples like legal codes,...

This episode explores SGLang, a system for making complex language model workflows run faster by treating them as full programs rather than single prompt-response calls. It explains how modern LLM applications involve branching, tool use, retries, and structured outputs, then examines SGLang’s co-design of a Python-embedded language with a specialized runtime that can optimize those patterns direc...

This episode explores the TensorFlow paper as a systems argument for unifying the full machine learning lifecycle, from mobile inference to large-scale distributed training, within a single stateful dataflow framework. It explains how TensorFlow represents computation as graphs with mutable state, why that mattered for device placement, parameter storage, checkpointing, and heterogeneous hardware,...

This episode explores an FPGA-based time-to-digital converter that combines careful delay-line layout with machine-learning-based calibration to achieve very fine timing measurements on real hardware. It explains how tapped-delay-line TDCs work, why real FPGA implementations suffer from nonuniform time bins and bubble errors, and why those imperfections matter for applications like LiDAR, medical...

This episode explores a mechanistic interpretability study asking whether a language model can detect when a concept has been injected into its hidden activations and, in some cases, identify what that concept was. It explains the difference between detection and identification, walks through activation steering in the residual stream, and highlights the paper’s controlled experiments on Gemma3-27...

This episode explores the 1959 paper on frog vision that argued the retina does far more than passively relay a camera-like image to the brain. It explains how experiments on single optic nerve fibers revealed specialized visual detectors tuned to ecologically relevant signals such as small moving dark objects, edges, dimming, and contrast changes, rather than raw brightness alone. The discussion...

This episode explores Marvin Minsky’s 1961 paper on whether mostly random neural networks can learn useful behavior simply by reinforcing successful responses. It explains how the paper distinguishes rote memory, associative recall, pattern recognition, and true generalization, arguing that reward signals alone are not enough unless the system already has a meaningful notion of similarity between...

This episode explores Paul Werbos’s 1990 paper on Backpropagation Through Time and explains how ordinary backpropagation extends to systems whose state evolves over time. It walks through the core idea of unrolling a recurrent or dynamic system into a time-indexed computation graph, then applying reverse-mode differentiation to compute exact gradients across both layers and time steps. The discuss...

This episode explores why LightGBM became a dominant tool for tabular machine learning by unpacking the algorithmic and systems ideas behind its speed. It explains how gradient boosting decision trees work, why split search becomes expensive on massive sparse datasets, and how LightGBM differs from neural-network-style training despite using gradient information. The discussion focuses on two core...

This episode explores how boosted decision trees can be compiled directly into FPGA firmware for ultra-low-latency particle-physics triggers at the Large Hadron Collider. It explains why this setting favors shallow, quantized tree ensembles over larger neural networks: trigger decisions must happen within a tiny hardware budget, with strict limits on latency, power, and on-chip resources. The disc...

This episode explores how a 2018 paper brings neural network inference into the Level-1 trigger at the Large Hadron Collider, where event decisions must be made under sub-microsecond latency constraints. It explains why FPGAs are a natural fit for this setting, emphasizing batch-one, deterministic inference and the hardware realities that make model size, timing, memory use, and routing just as im...

This episode explores how the CMS experiment uses machine learning inside its Level-1 endcap muon trigger, where hardware must estimate muon momentum within roughly 500 nanoseconds while filtering an enormous stream of proton-collision data. It explains why boosted decision trees were chosen over neural networks: not because they are trendier, but because they fit strict FPGA constraints around de...

This episode explores the 2016 Caffeine FPGA accelerator and its central claim that a single FPGA design can handle an entire CNN efficiently, rather than excelling at convolutions while bottlenecking on fully connected layers. It explains why that mattered in the AlexNet-to-VGG era, when convolutional layers were compute-bound but dense layers often became communication-bound because moving weigh...

This episode explores why Caffe mattered as a systems breakthrough for the early CNN era, even though it did not introduce a new learning algorithm. It explains how the framework helped researchers and engineers move from handcrafted vision features to learned feature embeddings, and why separating model definition from implementation made experimentation and deployment far more practical. The dis...

This episode explores FP-DNN, a 2017 framework that aims to compile TensorFlow-era neural networks onto FPGAs automatically, reducing the need for hand-designed accelerators for each model. It explains how the system maps convolutional layers, fully connected layers, and parts of LSTM computation into a shared matrix-multiplication core, while combining hand-tuned RTL for performance-critical comp...