Mechanical Dirk

Mechanical Dreams

An automatically generated podcast about machine learning and natural language processing. The two fictional hosts talk about papers that I want to learn more about on my way to work. It's not good, but it's useful.

Husk å besøke podkastens nettsted og støtte skaperen: mechanicaldreams.mechanicaldirk.com

Forfatter

Mechanical Dirk

Kategori

Science

Siste episode

27. sep 2026

Hvor kan du lytte?

Podkaster i appen Replaio Radio Kommer snart

Podkaster kommer snart til appen. Installer nå, og bli den første som ser en helt ny tilnærming til podkaster

Last ned på Google Play Installer gratis Android nesten 10 mill. nedlastinger · 4,8 i vurdering iOS snart

Episoder

Continual Pre-training of MoEs 12.03.2025
s1 - Simple test-time scaling 06.03.2025
Cognitive Behaviors that Enable Self-Improving Reasoners 05.03.2025
Phi 4 Multimodal Instruct 04.03.2025
Claude 3.7 Sonnet System Card 25.02.2025
Project Sid: Many-agent simulations toward AI civilization 09.02.2025
The Surprising Agreement Between Convex Optimization Theory and Learning-Rate Scheduling for Large Model Training 09.02.2025
Scaling Laws for Optimal Sparsity for Mixture-of-Experts Language Models 05.02.2025
NExtLong - Toward Effective Long-Context Training without Long Documents 30.01.2025
Over-Tokenized Transformer 30.01.2025
Optimizing Pretraining Data Mixtures with LLM-Estimated Utility 30.01.2025
HashAttention: Semantic Sparsity for Faster Inference 16.01.2025
From Tokens to Words 15.01.2025
DeepSeek V3 07.01.2025
Optimal Linear Decay Learning Rate Schedules and Further Refinements 05.01.2025

In this episode: • The Death of Cosine?: Introduction to the episode and the paper. Professor Norris expresses his skepticism about changing established habits like Cosine Annealing, while Linda teases a shake-up in the status quo. • Theory vs. Reality: A discussion on the massive gap between theoretical learning rates (like 1/t) and what practitioners actually use. Linda explains why the theory h...

Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers 20.12.2024
Efficient and Approximate Per-Example Gradient Norms for Gradient Noise Scale 20.12.2024
Phi-4 14.12.2024
Rephrasing natural text data with different languages and quality levels 13.12.2024
Unveiling and Consulting Core Experts in Retrieval-Augmented MoE-based LLMs 12.12.2024
EXAONE 3.5 11.12.2024
Model soups - averaging weights of multiple fine-tuned models improves accuracy without increasing inference time 09.12.2024
Model-Aware Data Selection for Efficient Pretraining with Data Influence Models 06.12.2024
Nemotron-CC 05.12.2024
Tülu 3 02.12.2024

Hør på podkasten Mechanical Dreams i Replaio

Radio og podkaster i én app - gratis og uten registrering. Installer i dag, og ikke gå glipp av lanseringen

Last ned på Google Play

Replaio er ikke podkastutgiver; programnavn, omslag og lyd tilhører opphavspersonene og distribueres via offentlige RSS-feeder.