BlueDot Narrated

Audio versions of blogs and papers from BlueDot courses. Using a sparse autoencoder, we extract a large number of interpretable features from a one-layer transformer. Mechanistic interpretability seeks to understand neural networks by breaking them into components that are more easily understood than the whole. By understanding the function of each component, and how they interact, we hope to be a...

Audio versions of blogs and papers from BlueDot courses. By studying the connections between neurons, we can find meaningful algorithms in the weights of neural networks.  Many important transition points in the history of science have been moments when science “zoomed in.” At these points, we develop a visualization or tool that allows us to see the world in a new level of detail, and a new field...

Audio versions of blogs and papers from BlueDot courses. Widely used alignment techniques, such as reinforcement learning from human feedback (RLHF), rely on the ability of humans to supervise model behavior—for example, to evaluate whether a model faithfully followed instructions or generated safe outputs. However, future superhuman models will behave in complex ways too difficult for humans to r...

Audio versions of blogs and papers from BlueDot courses. Reinforcement learning from human feedback (RLHF) has emerged as a powerful technique for steering large language models (LLMs) toward desired behaviours. However, relying on simple human feedback doesn’t work for tasks that are too complex for humans to accurately judge at the scale needed to train AI models. Scalable oversight techniques a...

Audio versions of blogs and papers from BlueDot courses. The two tasks of supervised learning: regression and classification. Linear regression, loss functions, and gradient descent. How much money will we make by spending more dollars on digital advertising? Will this loan applicant pay back the loan or not? What’s going to happen to the stock market tomorrow? Original article: https://medium.com...

Audio versions of blogs and papers from BlueDot courses. AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportuniti...

Audio versions of blogs and papers from BlueDot courses. It seems unlikely that humans are near the ceiling of possible intelligences, rather than simply being the first such intelligence that happened to evolve. Computers far outperform humans in many narrow niches (e.g. arithmetic, chess, memory size), and there is reason to believe that similar large improvements over human performance are poss...

Audio versions of blogs and papers from BlueDot courses. In 1972, the Nobel prize-winning physicist Philip Anderson wrote the essay "More Is Different". In it, he argues that quantitative changes can lead to qualitatively different and unexpected phenomena. While he focused on physics, one can find many examples of More is Different in other domains as well, including biology, economics,...

Audio versions of blogs and papers from BlueDot courses. Machine learning is touching increasingly many aspects of our society, and its effect will only continue to grow. Given this, I and many others care about risks from future ML systems and how to mitigate them. When thinking about safety risks from ML, there are two common approaches, which I'll call the Engineering approach and the Phil...

Audio versions of blogs and papers from BlueDot courses. I've been trying to review and summarize Eliezer Yudkowksy's recent dialogues on AI safety. Previously in sequence: Yudkowsky Contra Ngo On Agents . Now we’re up to Yudkowsky contra Cotra on biological anchors, but before we get there we need to figure out what Cotra's talking about and what's going on. The Open Philanthr...

Audio versions of blogs and papers from BlueDot courses. MIRI’s mission is to ensure that the creation of smarter-than-human artificial intelligence has a positive impact. Why is this mission important, and why do we think that there’s work we can do today to help ensure any such thing? In this post and my next one, I’ll try to answer those questions. This post will lay out what I see as the four...

Audio versions of blogs and papers from BlueDot courses. This report explores the core case for why the development of artificial general intelligence (AGI) might pose an existential threat to humanity. It stems from my dissatisfaction with existing arguments on this topic: early work is less relevant in the context of modern machine learning, while more recent work is scattered and brief. This re...

Audio versions of blogs and papers from BlueDot courses. Within the coming decades, artificial general intelligence (AGI) may surpass human capabilities at a wide range of important tasks. We outline a case for expecting that, without substantial effort to prevent it, AGIs could learn to pursue goals which are undesirable (i.e. misaligned) from a human perspective. We argue that if AGIs are traine...

Audio versions of blogs and papers from BlueDot courses. One approach to the AI control problem goes like this: Observe what the user of the system says and does. Infer the user’s preferences. Try to make the world better according to the user’s preference, perhaps while working alongside the user and asking clarifying questions. This approach has the major advantage that we can begin empirical wo...

Audio versions of blogs and papers from BlueDot courses. According to the orthogonality thesis, intelligent agents may have an enormous range of possible final goals. Nevertheless, according to what we may term the “instrumental convergence” thesis, there are some instrumental goals likely to be pursued by almost any intelligent agent, because there are some objectives that are useful intermediari...

Audio versions of blogs and papers from BlueDot courses. One step towards building safe AI systems is to remove the need for humans to write goal functions, since using a simple proxy for a complex goal, or getting the complex goal a bit wrong, can lead to undesirable and even dangerous behavior. In collaboration with DeepMind’s safety team, we’ve developed an algorithm which can infer what humans...

Audio versions of blogs and papers from BlueDot courses. Crossposted from the AI Alignment Forum. May contain more technical jargon than usual. The stereotyped image of AI catastrophe is a powerful, malicious AI system that takes its creators by surprise and quickly achieves a decisive advantage over the rest of humanity. I think this is probably not what failure will look like, and I want to try...

Audio versions of blogs and papers from BlueDot courses. Our goal here is to popularize obscure and hard-to-understand areas of AI alignment. So let’s try to understand the incomprehensible meme!  Our main source will be Hubinger et al 2019, Risks From Learned Optimization In Advanced Machine Learning Systems . Mesa- is a Greek prefix which means the opposite of meta-. To “go meta” is to go one le...

Audio versions of blogs and papers from BlueDot courses. As we build increasingly advanced AI systems, we want to make sure they don’t pursue undesired goals. This is the primary concern of the AI alignment community. Undesired behaviour in an AI agent is often the result of specification gaming —when the AI exploits an incorrectly specified reward. However, if we take on the perspective of the ag...

Audio versions of blogs and papers from BlueDot courses. Previously, I argued that we should expect future ML systems to often exhibit "emergent" behavior, where they acquire new capabilities that were not explicitly designed or intended, simply as a result of scaling. This was a special case of a general phenomenon in the physical sciences called More Is Different. I care about this bec...

Audio versions of blogs and papers from BlueDot courses. Previously, I've argued that future ML systems might exhibit unfamiliar, emergent capabilities, and that thought experiments provide one approach towards predicting these capabilities and their consequences. In this post I’ll describe a particular thought experiment in detail. We’ll see that taking thought experiments seriously often su...

Audio versions of blogs and papers from BlueDot courses. (Partially in response to AGI Ruin: A list of Lethalities. Written in the same rambling style. Not exhaustive.) Agreements Powerful AI systems have a good chance of deliberately and irreversibly disempowering humanity. This is a much easier failure mode than killing everyone with destructive physical technologies. Catastrophically risky AI s...

Audio versions of blogs and papers from BlueDot courses. I have several times failed to write up a well-organized list of reasons why AGI will kill you. People come in with different ideas about why AGI would be survivable, and want to hear different obviously key points addressed first. Some fraction of those people are loudly upset with me if the obviously most important points aren't addre...

Audio versions of blogs and papers from BlueDot courses. Why would we program AI that wants to harm us? Because we might not know how to do otherwise. Source: https://www.cold-takes.com/why-ai-alignment-could-be-hard-with-modern-deep-learning/ Crossposted from the Cold Takes Audio podcast. --- A podcast by BlueDot Impact .

Audio versions of blogs and papers from BlueDot courses. In 2008, thousands of blog readers - including yours truly, who had discovered the rationality community just a few months before - watched Robin Hanson debate Eliezer Yudkowsky on the future of AI. Robin thought the AI revolution would be a gradual affair, like the Agricultural or Industrial Revolutions. Various people invent and improve va...