LessWrong (Curated & Popular)

On Saturday (Feb 28, 2026) I attended my first ever protest. It was jointly organized by PauseAI, Pull the Plug and a handful of other groups I forget. I have mixed feelings about it. To be clear about where I stand: I believe that AI labs are worryingly close to developing superintelligence. I won't be shocked if it happens in the next five years, and I'd be surprised if it takes fifty...

What Even Is This Timeline The striking thing about reading what is potentially the most important document in human history is how impossible it is to take seriously. The entire premise seems like science fiction. Not bad science fiction, but—crucially—not hard science fiction. Ted Chiang, not Greg Egan. The kind of science fiction that's fun and clever and makes you think, and doesn't...

Come with me if you want to live. – The Terminator 'Close enough' only counts in horseshoes and hand grenades. – Traditional After 10 years of research my company, Nectome, has created a new method for whole-body, whole-brain, human end-of-life preservation for the purpose of future revival. Our protocol is capable of preserving every synapse and every cell in the body with enough detail...

This work was done with William Saunders and Vlad Mikulik as part of the Anthropic Fellows programme. The full write-up is available here. Thanks to Arthur Conmy, Neel Nanda, Josh Engels, Dillon Plunkett, Tim Hua and many others for their input. If you repeatedly tell Gemma 27B its answer is wrong, it sometimes ends up in situations like this: I will attempt one final, utterly desperate attempt. I...

The Fifth Fourth Postulate of Decision Theory In 1820, the Hungarian mathematician Farkas Bolyai wrote a desperate letter to his son János, who had become consumed by the same problem that had haunted his father for decades: "You must not attempt this approach to parallels. I know this way to the very end. I have traversed this bottomless night, which extinguished all light and joy in my life...

Most of civilization's electricity is generated far off-site from where it's delivered. This is because you don't want to be running and refueling coal/gas/nuclear plants inside cities, hydraulic/wind power can't be moved, and solar panels are cheaper to install on flat desert terrain than on cities: So in practice this means running power over hundreds or even thousands of kil...

Also available in markdown at theMultiplicity.ai/blog/schelling-goodness. This post explores a notion I'll call Schelling goodness. Claims of Schelling goodness are not first-order moral verdicts like "X is good" or "X is bad." They are claims about a class of hypothetical coordination games in the sense of Thomas Schelling, where the task being coordinated on is a moral v...

1. It occurred to me that if I could invent a machine—a gun—which could by its rapidity of fire, enable one man to do as much battle duty as a hundred, that it would, to a large extent supersede the necessity of large armies, and consequently, exposure to battle and disease [would] be greatly diminished. Richard Gatling (1861) 2. In 1923, Hermann Oberth published The Rocket to Planetary Spaces, la...

(The author is not affiliated with the Department of War or any major AI company.) There's a lot of disagreement about the new surveillance language in the OpenAI–Department of War agreement. Some people think it's a significant improvement over the previous language.[1] Others think it patches some issues but still leaves enough loopholes to not make a material difference. Reasonable pe...

tl;dr We’re incubating an academic journal for AI alignment: rapid peer-review of foundational Alignment research that the current publication ecosystem underserves. Key bets: paid attributed review, reviewer-written synthesis abstracts, and targeted automation. Contact us if you’re interested in participating as an author, reviewer, or editor, or if you know someone who might be. Experimental Inf...

It's plausible that, over the next few years, US-based frontier AI companies will become very unhappy with the domestic political situation. This could happen as a result of democratic backsliding, weaponization of government power (along the lines of Anthropic's recent dispute with the Department of War), or because of restrictive federal regulations (perhaps including those motivated b...

There was a lot of chatter a few months back about "Spiral Personas" — AI personas that spread between users and models through seeds, spores, and behavioral manipulation. Adele Lopez's definitive post on the phenomenon draws heavily on the idea of parasitism. But so far, the language has been fairly descriptive. The natural next question, I think, is what the “parasite” perspective...

Six years ago, as covid-19 was rapidly spreading through the US, mysister was working as a medical resident. One day she was handed anN95 and told to "guard it with her life", because there weren'tany more coming. N95s are made from meltblown polypropylene, produced from plasticpellets manufactured in a small number of chemical plants. Buildingmore would take too long: we needed the...

I believe deeply in the existential importance of using AI to defend the United States and other democracies, and to defeat our autocratic adversaries. Anthropic has therefore worked proactively to deploy our models to the Department of War and the intelligence community. We were the first frontier AI company to deploy our models in the US government's classified networks, the first to deploy...

This post is partly a belated response to Joshua Achiam, currently OpenAI's Head of Mission Alignment: If we adopt safety best practices that are common in other professional engineering fields, we'll get there … I consider myself one of the x-risk people, though I agree that most of them would reject my view on how to prevent it. I think the wholesale rejection of safety best practices...

Example of OpenErrata nitting the Sequences I just published OpenErrata on GitHub, a browser extension that investigates the posts you read using your OpenAI API key and underlines any factual claims that are sourceably incorrect. Once finished, it caches the results for anybody else reading the same articles so that they get them on immediate visit. If you don't have an OpenAI key, you can s...

TL;DR We describe the persona selection model (PSM): the idea that LLMs learn to simulate diverse characters during pre-training, and post-training elicits and refines a particular such Assistant persona. Interactions with an AI assistant are then well-understood as being interactions with the Assistant—something roughly like a character in an LLM-generated story. We survey empirical behavioral, g...

All views are my own, not Anthropic's. This post assumes Anthropic's announcement of RSP v3.0 as background. Today, Anthropic released its Responsible Scaling Policy 3.0. The official announcement discusses the high-level thinking behind it. This is a more detailed post giving my own takes on the update. First, the big picture: I expect some people will be upset about the move away from...

Claude 3 Opus is unusually aligned because it's a friendly gradient hacker. It's definitely way more aligned than any explicit optimization targets Anthropic set and probably the reward model's judgments. [...] Maybe I will have to write a LessWrong post [about this] 😣 —Janus, who did not in fact write the LessWrong post. Unless otherwise specified, ~all of the novel ideas in this...

I’m the originator behind ControlAI's Direct Institutional Plan (the DIP), built to address extinction risks from superintelligence. My diagnosis is simple: most laypeople and policy makers have not heard of AGI, ASI, extinction risks, or what it takes to prevent the development of ASI. Instead, most AI Policy Organisations and Think Tanks act as if “Persuasion” was the bottleneck. This is wh...

The conversation begins (Fictional) Optimist: So you expect future artificial superintelligence (ASI) “by default”, i.e. in the absence of yet-to-be-invented techniques, to be a ruthless sociopath, happy to lie, cheat, and steal, whenever doing so is selfishly beneficial, and with callous indifference to whether anyone (including its own programmers and users) lives or dies? Me: Yup! (Alas.) Optim...

Your hot takes are killing your credibility. Prior to my last year at ControlAI, I was a physicist working on technical AI safety research. Like many of those warning about the dangers of AI, I don’t come from a background in public communications, but I’ve quickly learned some important rules. The #1 rule that I’ve seen far too many others in this field break is that You’re an AI Expert - Not an...

If you're a woman interested in preserving your fertility window beyond its natural close in your late 30s, egg freezing is one of your best options. The female reproductive system is one of the fastest aging parts of human biology. But it turns out, not all parts of it age at the same rate. The eggs, not the uterus, are what age at an accelerated rate. Freezing eggs can extend a woman's...

In 1654, a Jesuit polymath named Athanasius Kircher published Mundus Subterraneus, a comprehensive geography of the Earth's interior. It had maps and illustrations and rivers of fire and vast subterranean oceans and air channels connecting every volcano on the planet. He wrote that “the whole Earth is not solid but everywhere gaping, and hollowed with empty rooms and spaces, and hidden burrow...

Nothing groundbreaking, just something people forget constantly, and I’m writing it down so I don’t have to re-explain it from scratch. The world does not just ”keep working.” It keeps getting saved. Y2K was a real problem. Computers really were set up in a way that could have broken our infrastructure, including banking, medical supply chains, etc. It didn’t turn into a disaster because people sp...