LessWrong (Curated & Popular)

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual. Quintin Pope & Nora Belrose have a new “AI Optimists” website, along with a new essay “AI is easy to control”, arguing that the risk of human extinction due to future AI (“AI x-risk”) is a mere 1% (“a tail risk worth considering, but not the dominant source of risk in the world”). (I’m much more pessimistic....

Any community which ever adds new people will need to either routinely teach the new and (to established members) blindingly obvious information to those who genuinely haven’t heard it before, or accept that over time community members will only know the simplest basics by accident of osmosis or selection bias. There isn’t another way out of that. You don’t get to stop doing it. If you have a vibr...

The author's Substack: https://substack.com/@homosabiens Support ongoing human narrations of LessWrong's curated posts: www.patreon.com/LWCurated You know it must be out there, but you mostly never see it. Author's Note 1: In something like 75% of possible futures, this will be the last essay that I publish on LessWrong.  Future content will be available on my substack , where I&apo...

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual. Summary. You can’t optimise an allocation of resources if you don’t know what the current one is. Existing maps of alignment research are mostly too old to guide you and the field has nearly no ratchet, no common knowledge of what everyone is doing and why, what is abandoned and why, what is renamed, what relate...

Status: Vague, sorry. The point seems almost tautological to me, and yet also seems like the correct answer to the people going around saying “LLMs turned out to be not very want-y, when are the people who expected 'agents' going to update?”, so, here we are. Okay, so you know how AI today isn't great at certain... let's say "long-horizon" tasks? Like novel large-scal...

Support ongoing human narrations of curated posts: www.patreon.com/LWCurated Recently, I have been learning about industry norms, legal discovery proceedings, and incentive structures related to companies building risky systems. I wanted to share some findings in this post because they may be important for the frontier AI community to understand well.  TL;DR Documented communications of risks (esp...

Previously: OpenAI: Facts from a Weekend. On Friday afternoon, OpenAI's board fired CEO Sam Altman. Overnight, an agreement in principle was reached to reinstate Sam Altman as CEO of OpenAI, with an initial new board of Brad Taylor (ex-co-CEO of Salesforce, chair), Larry Summers and Adam D’Angelo. What happened? Why did it happen? How will it ultimately end? The fight is far from over. We do...

Approximately four GPTs and seven years ago, OpenAI's founders brought forth on this corporate landscape a new entity, conceived in liberty, and dedicated to the proposition that all men might live equally when AGI is created. Now we are engaged in a great corporate war, testing whether that entity, or any entity so conceived and so dedicated, can long endure. What matters is not theory but p...

This is a linkpost for https://openai.com/blog/openai-announces-leadership-transitionBasically just the title, see the OAI blog post for more details. Mr. Altman's departure follows a deliberative review process by the board, which concluded that he was not consistently candid in his communications with the board, hindering its ability to exercise its responsibilities. The board no longer has...

You know it must be out there, but you mostly never see it. Author's Note 1: I'm something like 75% confident that this will be the last essay that I publish on LessWrong. Future content will be available on my substack, where I'm hoping people will be willing to chip in a little commensurate with the value of the writing, and (after a delay) on my personal site. I decided to post t...

Here's a recent conversation I had with a friend: Me: "I wish I had more friends. You guys are great, but I only get to hang out with you like once or twice a week. It's painful being holed up in my house the entire rest of the time."Friend: "You know ${X}. You could talk to him."Me: "I haven't talked to ${X} since 2019."Friend: "Why does that matt...

Support ongoing human narrations of curated posts: www.patreon.com/LWCurated I'm sure Harry Potter and the Methods of Rationality taught me some of the obvious, overt things it set out to teach. Looking back on it a decade after I first read it however, what strikes me most strongly are often the brief, tossed off bits in the middle of the flow of a story. Fred and George exchanged worried gl...

It’s fairly common for EA orgs to provide fiscal sponsorship to other EA orgs.  Wait, no, that sentence is not quite right. The more accurate sentence is that there are very few EA organizations, in the legal sense; most of what you think of as orgs are projects that are legally hosted by a single org, and which governments therefore consider to be one legal entity. Source: https://www.lesswrong.c...

Support ongoing human narrations of curated posts: www.patreon.com/LWCurated How many years will pass before transformative AI is built? Three people who have thought about this question a lot are Ajeya Cotra from Open Philanthropy , Daniel Kokotajlo from OpenAI and Ege Erdil from Epoch . Despite each spending at least hundreds of hours investigating this question, they still still disagree substa...

habryka Ok, so we both had some feelings about the recent Conjecture post on "lots of people in AI Alignment are lying" , and the associated marketing campaign and stuff .  I would appreciate some context in which I can think through that, and also to share info we have in the space that might help us figure out what's going on.  I expect this will pretty quickly cause us to end up...

1. There's a supercharged, dire wolf form of the bystander effect that I’d like to shine a spotlight on. First, a quick recap. The Bystander Effect is a phenomenon where people are less likely to help when there's a group around. When I took basic medical training, I was told to always ask one specific person to take actions instead of asking a crowd at large. “You, in the green shirt! C...

davidad has a 10-min talk out on a proposal about which he says: “the first time I’ve seen a concrete plan that might work to get human uploads before 2040, maybe even faster, given unlimited funding”. I think the talk is a good watch, but the dialogue below is pretty readable even if you haven't seen it. I'm also putting some summary notes from the talk in the Appendix of this dialogue....

I guess there’s maybe a 10-20% chance of AI causing human extinction in the coming decades, but I feel more distressed about it than even that suggests—I think because in the case where it doesn’t cause human extinction, I find it hard to imagine life not going kind of off the rails. So many things I like about the world seem likely to be over or badly disrupted with superhuman AI (writing, explai...

Recently, I have been learning about industry norms, legal discovery proceedings, and incentive structures related to companies building risky systems. I wanted to share some findings in this post because they may be important for the frontier AI community to understand well.  TL;DR Documented communications of risks (especially by employees) make companies much more likely to be held liable in co...

Support ongoing human narrations of curated posts: www.patreon.com/LWCurated This is a linkpost for https://transformer-circuits.pub/2023/monosemantic-features/ Text of post based on our blog post as a linkpost for the full paper which is considerably longer and more detailed. Neural networks are trained on data, not programmed to follow rules. We understand the math of the trained network exactly...

Support ongoing human narrations of curated posts: www.patreon.com/LWCurated (You can sign up to play deception chess here if you haven't already.) This is the first of my analyses of the deception chess games. The introduction will describe the setup of the game, and the conclusion will sum up what happened in general terms; the rest of the post will mostly be chess analysis and skippable if...

This is a linkpost for https://nitter.net/ESYudkowsky/status/1718654143110512741 Comp sci in 2017: Student:  I get the feeling the compiler is just ignoring all my comments. Teaching assistant:  You have failed to understand not just compilers but the concept of computation itself. Comp sci in 2027: Student:  I get the feeling the compiler is just ignoring all my comments. TA:  That's weird. ...

A common theme implicit in many AI risk stories has been that broader society will either fail to anticipate the risks of AI until it is too late, or do little to address those risks in a serious manner. In my opinion, there are now clear signs that this assumption is false, and that society will address AI with something approaching both the attention and diligence it deserves. For example, one c...

I examined all the biorisk-relevant citations from a policy paper arguing that we should ban powerful open source LLMs. None of them provide good evidence for the paper's conclusion. The best of the set is evidence from statements from Anthropic -- which rest upon data that no one outside of Anthropic can even see, and on Anthropic's interpretation of that data. The rest of the evidence...

This is a linkpost for https://www.whitehouse.gov/briefing-room/statements-releases/2023/10/30/fact-sheet-president-biden-issues-executive-order-on-safe-secure-and-trustworthy-artificial-intelligence/ Released today (10/30/23) this is crazy, perhaps the most sweeping action taken by government on AI yet.  Below, I've segmented by x-risk and non-x-risk related proposals, excluding the proposal...