LessWrong (Curated & Popular)

https://www.lesswrong.com/posts/htrZrxduciZ5QaCjw/language-models-seem-to-be-much-better-than-humans-at-next Crossposted from the  AI Alignment Forum . May contain more technical jargon than usual. [Thanks to a variety of people for comments and assistance (especially Paul Christiano, Nostalgebraist, and Rafe Kennedy), and to various people for playing the game. Buck wrote the top-1 prediction web...

https://www.lesswrong.com/posts/PBRWb2Em5SNeWYwwB/humans-are-not-automatically-strategic Reply to: A "Failure to Evaluate Return-on-Time" Fallacy Lionhearted writes: [A] large majority of otherwise smart people spend time doing semi-productive things, when there are massively productive opportunities untapped. A somewhat silly example: Let's say someone aspires to be a comedian, the...

https://www.lesswrong.com/s/6xgy8XYEisLk3tCjH/p/CPP2uLcaywEokFKQG Tl;dr: I've noticed a dichotomy between "thinking in toolboxes" and "thinking in laws". The toolbox style of thinking says it's important to have a big bag of tools that you can adapt to context and circumstance; people who think very toolboxly tend to suspect that anyone who goes talking of a single op...

https://www.lesswrong.com/posts/jDQm7YJxLnMnSNHFu/moral-strategies-at-different-capability-levels Crossposted from the AI Alignment Forum . May contain more technical jargon than usual. Let’s consider three ways you can be altruistic towards another agent: You care about their welfare: some metric of how good their life is (as defined by you). I’ll call this care-morality - it endorses things like...

https://www.lesswrong.com/posts/xFotXGEotcKouifky/worlds-where-iterative-design-fails Crossposted from the AI Alignment Forum . May contain more technical jargon than usual. In most technical fields, we try designs, see what goes wrong, and iterate until it works. That’s the core iterative design loop. Humans are good at iterative design, and it works well in most fields in practice. In worlds whe...

https://www.lesswrong.com/posts/QBAjndPuFbhEXKcCr/my-understanding-of-what-everyone-in-technical-alignment-is Despite a clear need for it, a good source explaining who is doing what and why in technical AI alignment doesn't exist. This is our attempt to produce such a resource. We expect to be inaccurate in some ways, but it seems great to get out there and let Cunningham’s Law do its thing....

https://www.lesswrong.com/posts/rYDas2DDGGDRc8gGB/unifying-bargaining-notions-1-2 Crossposted from the AI Alignment Forum . May contain more technical jargon than usual. This is a two-part sequence of posts, in the ancient LessWrong tradition of decision-theory-posting. This first part will introduce various concepts of bargaining solutions and dividing gains from trade, which the reader may or ma...

https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators#fncrt8wagfir9 Summary TL;DR : Self-supervised learning may create AGI or its foundation. What would that look like? Unlike the limit of RL, the limit of self-supervised learning has received surprisingly little conceptual attention, and recent progress has made deconfusion in this domain more pressing. Existing AI taxonomies either fail...

https://www.lesswrong.com/posts/CjFZeDD6iCnNubDoS/humans-provide-an-untapped-wealth-of-evidence-about#fnref7a5ti4623qb Crossposted from the AI Alignment Forum . May contain more technical jargon than usual. TL;DR: To even consciously consider an alignment research direction,  you should have evidence to locate it as a promising lead. As best I can tell, many directions seem interesting but do not...

https://www.lesswrong.com/posts/DdDt5NXkfuxAnAvGJ/changing-the-world-through-slack-and-hobbies   Introduction In EA orthodoxy, if you're really serious about EA, the three alternatives that people most often seem to talk about are (1) “direct work” in a job that furthers a very important cause; (2) “earning to give” ; (3) earning “career capital” that will help you do those things in the futu...

https://www.lesswrong.com/posts/8oMF8Lv5jiGaQSFvo/boundaries-part-1-a-key-missing-concept-from-utility-theory Crossposted from the AI Alignment Forum . May contain more technical jargon than usual. This is Part 1 of my «Boundaries» Sequence on LessWrong. Summary: «Boundaries» are a missing concept from the axioms of game theory and bargaining theory, which might help pin-down certain features of m...

https://www.lesswrong.com/posts/MdZyLnLHuaHrCskjy/itt-passing-and-civility-are-good-charity-is-bad I often object to claims like "charity/steelmanning is an argumentative virtue". This post collects a few things I and others have said on this topic over the last few years. My current view is: Steelmanning ("the art of addressing the best form of the other person’s argument, even if...

https://www.lesswrong.com/posts/mmHctwkKjpvaQdC3c/what-should-you-change-in-response-to-an-emergency-and-ai Related to: Slack gives you the ability to notice/reflect on subtle things Epistemic status: A possibly annoying mixture of straightforward reasoning and hard-to-justify personal opinions. It is often stated (with some justification, IMO) that AI risk is an “emergency.”  Various people have...

https://www.lesswrong.com/posts/3pinFH3jerMzAvmza/on-how-various-plans-miss-the-hard-bits-of-the-alignment Crossposted from the AI Alignment Forum . May contain more technical jargon than usual. (As usual, this post was written by Nate Soares with some help and editing from Rob Bensinger.) In my   last post , I described a “hard bit” of the challenge of aligning AGI—the sharp left turn that comes...

https://www.lesswrong.com/posts/28zsuPaJpKAGSX4zq/humans-are-very-reliable-agents Over the last few years, deep-learning-based AI has progressed extremely rapidly in fields like natural language processing and image generation. However, self-driving cars seem stuck in perpetual beta mode, and aggressive predictions there have repeatedly been disappointing . Google's self-driving project start...

https://www.lesswrong.com/posts/2GxhAyn9aHqukap2S/looking-back-on-my-alignment-phd The funny thing about long periods of time is that they do, eventually, come to an end. I'm proud of what I accomplished during my PhD. That said, I'm going to first focus on mistakes I've made over the past four [1]  years. Mistakes I think I got significantly smarter in 2018–2019 , and kept learning...

https://www.lesswrong.com/posts/7iAABhWpcGeP5e6SB/it-s-probably-not-lithium A Chemical Hunger ( a ), a series by the authors of the blog Slime Mold Time Mold (SMTM) that has been   received positively   on LessWrong , argues that the obesity epidemic is entirely caused ( a ) by environmental contaminants. The authors’ top suspect  is lithium ( a ) [1] , primarily because it is known to cause weigh...

https://www.lesswrong.com/posts/bhLxWTkRc8GXunFcB/what-are-you-tracking-in-your-head A large chunk - plausibly the majority -  of real-world expertise seems to be in the form of  illegible skills : skills/knowledge which are hard to transmit by direct explanation. They’re not necessarily things which a teacher would even notice enough to consider important - just background skills or knowledge whi...

https://www.lesswrong.com/posts/Ke2ogqSEhL2KCJCNx/security-mindset-lessons-from-20-years-of-software-security Background I have been doing red team, blue team (offensive, defensive) computer security for a living since September 2000. The goal of this post is to compile a list of general principles I've learned during this time that are likely relevant to the field of AGI Alignment. If this i...

https://www.lesswrong.com/posts/CoZhXrhpQxpy9xw9y/where-i-agree-and-disagree-with-eliezer#fnh5ezxhd0an by paulfchristiano , 20th Jun 2022.  Crossposted from the AI Alignment Forum . May contain more technical jargon than usual. ( Partially in response to  AGI Ruin: A list of Lethalities .  Written in the same rambling style. Not exhaustive. ) Agreements Powerful AI systems have a good chance of de...

https://www.lesswrong.com/posts/keiYkaeoLHoKK4LYA/six-dimensions-of-operational-adequacy-in-agi-projects by Eliezer Yudkowsky Editor's note:  The following is a lightly edited copy of a document written by Eliezer Yudkowsky in November 2017. Since this is a snapshot of Eliezer’s thinking at a specific time, we’ve sprinkled reminders throughout that this is from 2017. A background note: It’s o...

https://www.lesswrong.com/posts/pL4WhsoPJwauRYkeK/moses-and-the-class-struggle "𝕿𝖆𝖐𝖊 𝖔𝖋𝖋 𝖞𝖔𝖚𝖗 𝖘𝖆𝖓𝖉𝖆𝖑𝖘. 𝕱𝖔𝖗 𝖞𝖔𝖚 𝖘𝖙𝖆𝖓𝖉 𝖔𝖓 𝖍𝖔𝖑𝖞 𝖌𝖗𝖔𝖚𝖓𝖉," said the bush. "No," said Moses. "Why not?" said the bush. "I am a Jew. If there's one thing I know about this universe it's that there's no such thing as God," said Moses. "You don't need to be certai...

https://www.lesswrong.com/posts/T6kzsMDJyKwxLGe3r/benign-boundary-violations Recently, my friend Eric asked me what sorts of things I wanted to have happen at my bachelor party. I said (among other things) that I'd really enjoy some benign boundary violations. Eric went ???? Subsequently: an essay. We use the word "boundary" to mean at least two things , when we're discussing p...

https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities   Crossposted from the AI Alignment Forum . May contain more technical jargon than usual. Preamble: (If you're already familiar with all basics and don't want any preamble, skip ahead to  Section B for technical difficulties of alignment proper.) I have several times failed to write up a well-organized list...