LessWrong (Curated & Popular)

https://www.lesswrong.com/posts/D7PumeYTDPfBTp3i7/the-waluigi-effect-mega-post In this article, I will present a mechanistic explanation of the Waluigi Effect and other bizarre "semiotic" phenomena which arise within large language models such as GPT-3/3.5/4 and their variants (ChatGPT, Sydney, etc). This article will be folklorish to some readers, and profoundly novel to others.

https://www.lesswrong.com/posts/3RSq3bfnzuL3sp46J/acausal-normalcy Crossposted from the AI Alignment Forum . May contain more technical jargon than usual. This post is also available on the EA Forum . Summary: Having thought a bunch about acausal trade — and proven some theorems relevant to its feasibility — I believe there do not exist powerful information hazards about it that stand up to clear...

https://www.lesswrong.com/posts/RryyWNmJNnLowbhfC/please-don-t-throw-your-mind-away [Warning: the following dialogue contains an incidental spoiler for "Music in Human Evolution" by Kevin Simler . That post is short, good, and worth reading without spoilers, and this post will still be here if you come back later. It's also possible to get the point of this post by skipping the dial...

https://www.lesswrong.com/posts/bxt7uCiHam4QXrQAA/cyborgism There is a lot of disagreement and confusion about the feasibility and risks associated with automating alignment research. Some see it as the default path toward building aligned AI, while others expect limited benefit from near term systems, expecting the ability to significantly speed up progress to appear well after misalignment and d...

https://www.lesswrong.com/posts/CYN7swrefEss4e3Qe/childhoods-of-exceptional-people This is a linkpost for https://escapingflatland.substack.com/p/childhoods Let’s start with one of those insights that are as obvious as they are easy to forget: if you want to master something, you should study the highest achievements of your field. If you want to learn writing, read great writers, etc. But this is...

https://www.lesswrong.com/posts/NJYmovr9ZZAyyTBwM/what-i-mean-by-alignment-is-in-large-part-about-making Crossposted from the AI Alignment Forum. May contain more technical jargon than usual. (Epistemic status: attempting to clear up a misunderstanding about points I have attempted to make in the past. This post is not intended as an argument for those points.) I have long said that the lion'...

https://www.lesswrong.com/posts/NRrbJJWnaSorrqvtZ/on-not-getting-contaminated-by-the-wrong-obesity-ideas A Chemical Hunger (a), a series by the authors of the blog Slime Mold Time Mold (SMTM), argues that the obesity epidemic is entirely caused (a) by environmental contaminants.  In my last post, I investigated SMTM’s main suspect (lithium).[1] This post collects other observations I have made abo...

https://www.lesswrong.com/posts/aPeJE8bSo6rAFoLqg/solidgoldmagikarp-plus-prompt-generation Work done at SERI-MATS , over the past two months, by Jessica Rumbelow and Matthew Watkins. TL;DR Anomalous tokens: a mysterious failure mode for GPT (which reliably insulted Matthew) We have found a set of anomalous tokens which result in a previously undocumented failure mode for GPT-2 and GPT-3 models. (T...

https://www.lesswrong.com/posts/Zp6wG5eQFLGWwcG6j/focus-on-the-places-where-you-feel-shocked-everyone-s Writing down something I’ve found myself repeating in different conversations: If you're looking for ways to help with the whole “the world looks pretty doomed” business, here's my advice: look around for places where we're all being total idiots. Look for places where everyone&ap...

https://www.lesswrong.com/posts/XPv4sYrKnPzeJASuk/basics-of-rationalist-discourse-1 Introduction This post is meant to be a linkable resource. Its core is a short list of guidelines (you can link directly to the list) that are intended to be fairly straightforward and uncontroversial, for the purpose of nurturing and strengthening a culture of clear thinking, clear communication, and collaborative...

https://www.lesswrong.com/posts/pDzdb4smpzT3Lwbym/my-model-of-ea-burnout (Probably somebody else has said most of this. But I personally haven't read it, and felt like writing it down myself, so here we go.) I think that EA [editor note: "Effective Altruism"] burnout usually results from prolonged dedication to satisfying the values you think you should have, while neglecting the va...

https://www.lesswrong.com/posts/PCrTQDbciG4oLgmQ5/sapir-whorf-for-rationalists Casus Belli: As I was scanning over my (rather long) list of essays-to-write, I realized that roughly a fifth of them were of the form "here's a useful standalone concept I'd like to reify," à la cup-stacking skills , fabricated options , split and commit , and sazen .  Some notable entries on that l...

https://www.lesswrong.com/posts/Xo7qmDakxiizG7B9c/the-social-recession-by-the-numbers This is a linkpost for https://novum.substack.com/p/social-recession-by-the-numbers Fewer friends, relationships on the decline, delayed adulthood, trust at an all-time low, and many diseases of despair. The prognosis is not great. One of the most discussed topics online recently has been friendships and loneline...

https://www.lesswrong.com/posts/pHfPvb4JMhGDr4B7n/recursive-middle-manager-hell I think Zvi's Immoral Mazes sequence is really important, but comes with more worldview-assumptions than are necessary to make the points actionable. I conceptualize Zvi as arguing for multiple hypotheses. In this post I want to articulate one sub-hypothesis, which I call "Recursive Middle Manager Hell"....

https://www.lesswrong.com/posts/L4anhrxjv8j2yRKKp/how-discovering-latent-knowledge-in-language-models-without Crossposted from the AI Alignment Forum . May contain more technical jargon than usual. Introduction A few collaborators and I recently released a new paper: Discovering Latent Knowledge in Language Models Without Supervision . For a quick summary of our paper, you can check out this Twitt...

https://www.lesswrong.com/posts/TWorNr22hhYegE4RT/models-don-t-get-reward Crossposted from the AI Alignment Forum . May contain more technical jargon than usual. In terms of content, this has a lot of overlap with Reward is not the optimization target . I'm basically rewriting a part of that post in language I personally find clearer, emphasising what I think is the core insight When thinking...

https://www.lesswrong.com/posts/mfPHTWsFhzmcXw8ta/the-feeling-of-idea-scarcity Here’s a story you may recognize. There's a bright up-and-coming young person - let's call her Alice. Alice has a cool idea. It seems like maybe an important idea, a big idea, an idea which might matter. A new and valuable idea. It’s the first time Alice has come up with a high-potential idea herself, somethin...

https://www.lesswrong.com/posts/qRtD4WqKRYEtT5pi3/the-next-decades-might-be-wild Crossposted from the AI Alignment Forum . May contain more technical jargon than usual. I’d like to thank Simon Grimm and Tamay Besiroglu for feedback and discussions. This post is inspired by What 2026 looks like and an AI vignette workshop guided by Tamay Besiroglu. I think of this post as “what would I expect the w...

https://www.lesswrong.com/posts/SqjQFhn5KTarfW8v7/lessons-learned-from-talking-to-greater-than-100-academics Crossposted from the AI Alignment Forum . May contain more technical jargon than usual. I’d like to thank MH, Jaime Sevilla and Tamay Besiroglu for their feedback. During my Master's and Ph. D. (still ongoing), I have spoken with many academics about AI safety. These conversations incl...

https://www.lesswrong.com/posts/6LzKRP88mhL9NKNrS/how-my-team-at-lightcone-sometimes-gets-stuff-done Disclaimer: I originally wrote this as a private doc for the Lightcone team. I then showed it to John and he said he would pay me to post it here. That sounded awfully compelling. However, I wanted to note that I’m an early founder who hasn't built anything truly great yet. I’m writing this do...

https://www.lesswrong.com/posts/rP66bz34crvDudzcJ/decision-theory-does-not-imply-that-we-get-to-have-nice Crossposted from the AI Alignment Forum . May contain more technical jargon than usual. ( Note: I wrote this with editing help from Rob and Eliezer. Eliezer's responsible for a few of the paragraphs. ) A common confusion I see in the tiny fragment of the world that knows about logical dec...

https://www.lesswrong.com/posts/6Xgy6CAf2jqHhynHL/what-2026-looks-like#2022 Crossposted from the AI Alignment Forum . May contain more technical jargon than usual. This was written for the Vignettes Workshop . [1] The goal is to write out a detailed future history (“trajectory”) that is as realistic (to me) as I can currently manage, i.e. I’m not aware of any alternative trajectory that is similar...

https://www.lesswrong.com/posts/REA49tL5jsh69X3aM/introduction-to-abstract-entropy#fnrefpi8b39u5hd7 This post, and much of the following sequence, was greatly aided by feedback from the following people (among others): Lawrence Chan , Joanna Morningstar , John Wentworth , Samira Nedungadi , Aysja Johnson , Cody Wild , Jeremy Gillen , Ryan Kidd , Justis Mills and Jonathan Mustin . Illustrations by...

https://www.lesswrong.com/posts/8vesjeKybhRggaEpT/consider-your-appetite-for-disagreements Poker There was a time about five years ago where I was trying to get good at poker. If you want to get good at poker, one thing you have to do is review hands. Preferably with other people. For example, suppose you have ace king offsuit on the button. Someone in the highjack opens to 3 big blinds preflop. Y...