Eliezer Yudkowsky seems really depressed these days.
There was this change, sometime around the “Late 2021 MIRI Conversations.” It’s visible in those dialogues and all his more recent output.
The change does not seem to consist in him changing his mind about anything, in the usual sense. Some of his “higher-level” opinions have changed – I don’t think he used to be as critical of literally all existing alignment research – but in a way that he struggles to explain in terms of specific, lower-level facts and mechanisms and arguments.
It doesn’t look as though he noticed a worrying trend, or devised a worrying argument, and became full of despair as a result. It looks like he just Became Full of Despair, as an atomic action.
And now he has this deep, intuitive sense that his brand of despair is deeply correct, true, fundamental, simply The Way The World Works – a sense that is beyond his power to transmit in words to anyone who does not already hear the same song ringing deep in their own mind.
The earlier “MIRI conversations” are full of lamentations that he cannot convey, or does not have the energy left to convey, the special thing(s) he knows, the ones no one else gets, which would drive you to Despair too, if only you could be shown:
In particular […] I also have a model in which people think “why not just build an AI which does X but not Y?” because they don’t realize what X and Y have in common, which is something that draws deeply on having deep models of intelligence. And it is hard to convey this deep theoretical grasp.
[…] past experience has led me to believe that conveying [”my intuitions about how cognition works”] in a form that the Other Mind will actually absorb and operate, is really quite hard and takes a long discussion, relative to my current abilities to Actually Explain things […] (source)
I don’t know what homework exercises to give people to make them able to see “consequentialism” all over the place, instead of inventing slightly new forms of consequentialist cognition and going “Well, now that isn’t consequentialism, right?” (source)
I think that to contain the concept of Utility as it exists in me, you would have to do homework exercises I don’t know how to prescribe. (source)
I just… don’t know what to do when people talk like this. […] This just - isn’t how to understand reality. […] This isn’t sane. (source)
Some of my current thoughts are a reiteration of old despair: It feels to me like the typical Other within EA has no experience with discovering unexpected order, with operating a generalization that you can expect will cover new cases even when that isn’t immediately obvious […] They have no experience operating genuinely useful, genuinely deep generalizations that extend to nonobvious things. […] So trying to convey the real source of the knowledge feels doomed. It’s a kind of idea that our civilization has lost, like that college class Feynman ran into. (source)
And empirically, it has already been shown to me that I do not have the power to break people out of the hypnosis of nodding along with Hansonian arguments, even by writing much longer essays than this. […] Reality just… doesn’t work like this on some deep level. […] There is a set of intuitive generalizations from experience which rules that out, which I do not know how to convey. […] But this, I empirically do not seem to know how to convey to people, in advance of the inevitable and predictable contradiction by a reality which is not as fond of Hansonian dynamics as Hanson. […]
And then there is another essay in 3 months. There is an infinite well of them. I would have to teach people to stop drinking from the well, instead of trying to whack them on the back until they cough up the drinks one by one, or actually, whacking them on the back and then they don’t cough them up until reality contradicts them, and then a third of them notice that and cough something up, and then they don’t learn the general lesson and go back to the well and drink again. And I don’t know how to teach people to stop drinking from the well. I tried to teach that. I failed. If I wrote another Sequence I have no idea to believe that Sequence would work.
So what EAs will believe at the end of the world, will look like whatever the content was of the latest bucket from the well of infinite slow-takeoff arguments that hasn’t yet been blatantly-even-to-them refuted by all the sharp jagged rapidly-generalizing things that happened along the way to the world’s end.
And I know, before anyone bothers to say, that all of this reply is not written in the calm way that is right and proper for such arguments. I am tired. I have lost a lot of hope. There are not obvious things I can do, let alone arguments I can make, which I expect to be actually useful in the sense that the world will not end once I do them. I don’t have the energy left for calm arguments. What’s left is despair that can be given voice. (source)
And like, sure, he’s always kind of talked like this, about how no one understands AI risk like he does, and that’s why they aren’t scared like he is.
But that’s just the thing – he has held something like this set of positions, and something like this role in relation to the broader “AI conversation,” for well over a decade. And yet he was not like this until very recently, until the change.
I could take him at his word, and suppose that a straw simply broke the camel’s back. That there was some specific number N such that he could beat this drum for N years but not N+1, some number M such that he could write M blog posts trying to explain the same thing but give up before writing the (M+1)st. Maybe that is true.
But it does seem noteworthy how the change happened so suddenly; how it did not seem driven by any particular set of events in the outside world; how it did not result in a simple sigh and the words “I’m tired of explaining,” but instead in a stream of posts and “conversations” attempting to communicate some new, dark view about the utter inexorability of utter failure, which even his closest colleagues struggle in vain to grasp on an intellectual level. How he now sounds like my own inner monologue does in spells of depression, when he never did before, not in all those 15-odd years of Cassandrahood.
I know “Yudkowsky critic” is supposed to be part of my online “brand,” or something, or at least it was a decade ago, but in all seriousness – I hope the guy is all right, and I hope there are people close to him who would be able to notice and help if he weren’t.
I also hope that his recent writing doesn’t send a bunch of other people spiraling into despair, beyond what would be licensed by its capacity to rationally persuade them of some despair-inducing set of conclusions. And if it does send some people into despair simply via its depressive tone, or because they think “a guy I respect is panicking, so I should panic too,” then I hope they can find their way back out swiftly.
