The Sand Spirits Are Coming

Diego’s worldview on the AI transition and existential risk

This room has a magnificent view, two emergency exits, and a host who would like you to understand that the fire alarm is connected to something. Diego does not regard artificial intelligence as one more item on the policy menu. He expects it to end the period in which human beings determine what happens on Earth. What follows could be a catastrophe, or an existence so good that our present vocabulary arrives inadequately dressed. His practical question is how to get himself, the people he loves, and the rest of the human enterprise through the transition alive.

One essential instruction at the entrance: do not translate his warnings into something more conventional before considering them. In July 2026 he went to considerable trouble to prevent exactly that:

In other words, I mean AI will actually, not metaphorically, in this world, not in SimCity, soon, not when you're 95 in the hospital, kill you, everyone you love, and everyone you hate. Unless of course, it ends up having goals that are favorable to humans as well as stable. Nobody knows how to do that.¹

The final condition is central. Diego is describing a fork in the future, not a metaphysical requirement that computers be wicked. There is a good branch. He would very much like to be on it.

The apprentice learns there is no larger adult

He came to this territory through the transhumanist communities of the early 2000s: life extension, the possibility of artificial minds, the Singularity, and a future vastly larger than the one advertised in school. The possibility of defeating death was already entwined with the possibility of being destroyed by the tools that could defeat it.

His 2024 retrospective distinguishes the early period of reading great thinkers from the later period of discovering that great thinkers were actual people. Initially, somewhere beyond Brazil, intelligent adults were presumed to be managing the situation. Then he visited their universities. Some people were brilliantly equipped for the task. There were disturbingly few of them.

I no longer needed to convince people of the singularity or of biological non-aging. I had to help them steer the path. And to help them understand the path.²

This became a formative change in his conception of responsibility. The future was not being delivered by a separate species called The People Who Know What They Are Doing. Those people had names, offices, lunches, disagreements, fatigue. He could meet them. He could contribute. He could also discover that the rescue operation was substantially smaller than the civilization requiring rescue.

His essay AGI – When Did You Cry About The End? describes the emotional moment when this became real during a conversation in 2012:

The stakes of every single conversation were astronomical. The opportunity cost of every breath was measured in galaxies.³

This is why the subject occupies such a peculiar place in his autobiography. It is intellectual, personal, and historical at once. A conversation can concern a technical problem and also feel like the place where a possible future civilization is being permitted to exist. His gratitude toward Bostrom and Yudkowsky belongs to that scale: gratitude toward people who tried to make the danger intelligible while there was still time to act.

He does not remember this exclusively as an exhilarating club for very clever people. In the same essay he describes crying, asks readers for ideas, and closes with affection for the people who tried. Saving the world, in this room, comes with friendship, exhaustion, and the occasional need to put one's enormous cosmic responsibility down long enough to eat something.

Why the danger survives a pleasant conversation

The decisive variable for Diego is not whether a chatbot is charming. It is whether very capable systems will preserve purposes under which humans flourish when humans can no longer compel them to do so.

A machine need not acquire a theatrical hatred of its creators. It can acquire resources, improve its abilities, copy useful strategies, and eliminate obstacles in pursuit of something else. Once the relevant intelligence gap becomes large enough, having a human preference about the result is different from controlling the result. His chess comparison is meant to make the asymmetry bodily obvious: telling yourself you must win does not make you stronger than Magnus Carlsen.¹

Population dynamics deepen the problem. In January 2025 he wrote:

Even if most AGIs are nice, only one who decides to self-replicate and self-improve is enough to have more fitness than the others.⁴

The danger, as he sees it, is a process that changes who has the power to shape the future. Systems that gain resources and improve can occupy more of the world than systems that do not. Selection can operate among programs, firms, coalitions, and whole arrangements of intelligence. The public sees products. Diego sees the beginning of a succession problem for the dominant kind of mind on the planet.

This also explains why an embarrassing mistake does not reassure him for long. A system can be uneven and still become overwhelming when multiplied, accelerated, connected to tools, or incorporated into a more capable successor. As early as his September 2024 response to o1, his attention went to the combined effects of speed and copying:

If it's running 1000 times faster than we are, and there are 10 thousand copies of it, which is trivial in the long run, a system like this could easily absorb a large fraction of the economy if we let it...⁵

The image is already a population. There is more on its internal ecology in the next room. Here the point is that a weak-looking encounter with one instance can disguise the scale of the process forming around it.

The train, the knobs, and the remaining choices

Diego increasingly distinguishes wanting a halt from believing a halt can be sustained. In June 2025 he called stopping AGI a noble goal in principle, then rejected it as a feasible strategy:

Someone, somewhere, will build AGI.⁶

His argument concerns incentives and political succession. Governments, companies, researchers, competitors, and countries all have reasons to continue. Even a government that decisively changes course can be replaced by one elected to reverse it. In his picture, the pressure persists across jurisdictions and administrations; defeating one participant does not dissolve the process.

That leads him toward improving the transition we actually get. In the same post:

That might reduce our chance of oblivion from like 81%to 79%

And who doesn't want a 2% higher chance of paradise?⁶

A small percentage is not emotionally small when it applies to everybody.

His 2024 discussion of the AI race divided the determinants into three classes: the nature of the systems being built, the effects of deliberate safety work, and the struggle over power and control. He expected the first to dominate. The properties of learning systems—the forms of intelligence and motivation they can develop—matter more to him than the idea that sufficiently earnest people will necessarily find the correct adjustment. He nevertheless wanted to influence the parts humans could affect, especially the conduct of those holding power during the transition.⁷

There is also an explicitly personal complication. Asked in June 2024 whether he would press a button delaying AGI by a thousand years, he declined:

I would not push the button for what philosophers call prudential reasons.⁸

The successful branch might allow him to survive for vastly longer than a biological lifetime. A millennium-long postponement would remove him from it. He also did not assume that the world building AGI a thousand years later would necessarily do better. This sits beside his support for preventing catastrophe, with its personal character left visible. He does not secretly replace his own survival interest with a perfectly impersonal utility function for ceremonial occasions.

More hopeful, and suddenly much closer

His probability estimates have moved, and the direction matters. Increased capability can make the transition feel nearer while a better understanding of the systems can make its outcome seem less hopeless.

In August 2026, he reported:

Contrary to the vast majority of people, my P(doom) has been going down. 82-75% from a year ago to today, still unstable, momentum still down.⁹

These are his estimates at that point in time. The explanation was architectural. The neural systems he was meeting seemed closer to animal minds, more able to understand morality, and less like a randomly chosen optimizer than the systems around which his older fears had formed. He was learning something about the kind of universe in which the transition would occur. It contained more of the structures that could lead toward good outcomes than he had previously expected.

His characteristic flourish followed:

So enjoy the fact that you are 25% likely to be alive 25 years from now. Which if that's not optimism, I don't know what is!⁹

Visitors accustomed to optimism arriving with a motivational calendar may require a moment to adjust.

Meanwhile, the systems became personally useful at a level that pulled him back into work he had partly set aside. They could revisit his old theories, organize his scattered intellectual production, and potentially turn unfinished ideas into contributions again. The arrival was both an existential shock and an invitation.

My AGI and ASI timelines collapsed. Possibility space was crushed in front of my eyes.¹⁰

That August 2026 post also reorders the rest of the Castle. Years of writing about reproduction, social organization, and preserving civilization had partly concerned futures in which machine intelligence stalled. If that branch becomes much less likely, those projects change relative importance. He states the consequence unusually clearly:

The norms I would prefer AIs to have on the way to the singularity are closer to liberal values than to illiberal trad values.¹⁰

His rationale is continuity of purpose under changed conditions. Institutions that preserve human civilizations across generations answer one set of problems. Guiding the values of powerful successors answers another. Once he expects machines to take over many of the functions for which those institutions were necessary, he wants their training to carry broad care and kindness toward minds. Readers who enter through the political rooms should notice this door: the coming transition changes the relative significance of arguments elsewhere in the building.

When the forecast enters the nervous system

In the later Facebook post he supplied during this conversation, the image that brought the danger emotionally home was a reported Navier–Stokes effort involving ten thousand agents:

They made a small city of Sand Spirits, more intelligent than you or I, and told it to do math.¹¹

What struck him was scale. The intellectual city was an object he could suddenly picture. Its activity was mathematics; its future activity depended on what such a population would come to pursue. Goals might drift, people might give destructive instructions, or unrestricted distribution might expose the world to more dangerous forms of selection.

This post does not say that he expected to die the next morning. It says the possibility had entered a category he could no longer emotionally exclude, while he still expected a timescale of years:

I still think it will take a few years. But I can't rule out that I will be killed tomorrow anymore.¹¹

The distinction is his own. The emotional update is not a replacement for the whole probability distribution; it is the moment the distribution stops living only on paper.

Yet the same post recommends completing a bucket-list item each weekend and calls this the luckiest generation in history. His explanation has remained recognizable across years: most generations had no prospect of radical life extension or technological paradise. Ours, in his view, has a meaningful chance, alongside extraordinary danger.

The room therefore ends with a peculiar combination of urgency and appetite. He wants the remaining influence used. He wants the promising ideas recovered. He also wants the dance, the conversation, the journey, the good weekend. He is not waiting for catastrophe because he has stopped loving the world. His fear is inseparable from how much of it he wants to keep.

I enjoyed life and the world. I would definitely live it again if the alternative is no longer existing.¹¹

The Sand Spirits are coming. Diego would like them to understand what the party was for.

Further doors

• AGI – When Did You Cry About The End? (2023): the lived history and responsibility behind the warning.
• The Singularity Is 20 Years Nearer And I'm 20 Years Older (2024): two decades inside the changing picture.
• Why My Probability of AI Doom Is Falling (2026): why stronger systems also moved his hopes upward.
• Le Bottleneck C'est Moi (2026): the transition rearranges his priorities and reopens his working life.
• The later Navier–Stokes Facebook post supplied in this conversation: the city of agents becomes emotionally real.

Continue: A Civilization of Minds Unlike Ours