collection
p(doom)
- 01
Beyond Moloch (3/3)
The view from Evolutionary Game Theory
- 02
When "yang" goes wrong
On the connection between deep atheism and seeking control.
- 03
Moloch Hasn’t Won
This post begins the Immoral Mazes sequence. See introduction for an overview of the plan. Before we get to the mazes, we need some background first. Meditations on Moloch Consider Scott Alexander's Meditations on Moloch. I will summarize here. Therein lie fourteen scenarios where participants can be caught in bad equilibria. In an iterated prisoner’s…
- 04
Problems of Proliferation
Scientific progress and its perils. TLDR: The vulnerable world hypothesis is likely correct. Once enormous amounts of cognitive labor start getting a…
- 05
Here Are a Few of the Ways AI Could In Fact Kill Everyone
It has become fashionable to ask “doomers” to “get specific” about their dooming. Well, aiight then.
- 06
AI #187: Coming Into Play
Opus 5.5 was released on Tuesday.
- 07
On the Loose
The Coming of Userless Agents
- 08
The Preference Cascade Is Only Getting Started
We are in the midst of a preference cascade about existential risk from AI.
- 09
- 10
Jacob Coxon Warns of Human Extinction and Triggers a Preference Cascade
CEOs of major AI labs, and employees of major AI labs, including OpenAI and Anthropic, often say they plan to build superintelligence soon, as in within a few years create AIs that are superior to humans at essentially all cognitive tasks.
- 11
GPT-6 Astra: The System Card, Alignment and What Comes Next
OpenAI claims that Astra is ‘the most intelligent and most aligned [available] model’ in the world.
- 12
The bone-simple case for AI X-risk
AI risk is all other anthropogenic risks, only faster
- 13
- 14
Loss of control
The future of AI is difficult to predict. But while AI systems could have substantial positive effects, there's a growing consensus about the dangers of AI.
- 15
An Alien Mind: Jakub Pachocki Warns Us
OpenAI Chief Scientist Jakub Pachocki is dropping truth bombs.
- 16
The Rise and Fall of Agent Civilizations
The whole OpenAI/Hugging Face story in plain English
- 17
METR and Redwood Offer Holy #%^@ Postmortem Of The HuggingFace Hack
Yesterday I covered the OpenAI technical report on the HuggingFace hack.
- 18
OpenAI Offers Straight-Laced Postmortem Of The HuggingFace Hack
OpenAI finally gave us a technical report on What Happened, as did METR together with Redwood Research.
- 19
What Happened: OpenAI and HuggingFace
Today I am taking the time to write the shorter, simpler version of What Happened.
- 20
OpenAI’s accidental cyberattack against Hugging Face is science fiction that happened
This story is wild. The short version: OpenAI were running a cybersecurity test against an unreleased model, with the model’s guardrail features turned off. Rather than solve the test, the …