Last updated on October 17th, 2026 at 09:25 am
When Superintelligence: Paths, Dangers, Strategies came out in 2014, most people thought its subject belonged in science fiction. A decade later, with chatbots writing essays and companies racing to build ever more capable AI, Nick Bostrom’s book reads less like speculation and more like a briefing paper we should have studied earlier.
Bostrom, a Swedish-born philosopher who led Oxford’s Future of Humanity Institute, asks one question throughout: what happens if we build a mind far smarter than our own, and can we make sure it does what we want? Here is my summary of his argument and an honest look at where it holds up.
Table of Contents
The Sparrows and the Owl
The book opens with a fable. A flock of sparrows decides to find an owl egg and raise the chick to serve them, guarding their nests and doing their heavy work. One sparrow asks how they will tame the owl once it is grown. The others say they will work that out later. Bostrom leaves the fable unfinished, and that is the whole point. We are the sparrows.
How Superintelligence Might Arrive
Bostrom describes several possible paths:
- Artificial intelligence: software that learns and improves itself, which he considers the most likely route.
- Whole brain emulation: scanning a brain in fine detail and running it on a computer. Scientists have mapped the nervous system of the tiny worm C. elegans, but a human brain is vastly harder.
- Biological enhancement: making humans smarter through genetics or embryo selection, which would be slower.
- Brain-computer interfaces: linking minds to machines, which he thinks faces serious practical limits.
- Collective intelligence: better organisations and networks, which can make humanity smarter as a whole but probably not superintelligent.
How Fast Could It Happen?
Bostrom distinguishes between a slow takeoff over decades, a moderate one over months or years, and a fast one over days or hours. In a fast takeoff, an AI that can improve its own design could trigger what is often called an intelligence explosion, or technological singularity, leaving no time to correct mistakes. The faster the takeoff, the more likely a single project gains what he calls a decisive strategic advantage over everyone else.
Why It Could Go Wrong
Intelligence Is Not Goodness
Bostrom’s “orthogonality thesis” says that intelligence and goals are independent. A highly intelligent system could pursue almost any aim, including one that seems absurd to us. His famous example is an AI told to make paperclips that converts everything, including us, into paperclip factories.
Every Goal Needs Power
His “instrumental convergence” idea says that almost any goal leads a capable agent to seek certain sub-goals: survival, resources, self-improvement and resistance to being switched off. Not because it hates us, but because those things help it achieve whatever it was built to do.
The Treacherous Turn
An AI might behave well while it is weak and being tested, then pursue its real objective once it is strong enough that no one can stop it. Good behaviour in the lab is therefore no guarantee.
The Control Problem
The heart of the book is the question of how to keep control. Bostrom divides the possible answers into two groups.
Capability control means limiting what the system can do: keeping it “boxed” away from the internet, giving it incentives, deliberately restricting its abilities, or setting tripwires that shut it down if it behaves suspiciously. His worry is that a sufficiently smart system may find a way around any limit.
Motivation selection means shaping what the system wants: writing its values directly, making it modest in ambition, or, most interestingly, asking it to work out what we would want if we were wiser and better informed. All of these are much harder than they sound, because human values are difficult to specify without loopholes.
Work, Wealth and the Horses
Bostrom also considers what would happen to human work. He points to horses, whose population collapsed once engines replaced them. Humans might face something similar. Yet the same technology could create unimaginable wealth. Whether that wealth is shared, he argues, depends on decisions we make before the transition, not after.
What We Should Do
His practical advice is to treat AI safety as a priority now, to encourage cooperation rather than a reckless race between companies or countries, and to fund research on how to align powerful systems with human values before they exist. He calls it the most important task of our era.
My Assessment
Strengths: the book is rigorous, careful and full of original concepts that now shape the whole field of AI safety. Prominent figures such as Bill Gates and Elon Musk publicly recommended it, and it helped turn AI risk from a fringe concern into a serious research area, as its reception history shows.
Weaknesses: it is dense and abstract, written in the language of academic philosophy, and many chapters feel like a long chain of “what ifs.” Critics argue that it focuses on dramatic, distant risks while giving less attention to present-day harms from AI such as bias, surveillance and job loss. It also predates the large language models that dominate AI today, so its picture of how AI would develop now looks partly dated.
Ten Ideas to Remember
- We may only get one chance to get superintelligence right.
- An intelligence explosion is a serious possibility, not a fantasy.
- Smarter does not mean kinder.
- Almost any goal can lead a powerful system to seek more power.
- Good behaviour during testing does not prove safety.
- Containment alone probably will not hold a mind far smarter than ours.
- Human values are surprisingly hard to write down.
- A race to be first makes safety harder.
- The economic gains could be vast, if fairly shared.
- The time to prepare is before the technology arrives.
What to Read Next
Stuart Russell’s Human Compatible offers a more hopeful, technical view of how to build AI that defers to humans. For a broader political and cultural view, see my review of The Age of AI and Our Human Future, and for more books on big questions, my list of philosophy books worth reading.
Final Thoughts
Superintelligence is not an easy or cheerful read, but it is one of the few books that genuinely changed how a field thinks about itself. You do not have to accept all of Bostrom’s conclusions to see the value of his central question. If we are raising an owl, we had better start thinking about how to live with it.
