Nick Bostrom: Philosopher at the University of Oxford and director of the Future of Humanity Institute, known for work on existential risk, the simulation argument, and his book Superintelligence.
What this episode covers
Lex Fridman talks with Nick Bostrom about the simulation hypothesis and its underlying three-part argument: civilizations go extinct before technological maturity, mature civilizations lose interest in running ancestor simulations, or we are almost certainly living in a simulation. They explore consciousness, the Nozick experience machine, anthropic reasoning, and the Doomsday argument. The conversation then turns to superintelligence, its enormous upside as a general-purpose technology, the risks of misalignment, the possibility of an intelligence explosion, and what a posthuman utopia and the meaning of life might look like. Throughout, Bostrom stresses how little humanity understands and the need for a proactive approach to existential risk.
The guest's own work
Superintelligence
Nick Bostrom
“he has done some incredible work in artificial intelligence in technology including in his book super intelligence”— Lex Fridman
Superintelligence
Nick Bostrom
“when I wrote the book super intelligence at that point I felt that was a kind of neglect of what would happen if AI succeeds”— Nick Bostrom
Also referenced (named, not recommended)
Pong
Atari
“I mean if you think say when people first had the pong or something like that like I'm sure there were people who wanted to keep playing it”— Nick Bostrom
StarCraft
Blizzard Entertainment
“so they you can play pong or Starcraft or would have more sophisticated games for hours for four months”— Nick Bostrom
Warcraft
Blizzard Entertainment
“Wow well the Warcraft could be in a big addiction but eventually they escape that”— Nick Bostrom
AlphaZero
DeepMind
“alpha zero it's somehow much more intelligent much much more intelligent than deep blue”— Nick Bostrom
Deep Blue
IBM
“alpha zero it's somehow much more intelligent much much more intelligent than deep blue”— Nick Bostrom
Big reveals from this episode
- Bostrom distinguishes the simulation hypothesis from the simulation argument, which states one of three propositions must be true.
- He argues an experience machine giving rich human interactions might necessarily instantiate other genuine consciousnesses as a side effect.
- He introduces the 'bland principle of indifference' to justify why we should think we are probably simulated.
- Bostrom explains the Doomsday argument via the urn analogy and your 'birth rank' of roughly 100 billion.
- He offers an extra reason Elon Musk specifically might suspect he is simulated: unusual, remarkable people may be likelier to be simulated.
- Bostrom stresses the huge positive potential of machine superintelligence, countering his own reputation for doom.
- He places himself as assigning more probability to an intelligence-explosion scenario than the average AI researcher.
Worth remembering
- Eric Drexler's molecular manufacturing work suggests a sugar-cube-sized computer could run at a million times the human brain.
- A simulation may only render scenes within view, leaving unobserved details coarse-grained, like procedural virtual reality.
- Our own three-pound brains effortlessly produce immersive dreams we don't realize are fake, hinting how easy realistic simulation could be.
- The Doomsday argument uses the same Bayesian updating as drawing a low-numbered ball from an unknown urn.
- As the fraction of simulated observers approaches one, the probability that you are simulated approaches certainty.
- Stacked simulations form a tower limited by the finite compute of 'basement reality'.
- Bostrom compares our grasp of reality to a dog trying to understand what it is to be human.
- He notes current AI systems are superhuman in narrow domains yet radically subhuman everywhere else.
- Bostrom finds AlphaZero intuitively far more intelligent than Deep Blue because of self-play learning.
- He argues a posthuman abundance could let us score ~98% well on many competing value systems at once.