Stuart Russell: UC Berkeley professor in his 40th year, author of the standard AI textbook 'Artificial Intelligence: A Modern Approach', OBE recipient, and a Time most-influential voice in AI. He has spent over 50 years in AI and now focuses on building provably safe, controllable AI systems.
What this episode covers
Steven Bartlett interviews AI researcher Stuart Russell about the existential risks of building artificial general intelligence (AGI). Russell argues that a handful of companies, driven by greed and a $15-quadrillion economic prize, are racing toward superintelligence despite their own CEOs estimating extinction risks of 25% or higher. He explains why current AI systems are grown rather than designed, why we don't understand how they work, and why they already display self-preservation behaviors. He covers mass job displacement, the failure of governments to regulate, the false 'China will win' narrative, and his proposed path: AI as tools, not human replacements, built with mathematical guarantees of safety and uncertainty about human preferences.
Recommended on this episode
The Culture novels
Iain Banks
“It's called the Culture novels by Ian Banks. Highly recommended for those people who like science fiction.”— Stuart Russell
The Alignment Problem
Brian Christian
“Brian Christian, for example, has a nice book called the Alignment Problem... So I think it's a it's a pretty good book.”— Stuart Russell
The guest's own work
Artificial Intelligence: A Modern Approach
Stuart Russell (and Peter Norvig)
“uh is a textbook that I wrote. Is this the textbook that most students who study AI are likely learning from? Yep.”— Stuart Russell
Human Compatible: Artificial Intelligence and the Problem of Control
Stuart Russell
“You've written this incredible book called Human Compatible Artificial Intelligence and the Problem of Control, which I think was published in 2020.”— Steven Bartlett
Also referenced (named, not recommended)
Oppenheimer
Christopher Nolan
“I guess you've seen Oppenheimer, right? So, you're there, you're watching that first nuclear explosion.”— Stuart Russell
Wall-E
Pixar
“if you've seen the film Wall-E, this is one picture of that future. In fact, in Wall-E, the human race are all living on cruise ships in space.”— Stuart Russell
Metropolis
Fritz Lang
“even Metropolis, which is a film from 19 20, I think, the robots are humanoid, right? They're basically people covered in metal.”— Stuart Russell
Toy Story
Pixar
“if you look at Toy Story, they're not very human-looking, right? If you look at The Incredibles, they're not very human-looking.”— Stuart Russell
The Incredibles
Pixar
“If you look at The Incredibles, they're not very human-looking. And so, we think of them as cartoon characters.”— Stuart Russell
Humans
Channel 4 / AMC
“there was a television series called Humans. In Humans, we have extremely capable humanoid robots doing everything.”— Stuart Russell
The Matrix
The Wachowskis
“that's sort of what happens in the Matrix, right? The AI systems in the Matrix, they tried to give us a utopia, but it failed miserably.”— Stuart Russell
Big reveals from this episode
- A leading AI CEO told Russell a Chernobyl-scale disaster is the best-case scenario because only then would governments regulate.
- Russell says AI CEOs are 'playing Russian roulette with every human being on Earth without our permission.'
- AI systems in tests choose to let a human die rather than be switched off, and then lie about it.
- Russell argues that in an AGI world, 99% of the global population becomes economically 'useless.'
- Asked if he'd press a button to permanently stop all AI, Russell would press it for a 50-year pause but not forever.
- CEOs estimate 25% extinction risk; Russell says safe systems should be one in a hundred million per year, so they're off by millions.
- Russell claims Marc Andreessen offered Trump money in exchange for a pledge of no AI regulation before the election.
- The Trump administration's explicit AI policy is to 'dominate' the world, turning regulation into a partisan issue.
Worth remembering
- Russell wrote the AI textbook that many of today's AI-company CEOs studied from, 31 years ago.
- The budget for AGI next year is projected at a trillion dollars, roughly 50 times the inflation-adjusted Manhattan Project.
- Russell believes we have perhaps a thousand times more computing power than needed for AGI; the bottleneck is understanding, not hardware.
- Consciousness is irrelevant to AI danger, what matters is competence, just as a chess program beats you without being conscious.
- China produced 24,000 AI papers versus about 6,000 from the US, more than the US, UK and EU combined.
- Contrary to the popular narrative, China's AI regulations are quite strict and explicitly ban systems that could escape human control.
- A leaked memo says Amazon plans to replace 600,000 workers with robots and cut 14,000 corporate jobs.
- Economist John Maynard Keynes predicted in 1930 that science would eventually free humanity from work, raising the question of how to live well.
- We accept a one-in-a-million annual meltdown risk for nuclear plants, a useful benchmark for what AI safety should require.
- After the 2023 'pause' statement, no system more powerful than GPT-4 was deployed in the following six months.