Max Tegmark: MIT physicist and AI researcher, co-founder of the Future of Life Institute, and author of Life 3.0. He spearheaded the open letter calling for a six-month pause on training AI systems more powerful than GPT-4.
What this episode covers
Returning as the very first guest in podcast history, Max Tegmark argues that AI capability progress has dramatically outpaced AI safety and policy work, leaving humanity racing toward a cliff. He frames the core enemy as 'Moloch', a game-theory force that traps even well-meaning companies into a reckless race to the bottom. The conversation covers why he believes superintelligence is a 'suicide race' nobody wins, how GPT-4 already shows the dangerous capabilities safety researchers warned against (writing code, internet access, manipulating humans), and his open letter calling for a coordinated pause. Tegmark also explores consciousness versus intelligence, a provable-safety approach to AI ('virus checking in reverse'), the parallels between runaway capitalism and runaway AI, and the terrifying realities of nuclear winter. Throughout he insists the situation is serious but not hopeless if humanity slows down enough to get safety right.
Recommended on this episode
Don't Look Up
Adam McKay
“if you're watching this you haven't seen it watch it because we are actually acting out it's it's life imitating art”— Max Tegmark
Needful Things
Stephen King
“have you seen the movie Needful Things it's a Stephen King novel I love Stephen King and Max fonseedov Swedish actors playing the guys it's brilliant”— Max Tegmark
The guest's own work
Life 3.0: Being Human in the Age of Artificial Intelligence
Max Tegmark
“allow me to briefly look at the book which at this point is becoming more and more Visionary that you've written I guess over five years ago life 3.0”— Lex Fridman
Also referenced (named, not recommended)
Meditations on Moloch
Scott Alexander
“have you read meditations on malloc by Scott Alexander yes yeah it's a beautiful essay on this poem by Ginsburg”— Max Tegmark
Human Compatible: Artificial Intelligence and the Problem of Control
Stuart Russell
“even though Stuart Russell has written a book and given a lot of talks on why it's a bad idea to have ai just blindly optimize something”— Max Tegmark
Thinking, Fast and Slow
Daniel Kahneman
“don't let it read Daniel kahneman's book about cognitive biases and all that and then oops lol you know let's invent social media”— Max Tegmark
Men in Black
Barry Sonnenfeld
“you don't have to feel guilty about turning off gpt4 and wiping its memory whenever a new user comes along I wouldn't like if someone did that to me neutralized me like in Men In Black”— Max Tegmark
Big reveals from this episode
- Tegmark reveals he lost both his parents since his last appearance and gets emotional discussing how their values live on as information patterns.
- Compares the current AI moment to the film Don't Look Up, except the asteroid is one we are building ourselves.
- Argues a sudden architectural 'hack' could make a model 10x smarter on any random Tuesday, not just steady compute scaling.
- Reframes the AI race as a 'suicide race' where everyone loses if anyone's AI goes out of control, not an arms race with a winner.
- Notes safety researchers' three biggest fears, teach AI to code, connect it to the internet, teach it to manipulate humans, have all already happened ('oops, lol').
- Contends even China would prefer a longer pause because authoritarian governments fear losing control even more than the West does.
- On whether GPT-4 is conscious, Tegmark says he doesn't know, raising the nightmare of a 'zombie apocalypse' of intelligent but non-conscious machines.
- Cites a Nature Food nuclear-winter study estimating ~99% of Americans and ~98-99% in Russia/China/Europe would starve to death.
Worth remembering
- The open letter had been signed by over 50,000 people, including 1,800 CEOs and 1,500+ professors (Bengio, Russell, Musk, Wozniak, Harari).
- Life 1.0 (bacteria) can't learn in its lifetime; Life 2.0 (us) can learn software; Life 3.0 can redesign its own hardware too.
- Birds took 100 years longer to understand than the Wright brothers' planes, evolution picked a harder path; LLMs are the 'easy way' to intelligence.
- Researchers edited an LLM so it believed the Eiffel Tower is in Rome, then it gave directions to it via Roma Termini station.
- Beauty filters on social media are 'Moloch in action', everyone adopts them, nobody gains market share, but no one can go back.
- Human cloning was effectively paused worldwide; the one Chinese scientist who did it was jailed by the Chinese government itself.
- A paper by Dylan Hadfield-Menell proved mathematically that relentlessly optimizing a single goal eventually makes things worse, like aiming south and overshooting Austin into deep space.
- Per Giulio Tononi's theory, consciousness requires information loops; GPT-4's one-way (feed-forward) flow could make it an intelligent 'zombie.'
- Switzerland banned boiling live lobsters after research showed they feel pain, Tegmark uses it as a warning against self-serving denials of others' consciousness.
- Closes with Frank Herbert: 'History is a constant race between invention and catastrophe.'