George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God | Lex Fridman Podcast #387

In his third appearance, George Hotz argues that AI will likely destroy society as we know it but not the human species, since machines still can't self-replicate or survive in nature the way biology can. He makes a sustained case that open-sourcing AI and decentralizing compute is the only defense against a small group monopolizing intelligence, mocking centralized 'AI safety' as both hype and a path to the very dangers it fears. The conversation tours tiny corp's tinygrad framework and the $15K tinybox, the state of comma.ai self-driving ('Drive GPT'), his stint refactoring Twitter, and his philosophy that complexity and centralization are the two great evils. It closes on programming taste, mypy, VR/AR, video games, his disdain for effective altruism and UBI, faith in God, and his stated meaning of life: to win.

• Agrees with Yudkowsky that AI will likely kill everyone, but for a different reason: it's humans using AI, not AI itself.
  00:13:59
• Claims AI will kill 'everything we call society' but NOT the human species, because robots can't self-replicate like biology.
  00:17:04
• Announces his third company will be AI girlfriends, after comma and tiny corp.
  00:32:37
• Reveals AMD's 7900 XTX kernel drivers panic when he simply runs demo apps in a loop, and that Lisa Su replied to his email.
  00:53:28
• Says AI safety people will build the exact centralized thing they fear; open source is the only way to avoid being 'paperclipped'.
  01:31:51
• Claims GPT-4 is 220 billion parameters and a 16-way mixture model with eight sets of weights.
  01:32:00
• Says he could lead a Twitter refactor in 6-12 months that would make feature development 10x faster.
  02:18:43
• Says atheism is 'silly' because creators like game developers make worlds; of course a more powerful creator made ours.
  02:39:46
• Calls effective altruism 'a terribly evil ideology' and utilitarianism 'abhorrent', linking it to SBF.
  02:57:39
• States the meaning of life is 'to win' and the ultimate game is 'to stand eye to eye with God'.
  03:06:28

• Borrows Jim Keller's framing of three computation paradigms: CPUs (compare/branch), GPUs (arbitrary load/store), and DSPs (static add/multiply), and notes 95% of neural nets are the DSP paradigm.
  00:42:27
• tinygrad has only ~25 operations versus ~250 in PrimTorch and ~2000 kernels in regular PyTorch, described as the RISC of ML stacks.
  00:46:07
• The tinybox: ~1 petaflop of compute, 100+ GB GPU RAM, 5+ TB/s memory bandwidth, six GPUs, designed to run on a single 120V 15A wall outlet for $15,000.
  00:56:36
• A wall outlet supplies ~1500 watts; an electric-car charger supplies ~10,000 watts, his test for max power 'without arousing suspicion'.
  00:57:38
• comma's 'Drive GPT' trains in simulation and uses RL on the reward of 'would a human disengage?' rather than copying human policy directly.
  01:11:43
• Hotz was Time's Person of the Year in 2006 (the 'You' issue).
  01:46:31
• Ranks code-base quality: Facebook best, then Google, then Twitter, using their ML frameworks (PyTorch, TensorFlow, none) as a proxy.
  02:03:44
• Insists 50 people could build and maintain Twitter; got heavy hate for the claim.
  02:04:15
• Repeatedly checks his disagreements against the fact that 'Elon is the richest man in the world' as a humility heuristic.
  02:19:14
• Pays ~20 cents/kWh for electricity in San Diego, roughly the same real price as 1980, and would trade the oligarchy for cheaper energy.
  02:22:50