Anca Dragan: A professor at UC Berkeley working on human-robot interaction and reward engineering algorithms, who also consults at Waymo. She studies how robots can generate behavior that accounts for coordinating with people.
What this episode covers
Anca Dragan explains her work on human-robot interaction, where the robot's job is to optimize for what people actually want rather than what a programmer literally specified. She argues that humans who look irrational may simply be operating under different assumptions or simpler internal models, and shows how robots can use their own actions to gather information about human intent. The conversation covers inverse reinforcement learning, the difficulty of designing reward functions, autonomous driving as a game-theoretic problem with humans, and semi-autonomous driving's risks. It closes on mortality, the meaning of life, and how finiteness might belong in our reward functions.
Recommended on this episode
WALL-E
Pixar
“my favorite fictional robot is Wally and I love how amazingly expressive it is some personal things a little bit about expressive motion”— guest
Artificial Intelligence: A Modern Approach
Stuart Russell and Peter Norvig
“I got my hands on a PDF copy in Romania of Russell Norvig a I modern approach... it was so captivating”— guest
Also referenced (named, not recommended)
Cozmo
Anki
“there was a a key had a robot called Cosmo where they put in some of these animations that part is easy”— guest
Lunar Lander
Atari
“I don't know if you know this Atari game it's called lunar lander it's it's really hard people really suck at landing”— guest
Cadillac Super Cruise
Cadillac
“Cadillac super guru system which has a driver facing camera that detects your state there's a bunch of basically Lane centering systems”— Lex Fridman
The Matrix
The Wachowskis
“I didn't know anything about AI at that point I was you know I had watched the movie The Matrix”— guest
The Good Place
Michael Schur
“I watch this show it's very still it's called a good place and they reflect a lot on this”— guest
Big reveals from this episode
- Anca reveals her husband proposed to her by building a seven-degree-of-freedom WALL-E robot that opened a Lego box.
- Robots can deliberately nudge forward to probe a human driver's aggressiveness and update their model based on the reaction.
- By modeling a human's intuitive (wrong) physics model, her team got people to actually land the Lunar Lander game.
- She claims if you removed all humans from downtown San Francisco, autonomous driving would essentially be a solved problem.
- She calls it irresponsible not to use lidar, while sympathizing with Musk's view of lidar as a crutch.
- Lex pushes back on human-factors orthodoxy, saying drivers can be more energized as observers in some semi-autonomous setups.
- The very state of the world (e.g. shoes lined up on the floor) leaks information about human preferences to a robot.
Worth remembering
- Inverse reinforcement learning infers what reward function a human's behavior is optimal with respect to.
- Boltzmann rationality models humans as choosing options stochastically in proportion to their utility.
- She frames human-robot interaction as an 'under-actuated' system where you influence but cannot directly control people.
- Anca's hobby is watching hundreds of hours of pedestrian video to learn about human behavior.
- 'Civil inattention': if you avoid eye contact while running, people move out of your way.
- Goodhart's law: once a metric becomes a target, it stops being a good metric.
- Robots should interpret specified rewards as evidence of intent, not as literal universal laws.
- Anca says we love existing so much precisely because it ends, and that finiteness should inform reward functions.