Sacha Arnoud: Director of Engineering and head of perception at Waymo, formerly on Google's Street View team leading the Street Smart deep-learning project.
What this episode covers
Sacha Arnoud, Waymo's director of engineering and perception lead, gives an MIT lecture tracing self-driving from Google's 2009 Chauffeur project to Waymo's driverless milestone of removing safety drivers in late 2017. He explains the rise of deep learning inside Google, from Street View house-number recognition to real-time embedded perception on the cars. He details perception techniques: sensor fusion across cameras, radar, and lidar; segmentation; single-shot detection; embeddings; and recurrent networks for behavior prediction. He emphasizes that algorithms are only part of the work, with labeling, compute, TensorFlow infrastructure, and a three-pronged testing program of real driving, simulation, and a structured test facility. He closes with future directions around expanding operating domains and deeper semantic understanding.
Also referenced (named, not recommended)
Chrysler Pacifica
Chrysler
“so it is based on on the chrysler pacifica you can already see a bunch of sensors”— Sacha Arnoud
Big reveals from this episode
- Waymo removed the safety driver entirely for the first time in November 2017, a major confidence and maturity milestone.
- Waymo has been continuously operating fully driverless cars in the Phoenix/Chandler, Arizona area since that event.
- Waymo completed ten challenging 1,000-mile loops across California 100% autonomously back in 2010.
- The first deep-learning application to succeed in production across Alphabet was Street View house-number recognition in 2012.
- Waymo crossed three million autonomous miles in May 2017 and four million by November 2017.
- Waymo drove 2.5 billion simulated miles in data centers in a single year, three orders of magnitude beyond real driving.
- Waymo runs a simulated fleet of 25,000 cars 24/7 in Google data centers.
Worth remembering
- 94% of US crashes today involve human error, much of it avoidable distraction.
- Arnoud's mantra: when you are 90% done you still have 90% to go, requiring 10x improvements across team, sensors, and quality.
- The word2vec embedding space famously yields king minus man plus woman approximately equals queen.
- Waymo's four million driven miles represent roughly 300 years of human driving equivalent, about 60 trips around the globe.
- Waymo built a 90-acre structured test facility on a former Air Force base in California with traffic lights and railroad crossings.
- Exhaust smoke can trigger false lidar laser points, which the system must learn to safely ignore.
- A car reflected in a bus can fool naive detection into seeing two cars, requiring semantic understanding to discard.
- A mannequin-like figure lying in a pickup truck bed shows why the car must understand a 'pedestrian' traveling with a vehicle is not on the road.