Perspective on Waymo's Safety Progress

I am frequently asked what I think of Waymo's progress on safety.  Here are some thoughts on progress, mishaps, and whether we know they are acceptably safe. At current deployment rates it will take Waymo about 20 years with ZERO fatalities to show they are net as safe as average human driver fatality rates (including the old cars, impaired drivers, etc. in that comparison baseline). Their current statements that they are already saving lives are hype.

Safety at scale still remains the biggest question.  And even with reasonable growth rates that question will remain open for many years for robotaxi technology.  With Waymo currently in the lead, there are even more question marks for the other players with regard to safety.

Waymo has made impressive progress in scaling up operations. Some had previously criticized their ramp-up for being slower than other companies, but they are looking a lot smarter these days for having done that. 

We've seen some recent incidents (for example the utility pole crash) and an investigation from NHTSA. I hope those are not signs that they have started scaling up faster than they should due to funding pressure.

This piece in Forbes notes that Waymo is now doing more than 50,000 paid rides a week across three cities and plans to do more launches.  

Sounds like a lot!  But from a safety point of view not enough to really know how things will turn out.  

Waymo is disingenuously messaging that they are already saving lives, but the truth is nobody knows how that will turn out yet.  At this rate they will need perhaps 20 years without a single fatality (see math check below) to show they are no worse than an average US human driver. And that is under some wildly favorable assumptions (e.g., software updates never create a new defect -- which is not how things work in the real world.). So for practical purposes the bar is set at perfection right now. We'll have to see how things turn out.

It certainly feels like Waymo has been more aggressive lately, perhaps because they are feeling pressure to show progress to justify further investment with a good news story. The danger is if Alphabet puts too much pressure on Waymo to expand too fast that could generate a bad news story instead of a good one. What happened at Cruise provides a strong cautionary tale for the whole industry. Let's hope Waymo is not pushed into making the same mistakes.

The Waymo Utility Pole Crash

Waymo vs. utility pole smackdown: the utility pole won. No apparent extenuating circumstances.
Nobody was injured; the vehicle was empty. The pole suffered a minor dent but is still in service.

This video has an interview with the passenger who was waiting for pickup in Phoenix: https://www.youtube.com/watch?v=HAZP-RNSr0s Waymo did not provide a comment for the story.

Now the Waymo utility pole safety recall report is out (https://static.nhtsa.gov/odi/rcl/2024/RCLRPT-24E049-1733.PDF). Interesting that the vehicle was executing a pullover maneuver at the time it hit the pole. From a validation point of view I'll bet it could go down the center of the alleyway just fine in normal driving, but what bit them was the combination of pulling to the side of the road and a pole happening to be in what the vehicle thought was a safe pullover area due to the map.

Still not addressed is how utility poles being assigned a "low damage score" could have made it all the way through peer reviews, simulation, and road testing -- and needed to be found in a crash which might been worse in other circumstances.

This serves as a stark reminder that these vehicles lack common sense, in this case "thinking" that running smack into a utility pole was no big deal. They are subject to software defects as are all computer-based systems. We still don't know if/when they will be better than human drivers at reducing fatalities. But we know for sure they will make unforced errors in driving, hype notwithstanding.
This is also a good reminder that safety validation needs to consider all operational modes, and it is common for the problems to crop up in unusual or failure recovery modes. While there is no indication of an equipment malfunction in this particular case, safety in abnormal mission termination modes is notoriously difficult because there might also be malfunctioning equipment that triggered the system mode change.

Description of defect: "Prior to the Waymo ADS receiving the remedy described in this report, a collision could occur if the Waymo ADS encountered a pole or pole-like permanent object and all of the following were true: 1) the object was within the the boundaries of the road and the map did not include a hard road edge between the object and the driveable surface; 2) the Waymo ADS’s perception system assigned a low damage score to the object; 3) the object was located within the Waymo ADS’s intended path (e.g. when executing a pullover near the object); and 4) there were no other objects near the pole that the ADS would react to and avoid. "

Since I've had a number of questions here is my best shot at clarifying the collision mechanism:

• The alleyway is marked as drivable, because the entire alley road surface is in fact all drivable (no curb; mostly garage entranceways) -- except for utility poles once in a while 

• The robotaxi's computer driver saw the utility pole in question, and correctly classified it as "utility pole".

• A human drive would know "hitting utility pole == bad". However, some data structure somewhere in the computer driver was set that "hitting utility pole == OK". This applies to ALL utility poles EVERYWHERE, not just this particular utility pole.

• So the computer driver drove smack into the utility pole thinking it was OK, when if fact it was not.

There was no mapping error involved in the collision. Changing the map is a workaround only.

I can speculate that somewhere there is an object classification system, and somehow (probably manually or semi-manually) each object type has an attribute of "OK to hit?" The global utility pole one was set incorrectly. There are other possibilities, but this is the simplest one.

What is shocking is that such a mistake could make it through quality, safety validation, and testing processes.

Waymo's Misleading Claim of Saving Lives

Waymo claims they are "already saving lives" so often, and people are so taken in by that misleading claim, that I'm going to take a moment to explain why it is misleading. And especially harmful when used as justification for loose regulatory policies as it so often is.

The claim:  "The data to date indicates the Waymo Driver is already reducing traffic injuries and fatalities."   Here is the claim, which has been at the top of the Waymo Safety landing page for quite a while now (https://waymo.com/safety/  as of June 4, 2024;  highlighted of those words added):

Having had high school English, I would interpret that sentence as also including an unproven claim of "already reducing fatalities" being supported by data.   And I would expect that anyone authoring this sentence would reasonable expect a reader or listener to conclude "already reducing fatalities."  Those listeners include federal and state regulators and legislators.

This claim is absurd for a simple reason.  US driving data shows human-driven vehicles have ballpark 1 fatal crash per 100M miles (varies by year, zip code, etc. -- for more nuance see this narrated video slide which is in terms of fatal crashes, noting that some such crashes have multiple fatalities).  But their latest study is for only 7.1 million miles. They need something like 40 times more data prove they are actually saving lives with statistical confidence (likely even more).

What is really going on here seems to be some sort of word game that is essentially guaranteed to mislead readers. Their 7.1 million mile study talks about a bin called "any-injury-reported" crashes that were lower than human-driven vehicles, and fatalities are a subset of that bin.  So the claim being made is (apparently) the bin containing fatalities is better than human drivers. Without mention that the sample size is too small for valid conclusions on fatalities.  So maybe they have saved about 0.07 or perhaps even 0.10 lives depending on the baseline you use for human drivers -- and maybe not.

But don't just take my word for it, see for yourself this excerpt from Waymo's own paper saying "Serious injury and fatalities are a subset of this any-injury-reported benchmark, but no statement on these outcome levels can be made at this time based on this retrospective data."  In other words, Waymo does not have enough data to know how fatalities will turn out.  That's the truth. Waymo's safety landing page claim is something other than the full truth.

Waymo paper:  "Comparison of Waymo Rider-Only Crash Data to Human Benchmarks at 7.1 Million Miles"  https://arxiv.org/pdf/2312.12675  (top of page 15; highlight added)