No, Mercedes-Benz will NOT take the blame for a Drive Pilot crash

Summary: Anyone turning on Mercedes-Benz Level 3 Drive Pilot should presume they will be blamed for any crash, even though journalists are saying Mercedes-Benz will take responsibility.

Drive Pilot usage while playing Tetris. (Source)

There seems to be widespread confusion about who will take the blame if the shiny new Mercedes-Benz Drive Pilot feature is involved in a crash in the US. The media is awash with non-specific claims that amount to "probably Mercedes-Benz will take responsibility."  (See here, here, here, and here)

But the short answer is: it will almost certainly be the human driver taking the initial blame, and they might well be stuck with it -- unless they can pony up serious resources to succeed at a multi-year engineering analysis effort to prove a design defect.

This one gets complicated. So it is understandable that journalists on deadline simply repeat misleading Mercedes-Benz (MB) marketing claims without necessarily understanding the nuances. This is a classic case of "the large print giveth, and the small print taketh away" lawyer phrasing.  The large print in this case is "MERCEDES-BENZ TAKES RESPONSIBILITY" and the small print is "but we're not talking about negligent driving behavior that causes a crash." 

The crux of the matter is that MB takes responsibility for product defect liability (which they have to any way -- they have no choice in the matter). But they are clearly not taking responsibility for tort liability related to a crash (i.e., "blame" and related concepts), which is the question everyone is trying to ask them.

Like I said, it is complicated.

The Crash

Here is a hypothetical scenario to set the stage. Our hero is the proud owner of a Drive Pilot equipped vehicle, and has activated the feature in a responsible way. Now they are merrily playing Tetris on the dashboard, reading e-mail, watching a movie, or otherwise using the MB-installed software according to the driver manual. The car is driving itself, and will notify our hero if it needs help, as befits an SAE Level 3 "self-driving" car feature.

A crash to another vehicle happens ahead in the road, but that other, crashed vehicle is out of the flow of traffic. So our hero's car sees a clear road ahead and does not issue a takeover request to the driver. Our hero is currently engrossed in watching a Netflix movie on the dashboard (using the relevant MB-approved app) full of explosions in an action scene, and does not notice.

Meanwhile, in the real world, a dazed crash victim child wanders out of their wrecked vehicle onto the roadway. But our hero's MB vehicle has a hypothetical design defect in that it can't detect children reliably. 

Our hero's vehicle hits and kills that child. (I emphasize this is a hypothetical, but plausible, defect. The point is that we, and the owner, have no way of knowing if such a defect is present right now. Other crash scenarios will play out in a similar way.)

Our hero then is charged by the police with negligent homicide (or the equivalent charge depending on the jurisdiction) for watching a movie while driving. Additionally a lawsuit for $100M is filed both against the driver and Mercedes-Benz for negligent driving under tort law. The judge determines that Mercedes-Benz did not have a duty of care to other road users under the relevant state law, so the tort lawsuit is changed to be just against the driver.

What happens to our hero next? 

Will MB will step up and pay to settle the $100M lawsuit? Will they also volunteer to go to jail instead of our hero? They have not actually said they will do either of these things, because their idea of "responsibility" is talking about something entirely different.

Tort Law

I will say right here I am not a lawyer. So this is an engineer's understanding of how the law works. But we are doing pretty basic legal stuff here, so probably this is about right. (If you are a journalist, contact law professor William Widen for his take.)

Tort law deals with compensation for a "harm" done to others. Put very simply, every driver has a so-called "duty of care" to other road users to avoid harming them. Failing to exercise reasonable care in a way that proximately causes harm can lead to a driver owing compensation under tort law. Unless there is a clear and unambiguous transfer of duty of care from our hero to the MB Drive Pilot feature, our hero remains on the hook under tort law.

The problem is that the duty of care remains with our hero even after activating Drive Pilot. Pressing a button does not magically make a human driver (who is still required to remain somewhat attentive in the driver seat) somehow not an actual driver under current law.

But, wait, you say. Mercedes says They Take Responsibility!  (This message is currently being splashed across the Internet in breathless reviews of how amazing this technology is. And the technology is truly amazing.)

Well no, not according to tort law they don't take responsibility. Instead, the MB position is that the human driver retains the duty of care for potential harm to other road users when using their Level 3 Drive Pilot system -- even while playing Tetris. Their representative admitted this on stage a couple weeks ago in Vienna Austria. I was in the crowd to hear it. Quoting from Junko Yoshida in that article:  "Yes, you heard it right. Despite its own hype, Mercedes-Benz is saying that the driving responsibility still rests on the human driver in today’s L3 system."

So what does MB mean when they "accept responsibility?"   The answer is -- they are not accepting the responsibility you think they are. In particular, they are not accepting liability that stems from negligent driving behavior.

Product Defect/Liability

A statement from Mercedes-Benz issued to the press makes it clear that MB accepts responsibility for product defects, but not tort law liability.  A key phrase is:  "In the context of Drive Pilot, this means that if a customer uses the system as intended and instructed and the system fails to perform as designed, we stand behind our product."  (Source here.)

This means that in our hypothetical scenario, our hero can hire a lawyer, who then hires a team of engineers to look at the engineering of Drive Pilot to see if it "performs as designed." Expect someone to pay $1M+ for this effort, because this is a full-on, multi-year product defect investigation. Maybe they find a defect. Maybe they don't, even if a defect is really there. Or maybe they find a reproducible defect, and it is so complex the jury doesn't buy the story. And maybe the performance as designed is that sensors will only detect 98% of pedestrians, and the victim in this tragic scenario is just one of that unlucky 2%. 

Perhaps the owner manual says our hero should have been alert to dazed crash victims wandering in the travel lane. Even while playing Tetris.  (That owner manual is not released yet, so we'll just have to wait and see what it says.) In which case MB will blame our hero for being negligent -- as if one is likely to notice a crash while watching the latest war movie with a high end sound system. And what if the owner didn't bother to read the manual and instead just took the dealer and dozens of car journalists saying "MB accepts responsibility" at face value. (OK, well maybe that's a different lawsuit. But we're talking a legal mess here, not a clear acceptance of the duty of care for driving by MB.)

Who is holding the bag here?

It's probably our hero, and not Mercedes-Benz holding the bag. Even though our hero believed the MB marketing that said it was OK to play video games instead of looking out the front window, and believed their claims of superior engineering quality, and so on.

Maybe a product defect can be found. Finding one might influence a jury in a tort law case to blame MB instead of our hero. (But that's not guaranteed. And what if the jury finds a reason to blame both?) Perhaps our hero has a robust umbrella insurance coverage instead of the state minimum that covers such a loss. Perhaps our hero is so broke they are "judgement proof" (no assets to collect -- but owning a Mercedes-Benz makes that less likely I'd think).

In effect, what we'll see is that the human driver will almost certainly be presumed guilty of negligence in a crash if watching the dashboard instead of the road, even if the vehicle operational mode is telling them it is OK to do exactly that. The onus will be upon the driver to prove a design defect as a defense. This is difficult, expensive, time consuming, and overall not an experience I would wish on anyone.

And there is the specter of criminal liability for negligent homicide (or the equivalent). Depending on the state, our hero might be charged criminally based on being the operator or even just the vehicle owner (even if someone else is driving!). It depends. Nobody really knows what will happen until we see a case go through the system. But outcomes so far in Level 2 vehicle automation cases are not encouraging for our hero's prospects.

Driving a Level 3 vehicle is only for the adventurous

Perhaps you believe any claim by MB that their feature will never cause a crash while engaged. Ever. Pinkie swear. But they have had recalls for software defects before, and there is no reason to believe this new, complex software is exempt from defects or malfunctions.

We don't know how this will turn out until a few such crashes make their way through the court system. But there is little doubt that it will be a rough ride for both the drivers and the crash victims after a crash while we see how this sorts out.

Anyone turning on Drive Pilot should presume they will be blamed for any crash, no matter what Mercedes-Benz says. As even MB admits, the human driver retains the duty of care for safety to other road users at all times, just as in a Level 2 "autopilot"-style system. Marketing statements made by MB about giving time back to the driver don't change that.

Who knows, maybe MB will decide to stand behind their product and pay out on the behalf of customers who find themselves embroiled in tort law cases. But they have not said they will do so. And maybe you believe that MB engineers are so good, that this particular software will have no defects AND will never mistakenly cause a crash.  But that's not a set of dice I'm not eager to roll.

This legal ambiguity is completely avoidable by establishing a statutory duty of care for computer drivers that assigns the manufacturer as the responsible party under tort and criminal law. But until that happens, our hero is going to be left holding the bag.

(Things get even worse if you want to dig into the complexities of the handoff process... but that is a story for another post.)

Update --  The MB Drive Pilot US manual is out, and it doubles down on these issues.  Drivers are apparently required to notice "irregularities .. in the traffic situation" -- which means paying attention to the road.  And for "obvious circumstances"

Five questions cities should ask when robotaxis come to town

If your city is getting robotaxis, here are five questions you should be asking yourself if you are in local government or an advocate for local stakeholders. Some of the answers to these questions might not be easy if you are operating under a state-imposed municipal preemption clause, which is quite common. But you should at least think through the situations up front instead of having to react later in crisis mode.

(There might well be benefits to your city. But the robotaxi companies will be plastering them over as much media as they can, so there is no real need to repeat them here. Instead, we're going to consider things that will matter if the rollout does not go ideally, informed by lessons San Francisco has learned the hard way.)

(1) How will you know that robotaxis are causing a disruption?

In San Francisco there has been a lot of concern about disruption of fire trucks, emergency response scenes, blocked traffic, and so on. The traffic blockages show up pretty quickly in other cities as well. While companies say "stopping the robotaxi is for safety" that is only half the story. A robotaxi that stays stopped for tens of minutes causes other safety issues, such as blocking emergency responders, as well as disrupts traffic flow.

How will you know this is happening? Do you plan to proactively collect data from emergency responders and traffic monitoring? Or wait for twitter blow-ups and irate citizens to start coning cars? What is your plan if companies cause excessive disruption?

(2) How will you share data with companies that they should be using to limit testing?

For example, you might wish to ask companies not to test near parades, first amendment events, active school zones, or construction areas. Is it easy for companies to access this information, preferably electronically? Are they interested in doing that? Will they follow your requested testing/operational exclusion areas? What do you plan to do if they ignore your requests to restrict testing in sensitive areas?

(3) How will you ensure testing and operational equity?

What if disruption, testing incidents, and other issues are concentrated in historically disadvantaged areas? (This might happen due to company policy, but might instead be an unintended emergent result due to higher-than-average population density and emergency response activity in such areas.)

How will you know whether exposure to testing risk is being imposed in an equitable manner, especially if robotaxi companies claim that where they test is a trade secret?

If robotaxis are being sold based on service to the disabled and for other social goods, how will you be able to measure whether companies are living up to their promises?

(4) How will you issue traffic tickets to a robotaxi?

Some states require a moving violation citation to be issued to a natural person, but robotaxis don't have a driver. Consider proactively moving to get state-level regulations fixed sooner rather than later to correct this. Without the ability to ticket robotaxis you might find yourself without viable enforcement tools for the worst robotaxi behaviors that might occur.

(5) How can you productively engage with companies despite municipal preemption laws?

Apparently in some cities there is a good working relationship with between city government and robotaxi operators. In San Francisco the city and the companies are practically at war. There are no magic solutions, but trying hard up front to build bridges before things get tense is better than reacting to bad news.

-----------------------------

Autonomous Vehicle State Policy Issues (Talk Video)

The commercial deployment of robotaxis in San Francisco has made it apparent many issues remain to be resolved regarding the regulation and governance of autonomous vehicle technology at the state and local levels. This talk is directed at state and local stakeholders who are considering how to set policies and regulations governing this technology.

Topics:

• Getting past Automated Vehicle (AV) safety rhetoric
• AV safety in a nutshell
• Safe as a human driver on average
• Avoiding risk transfer to vulnerable populations
• Avoiding negligent computer driving
• Conforming to industry consensus safety standards
• Addressing other ethical & equity concerns
• Policy points:
• Societal benefits
• Public road testing
• Municipal preemption
• SAE Level 2/2+/3 issues
• Federal vs. state regulation
• Other policy issues
• Revisiting common myths

Slides: https://users.ece.cmu.edu/~koopman/lectures/L136-AV_State_Policy_Issues.pdf

Youtube video: https://youtu.be/6YTSjmxu-mI

Youtube play list of individual slide videos:  https://www.youtube.com/watch?v=TSW4617EDeQ&list=PL_DQiOR0jhbWdRqQM17be0vylaUwd9WOc&pp=gAQBiAQB

Archive.org video: https://archive.org/details/L136-state-regulatory-advice-for-autonomous-vehicles

Washington State policy meeting in which I give this talk and answer questions: https://avworkgroupwa.org/committee-meeting/executive-committee-meeting-15

AV Safety Claims and More on My Congressional Testimony

I recently had the privilege of testifying before the US House E&C committee on self-driving car safety. You can see the materials here:

• The nearly 3-hour hearing video
• My written testimony
• Basic hearing information (other written testimony in the AGENDA bar)
• Document repository (might have other documents later)

A venue like this does not offer the best forum for nuance. In particular, one can make a precise statement for a reason and have that statement misunderstood (because there is limited time to explain), or misconstrued. The result can be talking past each other for reasons ranging from simple misunderstanding, to ideological differences, to the other side needing to show they are right regardless of the counter-arguments. I do not attempt to cover all topics here; just ones that feel like they could use some more discussion. (See my written testimony for the full list of topics.)

The venue also requires expressing opinions about the best path forward, which can legitimately have different views. I happen to believe that setting up a requirement to follow safety standards is our best bet to be competitive long term with international competitors (who end up having that same requirement). Others disagree.

In this blog I have the luxury of spending some time on some areas that could not be covered with as much nuance/detail in the official proceedings.

US House E&C Hearing on July 26, 2023

Claims that AVs are already safe are premature

Ultimately AVs will win or lose based on public trust. I believe that making overly aggressive claims about safety degrade that trust.

The AV companies are busy messaging that they have already proven are they are better than human drivers, and framing it as a discussion of fatality rates. In other words, they are declaring victory on public road safety in terms of reducing road fatalities. But the data analysis does not support that they are reducing fatalities, and it is still not really clear what the crash/injury rate outcomes are.

Their messaging amounts to 40,000 Americans die on roads every year. We have proven we are safer. Delaying us will kill people.   (Where "us" is the AV industry.)  (Cruise: "Humans are terrible drivers" and computers "never drive distracted, drowsy, or drunk")  Cruise has also published some bar graphs of unclear meaning, because the baseline data and details are not public, and the bars selected tell only part of the story (e.g., "collision with meaningful risk of injury" instead of injuries when we know they have already had a multi-injury crash, which therefore undercounts injuries; and only collisions with "primary contribution" when we know they were partially at fault for that multi-injury crash, even if not at "primary" fault.) I have not seen Cruise explicitly say they have reduced fatalities (they say they "are on track to far exceed this projected safety benefit"), but the implied message of declaring victory on safety is quite clear from them ("our driverless AVs have outperformed the average human drier in San Francisco by a large margin.")

Other messaging might be based on reasonable data analysis that is extended to conclusions that go beyond the available data. Waymo: "the Waymo Driver is already reducing traffic injuries and fatalities" -- where the fatality rate is an early estimate, and the serious injury rate numbers are small enough to still be in the data collection phase.  Did I say their report is wrong? I did not. I said that the marketing claims being made are unsupported. If they claimed "our modeling projects we are reducing traffic injuries and shows us on track for reducing fatalities" then that might well be a reasonable claim. But it is not the claim they are making. I note their academic-style papers do a much more rigorous job of stating claims than their marketing material. So this is a matter of overly-aggressive marketing.

It is premature to declare victory. (Did I say the claim of reduced fatalities is definitely false? No. I said it is premature to make that claim. In other words, nobody knows how this will turn out.)

Waymo and Cruise have 1 to 3 million miles each without a driver. Mean time between human driver fatal crashes is ballpark 100 Million miles (details and nuances, but we know human drivers -- including the drunks -- can do this on US public roads in a good year). So at a few million miles there is insufficient experience to know how fatalities will actually turn out.

We are much further away from the data it will take to understand fatalities, which ranges from 300 million to 1 billion miles for a high statistical confidence. A single fatality by any AV company in the next year or so would likely prove that AVs are not safe enough, but we don't know if that will happen.

Missy Cummings has recent results that shows that Waymo has about 4x more non-fatal crashes on non-interstate roads than average human drivers (also on non-interstate roads) -- and Cruise has about 8x more. However, these crash rates are similar to Lyft and Uber in California.  (There is actual research backing up that statement that will be published in due course.)

Also, even if one company shows it is safe enough, that does not automatically make other companies safe. We've already seen differences between Waymo (no injury crashes) and Cruise (a multi-injury crash). Whether that is just bad luck or representative still takes more data. Industry messaging that amounts to "Waymo is safe therefore all AVs are safe" is also problematic, especially if it claims victory on fatality rates.

The reality is that both Waymo and Cruise are using statistical models of varying degrees of sophistication to predict their safety outcomes. Predictions can be wrong. In safety predictions often are wrong for companies with poor safety cultures or who decide not to follow industry safety standards -- but we don't find out until the catastrophic failure makes the news. We can hope that won't happen here, but it is hope, not time for a victory dance.

Summary: Companies are predicting they will reduce fatalities. That is not the same as actually proving it. There is a long way to run here, and the only thing I am sure is there will be surprises. Perhaps in a year we'll have enough data to get some more clarity about property damage and injury crashes, but only for companies that want to be transparent about their data.  It will be even longer to show that the fatality rate is on a par with human drivers. If bad news arrives, it will come sooner. We should not make policy assuming they are safer.

Blame and AV safety

Blaming someone does not improve safety if it deflects the need to make a safety improvement. In particular, saying that a crash was not the fault of an AV company is irrelevant to measuring and improving safety. Much of road safety comes not from being blameless, but rather for compensating for mistakes, infrastructure faults, and other hazards not one's own fault. 

Any emphasis on metrics that emphasizes "but it was not mostly our fault" is about public relations, not about safety.  I guess PR is fine for investors, but baking that into a safety management system means lost opportunities to improve safety. That is not the behavior appropriate for any company who claims safety is their most important priority.  If a company wants to publish both "crashes" and "at fault crashes" then I guess OK (although "at fault" should include partially at fault, not 49% at fault rounds down to 0% at fault). But publishing only "at fault" crashes is about publicity, not about safety transparency. (Even worse is lobbying that only "at fault" crashes should be reported in data collection.)

On the other hand, it is important to hold AV companies accountable for safety, just as we hold human drivers accountable. A computer driver should have the same duty of care as a human driver on public roads. This is not formally the situation now, and this part of tort law will take a lot of cases to resolve, wasting a lot of time and resources. The manufacturer should be the responsible party for any negligent driving (i.e., driving behavior that would be negligent if a human driver were to do it) by their computer driver. Not the owner, and not the operator, because neither has the ability to design and validate the computer driver's behavior. This aspect of blame will use tort law in its primary role: to put pressure on the responsible party to avoid negligent driving behavior. The same rules should apply to human and computer drivers.

There is a nuanced issue regarding liability here. Companies seem to want to restrict their exposure to being only product liability, and evade tort law. However, if a computer driver runs a red light, that should be treated exactly as a human driver negligence situation. There should be no need to reverse engineer a huge neural network to prove a specific design defect (product liability) -- the fact of running a red light should be the basis for making a claim based on negligent behavior alone (tort law) without having the burden to prove a product defect. Product liability is more expensive and more difficult to pursue. The emphasis should be on using tort law when possible, and product liability only as a secondary path. That will keep costs down and make deserved compensation more accessible on the same basis it is for human driver negligence.

Also, aggressively blaming others rather than saying at the very least "we could have helped avoid this crash even if other driver is assigned blame" degrades trust.

Summary: Statistics that incorporate blame impair transparency. However, it is helpful for tort law to hold the manufacturers accountable for negligent behavior by computer drivers. And you would think computer drivers should have near-zero negligent driving rates? Insisting on product liability rather than tort law is a way for manufacturers to decrease their accountability for computer driver problems, harming the ability other road users to seek justified compensation if harmed.

Level 2/2+:

All this attention to AVs is distracting the discussion from a much bigger and more pressing economic and safety issue: auto-pilot systems and the like. The need to regulate those systems is much more urgent from a societal point of view. But it's not the discussion because the auto industry has already gotten itself a situation with no regulation other than a data reporting requirement and the occasional (perhaps after many years) recall.

Driver monitoring effectiveness and designing a human/computer interaction approach that does not turn human drivers into moral crumple zones needs a lot more attention. It will take a long time for NHTSA to address this beyond doing recalls for the more egregious issues. Tort law (holding the computer driver accountable when it is steering) seems the only viable way to put some guard rails in place in the near- to mid-term.

Opinion: Level 2/2+ is what matters for the car industry now for both safety and economic benefits. AVs are still a longer term bet. 

Don't sell on safety:

Companies should not sell solving the 40K/year fatality problem. There are many other technologies that can make a much quicker difference in that area. And social change for that matter. If what we want is better road safety, investing tens of billions of dollars in robotaxi technology is one of the least economically efficient ways to do this. Instead we could improve active safety systems, encourage a switch to safer mass transit, press harder for social change on impaired/distracted driving, and so on. While one hopes this will long term help with fatalities, this is simply the wrong battle for the industry to try to fight with this technology for at least a decade. (Even if the perfect robotaxi were invented today -- which we are a long way from -- it would take many years to see a big drop in fatalities due to the time to turn over the automotive fleet that has an average age of about 12 years.)

Companies should sell on economic benefit, being better for cities, being better for consumers, transportation equity, and so on -- while not creating safety issues. Safety promises should simply indicate they are doing no harm. This is much easier to show, assuming it is true. And it does not set the industry up for collapse when the next (remember Uber ATG?) fatality eventually arrives.

The issue is that any statement about reducing fatalities is a prediction, not a conclusion. I would hope  that car companies would not release a driverless car onto public roads unless they can predict it is safer than a human driver. They should disclose that argument in a transparent way. But it is a prediction, not a certainty. It will take years to prove. Why pick a fight that is so difficult when there is really no need to do so? 

A smarter way to explain to the public how they are ensuring a safe and responsible release is to use an approach such as:

1. Follow industry safety standards to set a reasonable expectation of safe deployment and publicly disclose independent conformance checks.
2. Establish metrics that will be used to prove safety in advance (not cherry-picked after the fact).
3. Transparent monthly reports of those metric outcome vs. goals
4. Show that issues identified are resolved vs. continuing to scale up despite problems. Problems includes not only crashes, but also negative externalities on other road users
5. Publish lessons learned in a generic way
6. Show public benefit is being delivered beyond safety, again with periodic metric publications.

Three principles for safety, all of which are a problem with the industry's current adversarial approach to regulatory oversight, are:

1. Transparency
2. Accountability
3. Independent oversight

It is not only that you need to do those things to actually get safety. It is also that these things build trust.

Other key points:

• Any person or organization who promotes the "human drivers are bad, so computers will be safe " and/or the "94% of crashes are caused by human error" talking points should be presumptively considered an unreliable source of information. At this point I feel those are propaganda points. Any organization saying that Safety is their #1 priority should know better.

• The main challenge to the industry is not regulations -- it is the ability to build reliable, safe vehicles that scale up in the face of the complexity of the real world. Expectations of exponential numbers of cars deploying any time soon seem unrealistic. The current industry city-by-city approach is likely to continue to grind away for years to come. Being realistic about this will avoid pressure to make overly aggressive deployments that compromise safety.

• In other industries (e.g., aviation, rail) following their own industry standards is an essential part of assuring safety. The car companies should be required to follow their standards too (e.g., ISO 26262, ISO 21448, UL 4600, ISO/SAE 21434, perhaps ISO TS 5083 when we find out what is in it). This varies across companies, with some companies being very clearly against following those standards.

• There is already a regulatory framework, written by the previous administration. This gives us an existing process with an existing potential bipartisan starting point to move the discussion forward instead of starting from scratch with rule making. That framework includes a significant shift in government policy to require the industry to follow its own consensus safety standards. My understanding is that US Government policy is to use such standards whenever feasible. It is time for US DOT to get with the program here (as they proposed to do several years ago -- but stalled ever since).

• Absolute municipal and state preemption are a problem, especially for "performance" aspects of a computer driver:
• This leaves states and localities prevented from protecting their constituents (if they choose to do so) while the Federal Government is still working on AV regulations
• Even after there are federal regulations, state and local governments need to be able to create and enforce traffic laws, rules of the road, and hold computer drivers accountable (e.g., issue and revoke licenses based on factors such as computer driver negligence)
• In the end, the Federal Government should regulate the ability of equipment to follow whatever road rules are in place. States and localities should be able to set behavioral rules for road use and enforce compliance for computer drivers without the Federal Government subsuming that traditional State/Local ability to adapt traffic rules to local conditions.

• Do you remember how ride hail networks were supposed to solve the transportation equity problem? Didn't really happen, did it? Forced arbitration was a part of that outcome, especially for the disabled. We need to make sure that the AV story has a better ending by avoiding forced arbitration being imposed on road users. It is even possible that taking one ride hail ride might force you into arbitration if you are later hurt as a pedestrian by a car from that company (depends on the contract language -- the one you clicked without really reading or understanding even if you did read it). Other aspects of equity matter too, such as equity in exposing vulnerable populations to the risks of public road testing.

• There are numerous other points summarized after the end of my written narrative that also matter, covering safety technology, jobs/economic impact, liability, data reporting, regulating safety, avoiding complete preemption, and debunking industry-promoted myths.

• There is a Q&A at the end of my testimony where I have the time to give more robust answers to some of the questions I was asked, and more.

Last update 7/27/2023

AV Safety and the False Dilemma Fallacy

The current AV company messaging strategy is a classic case of a false dilemma fallacy. They frame the situation as a choice between continued human drivers killing people (without statistical context) vs. immature robotaxis who don't drink and drive (but make other mistakes). (Wikipedia: False dilemma)

The recent Cruise ad in particular is a plainly ridiculous doubling-down on the industry's long discredited propaganda playbook.

Cruise NY Times ad: https://twitter.com/kvogt/status/1679517290847694848

Analysis of AV industry playbook: https://www.eetimes.com/autonomous-vehicle-myths-the-dirty-dozen/

A more reasonable message would be cities need robotaxis for <reasons> and robotaxi companies will use <defined, balanced metrics, stated in advance rather than cherry picked later> to show they are no worse than human drivers during development, with monthly report card disclosures. Improved safety comes later -- we all hope.

Here is where things really stand:

• It is too early to know whether current robotaxi technology is safer than human drivers for fatalities. The industry is stringing us along hoping they can show they are safe over time (starting now, but not really there yet).
• Non-autonomous technology (AEB) is making far more of a contribution right now -- but is missing from the false dilemma.
• Public transit (much safer) also not in the discussion.
• Improving road safety (speed limits, traffic calming, etc.) also not in the discussion. Also missing are specific pedestrian and cyclist safety improvements. While we're at it, seat belts, drunk driving, and motorcycle safety measures.
• The messaging from both sides (parts of the SF govt and especially Cruise) on crashes does not address factors required for a reasonable comparison. (ODD, baseline driver population, etc.)
• It is clear that vehicles from both Waymo and Cruise are creating public road disruption. There is no excuse for impeding emergency responders just to get "Look Ma, No Driver!!" optics.
• The technology can be advanced by continuing to test while having human drivers or in-car valets (employee in the front passenger seat) to mitigate problems. Their Safety Management System should include a step of adding/removing in-car vehicle supervisors until issues that cause public disruption are shown to be resolved in deployment.
• Cruise in particular needs to get more diligent about pre-deployment testing. There is simply no excuse for rear-ending a Muni bus due to a software defect in an uncrewed vehicle that occurred in a pretty normal situation. Waymo isn't perfect, but their failures are more at the edge.
• The public outrage is entirely self-inflicted by companies due to their exploitation of the municipal preemption clause in state-level regulations rather than being responsible road users. Playing the "we should forgive their drivers who are still learning" card has worn out its welcome.

Some might want to point out that some companies are worse actors than others, but all the companies have their issues. (For example, good work by the Waymo safety team is hurt by their government relations breathless safety hype messaging.)

And the reality is that a crash or adverse news for one company hurts them all.

A Liability (Duty of Care) Approach for Automated Vehicles in Three Parts

I'm delighted that months of collaboration with co-author and law professor William Widen have resulted in a trio of papers that together provide a framework for resolving the vast majority of automated vehicle legal questions. Product liability will still be a thing, but that should be reserved for its more usual role, and not be the sole means of recourse for everyday Computer Driver road mishaps that will displace the everyday Human Driver road mishaps. A tort law approach based on assigning a duty of care (negligence) is a far better fit and will require far less disruption to existing legal and regulatory systems while providing a fair basis for compensation for anyone harmed by this novel technology.

The three parts are in three separate SSRN papers intended to be used as a set, although each paper is self-contained. Below are very simplified summaries to give an overview.

25 minute video with overview of the concepts:  https://youtu.be/i0ZGSEFHwE8 or https://archive.org/details/l-139-computer-driver

Podcast discussion and summary to warm up with:  https://ojoyoshidareport.com/podcast-lets-talk-about-av-liability/

These topics and many more based on what we have learned since these papers were written are discussed in my books:  How Safe Is Safe Enough (2022), and Embodied AI Safety (2025)

(1) Computer Driver: Define the concept of synthetic negligence for a Computer Driver. A Computer Driver should be held to the same standards of negligence for harm it causes as a Human Driver. The manufacturer should be the responsible party for any negligent behavior on the part of a Computer Driver because they are the ones who should be incentivized to produce safe automated driving systems. The behavioral standard is not an "average driver" but rather a "reasonable driver."

Winning the Imitation Game: Setting Safety Expectations for Automated Vehicles, 25 Minn. J.L. Sci. & Tech. 113 (2023)  https://scholarship.law.umn.edu/mjlst/vol25/iss1/5/

Also see this shorter summary paper of the same material from WAISE 2023 for a more general and technical audience: Koopman, P. & Widen, W., "A Reasonable Driver Standard for Automated Vehicle Safety," Safecomp WAISE workshop, Sept. 2023

Also see this Jurist piece on how this might work with criminal law, especially for a Level 3 vehicle in which the driver has been told it is OK not to watch the road: Widen, W. & Koopman, P., Level 3 Automated Vehicles and Criminal Law, Jurist, Aug. 2023

Video about why using product liability for computer drivers is likely to break the court system: https://www.youtube.com/watch?v=WhtxTDRvTOE

(2) Liability Transfer Rules: Define the rules of transfer of liability between the Human Driver and the Computer Driver depending on the operational mode per the summary figure below. Shared responsibility (Human Driver supervises safety of Computer Driver) requires special attention to avoid the person being used as a moral crumple zone. Two key rules come into play regarding the need for effective driver monitoring and the obligation of a person to intervene when it is reasonable that they would know to do so.

The Awkward Middle for Automated Vehicles: Liability Attribution Rules When Humans and Computers Share Driving Responsibilities
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4444854
https://www.americanbar.org/groups/science_technology/publications/jurimetrics/2024/jurimetrics-fall-2023/

(3) Definitions and Statute Outline: Create a set of definitions and statute-oriented rules to make the first two papers more actionable. We envision this as a robust starting point for state legislatures that find this approach useful.

Liability Rules for Automated Vehicles: Definitions & Detail
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4444848
https://scholar.smu.edu/scitech/vol27/iss1/5/

Kia power door pinch recall -- what should be done?

 Sometimes how to handle a safety issue is not so clear. The Kia power sliding door recall is an interesting example of the types of issues that can come up that are not clear-cut.

- Nine confirmed injuries due to minivan power door sliding closed on people, ranging from bruising to a fractured thumb to a broken arm.

- Investigation reveals that system works exactly as designed, noting auto-reverse to prevent injury is a "supplemental" rather than mission-critical safety feature (i.e., best effort is permissible, and their idea of best effort includes a broken arm)

- Investigation results claim all the competitor sliding doors are just as dangerous mechanically (comparable closing force/reversal properties)

- The remedy slows down the final inches of closing and sounds a couple warning beeps when moving the door. That does not sound like a particularly robust fix. It sounds like "we need to do something, how about this..." (And why weren't these in place before?)

- Efforts throughout focus on "customer awareness," mailing customers reminders of functions, etc. Classical playbook of blame the user and "educate" them even harder despite evidence that education isn't working.

I have diverse reactions to this:

- Why did it take a NHTSA recall to get Kia to slow door and add beeps after the broken arm incident? Why didn't they make these changes earlier?

- I note the Kia did not say they compared its human interface to others -- just the closing mechanics. I remember a different brand minivan I owned having beeps for the door and slowing down on closing 20+ years ago, so not rocket science here.

- Hard to believe that beeps will keep kids from reaching in to grab something as the door closes (one of the incidents reported)

- A broken arm is no fun, but it's only a broken arm and only one of them. (My career in the elevator industry started with getting my hand stuck in an elevator door when I was probably 4 years old because I was curious to see how it worked; I remember that incident pretty well...)

- If there is a safety issue across this vehicle class as Kia implies in their report, it is obvious the companies think everything is fine so nothing will change. Should it change? Is there a way to get it to change without NHTSA having to beat up the whole industry (which will likely take more bad things to happen before they have a mandate to do that)? Or is this just that Kia got the human factors wrong and there is no standard/common practice for this.

- Perhaps an industry standard could address this before NHTSA is eventually forced to act. Of course the companies would actually have to follow the industry safety standard, for which they have a poor track record in automotive.

This suggests that the car companies don't have a lot of incentive, resources, inclination, or something to think about the lesser severity safety issues. It is really easy to blame these injuries on children not behaving in a mature way. But it's a minivan -- having kids around a power door is pretty much the whole reason these vehicles exist. It feels like this, and many other less critical safety issues, could be handled better. But that will take a big cultural pivot away from the blame-the-driver situation we have today.

IMHO weak safety culture operating under air cover of cost savings (even when that isn't really a valid excuse) is the big overarching issue. Power door pinches are just another symptom.

Interested in everyone's thoughts on this one...

NHTSA recall 23V236 timeline narrative: https://static.nhtsa.gov/odi/rcl/2023/RMISC-23V236-8289.pdf

Article summary: https://www.autoevolution.com/news/kia-recalls-new-carnival-over-software-issue-automaker-aware-of-nine-confirmed-injuries-213320.html

A Liability-Based Regulatory Framework for Vehicle Automation Technology

State liability laws might be the way out of the automated vehicle regulatory dilemma. From phantom braking to reckless public road testing to permitting using human drivers as moral crumple zones, vehicle automation regulation is a hot mess. States are busy creating absurd laws that assign safety responsibility to a non-legal-person computer, while the best the feds can do under the circumstances is play recall whack-a-mole with unsafe features that are deployed faster than they can investigate.

What has become clear is that attempting to regulate the technology directly is not working out. In the long term it will have to be done, but we will likely need to see fundamental changes at US DOT before we see viable regulatory approaches to automated vehicles. (As a start, they need to abandon the use of SAE Levels for regulatory purposes.) That process will take years, and if history is any guide, one or more horrific tragedies before things settle out. Meanwhile, as companies aggressively exploit the "Level 2 loophole" it is the wild west on public roads. Various companies are taking safety with different levels of seriousness, but there is a dramatic lack of transparency and accountability across the industry that will only get worse with time.

As a short- to mid-term approach we should revisit how liability laws work at the state level to buy time to let the technology mature while avoiding needless harm to constituents. There are three fundamental things that have changed that make the current tort system unworkable in practice for automated vehicle technology:

#1: Machine learning-based technology is inherently unsuitable to traditional software safety analysis. The current legal system which puts the burden of showing technology is defective on victims is simply not viable when even the engineers who designed a system can't necessarily explain why the computer driver did what it did.

#2: Asymmetric access to information makes it easy for car companies to know what happened in a crash (or even if automated driving was activated), but it is very difficult for victims to access, much less interpret such information.

#3: The litigation cost of pursuing a claim against software with non-deterministic defects that require source code analysis is huge, depriving all but the largest cases from having an effective ability to prove a product defect claim, if one is justified.

In response to these realities, a (rebuttable) presumption of liability and burden of proof should be shifted to manufacturers in situations for which it is unreasonable to expect a civilian human driver to be able to ensure safety. The attached summary sketches an approach, with more detail to come.

Read the one-pager policy summary here: https://archive.org/details/2023-03-av-liability-one-pager-published-v-1-00