Saturday, August 26, 2023

Autonomous Vehicle State Policy Issues (Talk Video)

The commercial deployment of robotaxis in San Francisco has made it apparent many issues remain to be resolved regarding the regulation and governance of autonomous vehicle technology at the state and local levels. This talk is directed at state and local stakeholders who are considering how to set policies and regulations governing this technology.

Topics:

  • Getting past Automated Vehicle (AV) safety rhetoric
  • AV safety in a nutshell
    • Safe as a human driver on average
    • Avoiding risk transfer to vulnerable populations
    • Avoiding negligent computer driving
    • Conforming to industry consensus safety standards
    • Addressing other ethical & equity concerns
  • Policy points:
    • Societal benefits
    • Public road testing
    • Municipal preemption
    • SAE Level 2/2+/3 issues
    • Federal vs. state regulation
    • Other policy issues
  • Revisiting common myths


Washington State policy meeting in which I give this talk and answer questions: https://avworkgroupwa.org/committee-meeting/executive-committee-meeting-15




Thursday, July 27, 2023

AV Safety Claims and More on My Congressional Testimony

I recently had the privilege of testifying before the US House E&C committee on self-driving car safety. You can see the materials here:

A venue like this does not offer the best forum for nuance. In particular, one can make a precise statement for a reason and have that statement misunderstood (because there is limited time to explain), or misconstrued. The result can be talking past each other for reasons ranging from simple misunderstanding, to ideological differences, to the other side needing to show they are right regardless of the counter-arguments. I do not attempt to cover all topics here; just ones that feel like they could use some more discussion. (See my written testimony for the full list of topics.)

The venue also requires expressing opinions about the best path forward, which can legitimately have different views. I happen to believe that setting up a requirement to follow safety standards is our best bet to be competitive long term with international competitors (who end up having that same requirement). Others disagree.

In this blog I have the luxury of spending some time on some areas that could not be covered with as much nuance/detail in the official proceedings.

US House E&C Hearing on July 26, 2023


Claims that AVs are already safe are premature

Ultimately AVs will win or lose based on public trust. I believe that making overly aggressive claims about safety degrade that trust.

The AV companies are busy messaging that they have already proven are they are better than human drivers, and framing it as a discussion of fatality rates. In other words, they are declaring victory on public road safety in terms of reducing road fatalities. But the data analysis does not support that they are reducing fatalities, and it is still not really clear what the crash/injury rate outcomes are.

Their messaging amounts to 40,000 Americans die on roads every year. We have proven we are safer. Delaying us will kill people.   (Where "us" is the AV industry.)  (Cruise: "Humans are terrible drivers" and computers "never drive distracted, drowsy, or drunk"Cruise has also published some bar graphs of unclear meaning, because the baseline data and details are not public, and the bars selected tell only part of the story (e.g., "collision with meaningful risk of injury" instead of injuries when we know they have already had a multi-injury crash, which therefore undercounts injuries; and only collisions with "primary contribution" when we know they were partially at fault for that multi-injury crash, even if not at "primary" fault.) I have not seen Cruise explicitly say they have reduced fatalities (they say they "are on track to far exceed this projected safety benefit"), but the implied message of declaring victory on safety is quite clear from them ("our driverless AVs have outperformed the average human drier in San Francisco by a large margin.")

Other messaging might be based on reasonable data analysis that is extended to conclusions that go beyond the available data. Waymo: "the Waymo Driver is already reducing traffic injuries and fatalities" -- where the fatality rate is an early estimate, and the serious injury rate numbers are small enough to still be in the data collection phase.  Did I say their report is wrong? I did not. I said that the marketing claims being made are unsupported. If they claimed "our modeling projects we are reducing traffic injuries and shows us on track for reducing fatalities" then that might well be a reasonable claim. But it is not the claim they are making. I note their academic-style papers do a much more rigorous job of stating claims than their marketing material. So this is a matter of overly-aggressive marketing.

It is premature to declare victory. (Did I say the claim of reduced fatalities is definitely false? No. I said it is premature to make that claim. In other words, nobody knows how this will turn out.)

Waymo and Cruise have 1 to 3 million miles each without a driver. Mean time between human driver fatal crashes is ballpark 100 Million miles (details and nuances, but we know human drivers -- including the drunks -- can do this on US public roads in a good year). So at a few million miles there is insufficient experience to know how fatalities will actually turn out.

We are much further away from the data it will take to understand fatalities, which ranges from 300 million to 1 billion miles for a high statistical confidence. A single fatality by any AV company in the next year or so would likely prove that AVs are not safe enough, but we don't know if that will happen.

Missy Cummings has recent results that shows that Waymo has about 4x more non-fatal crashes on non-interstate roads than average human drivers (also on non-interstate roads) -- and Cruise has about 8x more. However, these crash rates are similar to Lyft and Uber in California.  (There is actual research backing up that statement that will be published in due course.)

Also, even if one company shows it is safe enough, that does not automatically make other companies safe. We've already seen differences between Waymo (no injury crashes) and Cruise (a multi-injury crash). Whether that is just bad luck or representative still takes more data. Industry messaging that amounts to "Waymo is safe therefore all AVs are safe" is also problematic, especially if it claims victory on fatality rates.

The reality is that both Waymo and Cruise are using statistical models of varying degrees of sophistication to predict their safety outcomes. Predictions can be wrong. In safety predictions often are wrong for companies with poor safety cultures or who decide not to follow industry safety standards -- but we don't find out until the catastrophic failure makes the news. We can hope that won't happen here, but it is hope, not time for a victory dance.

Summary: Companies are predicting they will reduce fatalities. That is not the same as actually proving it. There is a long way to run here, and the only thing I am sure is there will be surprises. Perhaps in a year we'll have enough data to get some more clarity about property damage and injury crashes, but only for companies that want to be transparent about their data.  It will be even longer to show that the fatality rate is on a par with human drivers. If bad news arrives, it will come sooner. We should not make policy assuming they are safer.

Blame and AV safety

Blaming someone does not improve safety if it deflects the need to make a safety improvement. In particular, saying that a crash was not the fault of an AV company is irrelevant to measuring and improving safety. Much of road safety comes not from being blameless, but rather for compensating for mistakes, infrastructure faults, and other hazards not one's own fault. 

Any emphasis on metrics that emphasizes "but it was not mostly our fault" is about public relations, not about safety.  I guess PR is fine for investors, but baking that into a safety management system means lost opportunities to improve safety. That is not the behavior appropriate for any company who claims safety is their most important priority.  If a company wants to publish both "crashes" and "at fault crashes" then I guess OK (although "at fault" should include partially at fault, not 49% at fault rounds down to 0% at fault). But publishing only "at fault" crashes is about publicity, not about safety transparency. (Even worse is lobbying that only "at fault" crashes should be reported in data collection.)

On the other hand, it is important to hold AV companies accountable for safety, just as we hold human drivers accountable. A computer driver should have the same duty of care as a human driver on public roads. This is not formally the situation now, and this part of tort law will take a lot of cases to resolve, wasting a lot of time and resources. The manufacturer should be the responsible party for any negligent driving (i.e., driving behavior that would be negligent if a human driver were to do it) by their computer driver. Not the owner, and not the operator, because neither has the ability to design and validate the computer driver's behavior. This aspect of blame will use tort law in its primary role: to put pressure on the responsible party to avoid negligent driving behavior. The same rules should apply to human and computer drivers.

There is a nuanced issue regarding liability here. Companies seem to want to restrict their exposure to being only product liability, and evade tort law. However, if a computer driver runs a red light, that should be treated exactly as a human driver negligence situation. There should be no need to reverse engineer a huge neural network to prove a specific design defect (product liability) -- the fact of running a red light should be the basis for making a claim based on negligent behavior alone (tort law) without having the burden to prove a product defect. Product liability is more expensive and more difficult to pursue. The emphasis should be on using tort law when possible, and product liability only as a secondary path. That will keep costs down and make deserved compensation more accessible on the same basis it is for human driver negligence.

Also, aggressively blaming others rather than saying at the very least "we could have helped avoid this crash even if other driver is assigned blame" degrades trust.

Summary: Statistics that incorporate blame impair transparency. However, it is helpful for tort law to hold the manufacturers accountable for negligent behavior by computer drivers. And you would think computer drivers should have near-zero negligent driving rates? Insisting on product liability rather than tort law is a way for manufacturers to decrease their accountability for computer driver problems, harming the ability other road users to seek justified compensation if harmed.

Level 2/2+:

All this attention to AVs is distracting the discussion from a much bigger and more pressing economic and safety issue: auto-pilot systems and the like. The need to regulate those systems is much more urgent from a societal point of view. But it's not the discussion because the auto industry has already gotten itself a situation with no regulation other than a data reporting requirement and the occasional (perhaps after many years) recall.

Driver monitoring effectiveness and designing a human/computer interaction approach that does not turn human drivers into moral crumple zones needs a lot more attention. It will take a long time for NHTSA to address this beyond doing recalls for the more egregious issues. Tort law (holding the computer driver accountable when it is steering) seems the only viable way to put some guard rails in place in the near- to mid-term.

Opinion: Level 2/2+ is what matters for the car industry now for both safety and economic benefits. AVs are still a longer term bet. 

Don't sell on safety:

Companies should not sell solving the 40K/year fatality problem. There are many other technologies that can make a much quicker difference in that area. And social change for that matter. If what we want is better road safety, investing tens of billions of dollars in robotaxi technology is one of the least economically efficient ways to do this. Instead we could improve active safety systems, encourage a switch to safer mass transit, press harder for social change on impaired/distracted driving, and so on. While one hopes this will long term help with fatalities, this is simply the wrong battle for the industry to try to fight with this technology for at least a decade. (Even if the perfect robotaxi were invented today -- which we are a long way from -- it would take many years to see a big drop in fatalities due to the time to turn over the automotive fleet that has an average age of about 12 years.)

Companies should sell on economic benefit, being better for cities, being better for consumers, transportation equity, and so on -- while not creating safety issues. Safety promises should simply indicate they are doing no harm. This is much easier to show, assuming it is true. And it does not set the industry up for collapse when the next (remember Uber ATG?) fatality eventually arrives.

The issue is that any statement about reducing fatalities is a prediction, not a conclusion. I would hope  that car companies would not release a driverless car onto public roads unless they can predict it is safer than a human driver. They should disclose that argument in a transparent way. But it is a prediction, not a certainty. It will take years to prove. Why pick a fight that is so difficult when there is really no need to do so? 

A smarter way to explain to the public how they are ensuring a safe and responsible release is to use an approach such as:
  1. Follow industry safety standards to set a reasonable expectation of safe deployment and publicly disclose independent conformance checks.
  2. Establish metrics that will be used to prove safety in advance (not cherry-picked after the fact).
  3. Transparent monthly reports of those metric outcome vs. goals
  4. Show that issues identified are resolved vs. continuing to scale up despite problems. Problems includes not only crashes, but also negative externalities on other road users
  5. Publish lessons learned in a generic way
  6. Show public benefit is being delivered beyond safety, again with periodic metric publications.
Three principles for safety, all of which are a problem with the industry's current adversarial approach to regulatory oversight, are:
  1. Transparency
  2. Accountability
  3. Independent oversight
It is not only that you need to do those things to actually get safety. It is also that these things build trust.

Other key points:

  • Any person or organization who promotes the "human drivers are bad, so computers will be safe " and/or the "94% of crashes are caused by human error" talking points should be presumptively considered an unreliable source of information. At this point I feel those are propaganda points. Any organization saying that Safety is their #1 priority should know better.
  • The main challenge to the industry is not regulations -- it is the ability to build reliable, safe vehicles that scale up in the face of the complexity of the real world. Expectations of exponential numbers of cars deploying any time soon seem unrealistic. The current industry city-by-city approach is likely to continue to grind away for years to come. Being realistic about this will avoid pressure to make overly aggressive deployments that compromise safety.
  • In other industries (e.g., aviation, rail) following their own industry standards is an essential part of assuring safety. The car companies should be required to follow their standards too (e.g., ISO 26262, ISO 21448, UL 4600, ISO/SAE 21434, perhaps ISO TS 5083 when we find out what is in it). This varies across companies, with some companies being very clearly against following those standards.
  • There is already a regulatory framework, written by the previous administration. This gives us an existing process with an existing potential bipartisan starting point to move the discussion forward instead of starting from scratch with rule making. That framework includes a significant shift in government policy to require the industry to follow its own consensus safety standards. My understanding is that US Government policy is to use such standards whenever feasible. It is time for US DOT to get with the program here (as they proposed to do several years ago -- but stalled ever since).
  • Absolute municipal and state preemption are a problem, especially for "performance" aspects of a computer driver:
    • This leaves states and localities prevented from protecting their constituents (if they choose to do so) while the Federal Government is still working on AV regulations
    • Even after there are federal regulations, state and local governments need to be able to create and enforce traffic laws, rules of the road, and hold computer drivers accountable (e.g., issue and revoke licenses based on factors such as computer driver negligence)
    • In the end, the Federal Government should regulate the ability of equipment to follow whatever road rules are in place. States and localities should be able to set behavioral rules for road use and enforce compliance for computer drivers without the Federal Government subsuming that traditional State/Local ability to adapt traffic rules to local conditions.
  • Do you remember how ride hail networks were supposed to solve the transportation equity problem? Didn't really happen, did it? Forced arbitration was a part of that outcome, especially for the disabled. We need to make sure that the AV story has a better ending by avoiding forced arbitration being imposed on road users. It is even possible that taking one ride hail ride might force you into arbitration if you are later hurt as a pedestrian by a car from that company (depends on the contract language -- the one you clicked without really reading or understanding even if you did read it). Other aspects of equity matter too, such as equity in exposing vulnerable populations to the risks of public road testing.
  • There are numerous other points summarized after the end of my written narrative that also matter, covering safety technology, jobs/economic impact, liability, data reporting, regulating safety, avoiding complete preemption, and debunking industry-promoted myths.
  • There is a Q&A at the end of my testimony where I have the time to give more robust answers to some of the questions I was asked, and more.

Last update 7/27/2023

Sunday, July 16, 2023

AV Safety and the False Dilemma Fallacy

The current AV company messaging strategy is a classic case of a false dilemma fallacy. They frame the situation as a choice between continued human drivers killing people (without statistical context) vs. immature robotaxis who don't drink and drive (but make other mistakes). (Wikipedia: False dilemma)


The recent Cruise ad in particular is a plainly ridiculous doubling-down on the industry's long discredited propaganda playbook.

Cruise NY Times ad: https://twitter.com/kvogt/status/1679517290847694848


Analysis of AV industry playbook: https://www.eetimes.com/autonomous-vehicle-myths-the-dirty-dozen/


A more reasonable message would be cities need robotaxis for <reasons> and robotaxi companies will use <defined, balanced metrics, stated in advance rather than cherry picked later> to show they are no worse than human drivers during development, with monthly report card disclosures. Improved safety comes later -- we all hope.


Here is where things really stand:

  • It is too early to know whether current robotaxi technology is safer than human drivers for fatalities. The industry is stringing us along hoping they can show they are safe over time (starting now, but not really there yet).
  • Non-autonomous technology (AEB) is making far more of a contribution right now -- but is missing from the false dilemma.
  • Public transit (much safer) also not in the discussion.
  • Improving road safety (speed limits, traffic calming, etc.) also not in the discussion. Also missing are specific pedestrian and cyclist safety improvements. While we're at it, seat belts, drunk driving, and motorcycle safety measures.
  • The messaging from both sides (parts of the SF govt and especially Cruise) on crashes does not address factors required for a reasonable comparison. (ODD, baseline driver population, etc.)
  • It is clear that vehicles from both Waymo and Cruise are creating public road disruption. There is no excuse for impeding emergency responders just to get "Look Ma, No Driver!!" optics.
  • The technology can be advanced by continuing to test while having human drivers or in-car valets (employee in the front passenger seat) to mitigate problems. Their Safety Management System should include a step of adding/removing in-car vehicle supervisors until issues that cause public disruption are shown to be resolved in deployment.
  • Cruise in particular needs to get more diligent about pre-deployment testing. There is simply no excuse for rear-ending a Muni bus due to a software defect in an uncrewed vehicle that occurred in a pretty normal situation. Waymo isn't perfect, but their failures are more at the edge.
  • The public outrage is entirely self-inflicted by companies due to their exploitation of the municipal preemption clause in state-level regulations rather than being responsible road users. Playing the "we should forgive their drivers who are still learning" card has worn out its welcome.


Some might want to point out that some companies are worse actors than others, but all the companies have their issues. (For example, good work by the Waymo safety team is hurt by their government relations breathless safety hype messaging.)


And the reality is that a crash or adverse news for one company hurts them all.


Friday, May 12, 2023

A Liability (Duty of Care) Approach for Automated Vehicles in Three Parts

I'm delighted that months of collaboration with co-author and law professor William Widen have resulted in a trio of papers that together provide a framework for resolving the vast majority of automated vehicle legal questions. Product liability will still be a thing, but that should be reserved for its more usual role, and not be the sole means of recourse for everyday Computer Driver road mishaps that will displace the everyday Human Driver road mishaps. A tort law approach based on assigning a duty of care (negligence) is a far better fit and will require far less disruption to existing legal and regulatory systems while providing a fair basis for compensation for anyone harmed by this novel technology.

The three parts are in three separate SSRN papers intended to be used as a set, although each paper is self-contained. Below are very simplified summaries to give an overview.

25 minute video with overview of the concepts:  https://youtu.be/i0ZGSEFHwE8 or https://archive.org/details/l-139-computer-driver

Podcast discussion and summary to warm up with:  https://ojoyoshidareport.com/podcast-lets-talk-about-av-liability/

These topics and many more based on what we have learned since these papers were written are discussed in my books:  How Safe Is Safe Enough (2022), and Embodied AI Safety (2025)

(1) Computer Driver: Define the concept of synthetic negligence for a Computer Driver. A Computer Driver should be held to the same standards of negligence for harm it causes as a Human Driver. The manufacturer should be the responsible party for any negligent behavior on the part of a Computer Driver because they are the ones who should be incentivized to produce safe automated driving systems. The behavioral standard is not an "average driver" but rather a "reasonable driver."

Winning the Imitation Game: Setting Safety Expectations for Automated Vehicles, 25 Minn. J.L. Sci. & Tech. 113 (2023)  https://scholarship.law.umn.edu/mjlst/vol25/iss1/5/

Also see this shorter summary paper of the same material from WAISE 2023 for a more general and technical audience: Koopman, P. & Widen, W., "A Reasonable Driver Standard for Automated Vehicle Safety," Safecomp WAISE workshop, Sept. 2023

Also see this Jurist piece on how this might work with criminal law, especially for a Level 3 vehicle in which the driver has been told it is OK not to watch the road: Widen, W. & Koopman, P., Level 3 Automated Vehicles and Criminal Law, Jurist, Aug. 2023

Video about why using product liability for computer drivers is likely to break the court system: https://www.youtube.com/watch?v=WhtxTDRvTOE

(2) Liability Transfer Rules: Define the rules of transfer of liability between the Human Driver and the Computer Driver depending on the operational mode per the summary figure below. Shared responsibility (Human Driver supervises safety of Computer Driver) requires special attention to avoid the person being used as a moral crumple zone. Two key rules come into play regarding the need for effective driver monitoring and the obligation of a person to intervene when it is reasonable that they would know to do so.

The Awkward Middle for Automated Vehicles: Liability Attribution Rules When Humans and Computers Share Driving Responsibilities
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4444854
https://www.americanbar.org/groups/science_technology/publications/jurimetrics/2024/jurimetrics-fall-2023/

(3) Definitions and Statute Outline: Create a set of definitions and statute-oriented rules to make the first two papers more actionable. We envision this as a robust starting point for state legislatures that find this approach useful.

Liability Rules for Automated Vehicles: Definitions & Detail
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4444848
https://scholar.smu.edu/scitech/vol27/iss1/5/





Wednesday, April 12, 2023

Kia power door pinch recall -- what should be done?

 Sometimes how to handle a safety issue is not so clear. The Kia power sliding door recall is an interesting example of the types of issues that can come up that are not clear-cut.

- Nine confirmed injuries due to minivan power door sliding closed on people, ranging from bruising to a fractured thumb to a broken arm.

- Investigation reveals that system works exactly as designed, noting auto-reverse to prevent injury is a "supplemental" rather than mission-critical safety feature (i.e., best effort is permissible, and their idea of best effort includes a broken arm)

- Investigation results claim all the competitor sliding doors are just as dangerous mechanically (comparable closing force/reversal properties)

- The remedy slows down the final inches of closing and sounds a couple warning beeps when moving the door. That does not sound like a particularly robust fix. It sounds like "we need to do something, how about this..." (And why weren't these in place before?)

- Efforts throughout focus on "customer awareness," mailing customers reminders of functions, etc. Classical playbook of blame the user and "educate" them even harder despite evidence that education isn't working.


I have diverse reactions to this:

- Why did it take a NHTSA recall to get Kia to slow door and add beeps after the broken arm incident? Why didn't they make these changes earlier?

- I note the Kia did not say they compared its human interface to others -- just the closing mechanics. I remember a different brand minivan I owned having beeps for the door and slowing down on closing 20+ years ago, so not rocket science here.

- Hard to believe that beeps will keep kids from reaching in to grab something as the door closes (one of the incidents reported)

- A broken arm is no fun, but it's only a broken arm and only one of them. (My career in the elevator industry started with getting my hand stuck in an elevator door when I was probably 4 years old because I was curious to see how it worked; I remember that incident pretty well...)

- If there is a safety issue across this vehicle class as Kia implies in their report, it is obvious the companies think everything is fine so nothing will change. Should it change? Is there a way to get it to change without NHTSA having to beat up the whole industry (which will likely take more bad things to happen before they have a mandate to do that)? Or is this just that Kia got the human factors wrong and there is no standard/common practice for this.

- Perhaps an industry standard could address this before NHTSA is eventually forced to act. Of course the companies would actually have to follow the industry safety standard, for which they have a poor track record in automotive.


This suggests that the car companies don't have a lot of incentive, resources, inclination, or something to think about the lesser severity safety issues. It is really easy to blame these injuries on children not behaving in a mature way. But it's a minivan -- having kids around a power door is pretty much the whole reason these vehicles exist. It feels like this, and many other less critical safety issues, could be handled better. But that will take a big cultural pivot away from the blame-the-driver situation we have today.


IMHO weak safety culture operating under air cover of cost savings (even when that isn't really a valid excuse) is the big overarching issue. Power door pinches are just another symptom.


Interested in everyone's thoughts on this one...


NHTSA recall 23V236 timeline narrative: https://static.nhtsa.gov/odi/rcl/2023/RMISC-23V236-8289.pdf


Article summary: https://www.autoevolution.com/news/kia-recalls-new-carnival-over-software-issue-automaker-aware-of-nine-confirmed-injuries-213320.html

Tuesday, March 21, 2023

A Liability-Based Regulatory Framework for Vehicle Automation Technology

State liability laws might be the way out of the automated vehicle regulatory dilemma. From phantom braking to reckless public road testing to permitting using human drivers as moral crumple zones, vehicle automation regulation is a hot mess. States are busy creating absurd laws that assign safety responsibility to a non-legal-person computer, while the best the feds can do under the circumstances is play recall whack-a-mole with unsafe features that are deployed faster than they can investigate.

What has become clear is that attempting to regulate the technology directly is not working out. In the long term it will have to be done, but we will likely need to see fundamental changes at US DOT before we see viable regulatory approaches to automated vehicles. (As a start, they need to abandon the use of SAE Levels for regulatory purposes.) That process will take years, and if history is any guide, one or more horrific tragedies before things settle out. Meanwhile, as companies aggressively exploit the "Level 2 loophole" it is the wild west on public roads. Various companies are taking safety with different levels of seriousness, but there is a dramatic lack of transparency and accountability across the industry that will only get worse with time.


As a short- to mid-term approach we should revisit how liability laws work at the state level to buy time to let the technology mature while avoiding needless harm to constituents. There are three fundamental things that have changed that make the current tort system unworkable in practice for automated vehicle technology:

#1: Machine learning-based technology is inherently unsuitable to traditional software safety analysis. The current legal system which puts the burden of showing technology is defective on victims is simply not viable when even the engineers who designed a system can't necessarily explain why the computer driver did what it did.

#2: Asymmetric access to information makes it easy for car companies to know what happened in a crash (or even if automated driving was activated), but it is very difficult for victims to access, much less interpret such information.

#3: The litigation cost of pursuing a claim against software with non-deterministic defects that require source code analysis is huge, depriving all but the largest cases from having an effective ability to prove a product defect claim, if one is justified.


In response to these realities, a (rebuttable) presumption of liability and burden of proof should be shifted to manufacturers in situations for which it is unreasonable to expect a civilian human driver to be able to ensure safety. The attached summary sketches an approach, with more detail to come.


Read the one-pager policy summary here: https://archive.org/details/2023-03-av-liability-one-pager-published-v-1-00



Tuesday, February 14, 2023

Insurance Does Not (and will not) Make AVs Acceptably Safe

I frequently hear arguments that insurance will make autonomous vehicles (AVs) safe. For example, : "the insurance company issued a policy, so the AV must be safe," and "economic pressure from insurance premiums will ensure safety." While it is true that insurance premium pressure (and companies) will mitigate egregiously dangerous AVs, they have nowhere near enough power to enforce safety acceptable to many stakeholders in an industry of risk-takers chasing a trillion-dollar market.

Dall-E picture of baby carriage, car and a dollar sign

Insurance policies do not make you safe

Getting an insurance policy does not mean you are objectively “safe.” You can insure plenty of things that might be considered risky by everyday standards: skydiving injury insurance, commercial rocket launch payload insurance, marine piracy insurance,[1] and life insurance for front-line military personnel are all routinely issued.

An insurance company issuing a policy does not mean any particular activity in general or AV in particular is objectively safe. Rather, it means that the insurance company thinks it understands the risks well enough to set a policy rate that is, on average, economically attractive (profitable to the insurance company) across the members of the risk pool.

Taking a lot of risk? Expect a higher premium. But you will still get an insurance policy for high-risk activities so long as the insurance company feels comfortable it can estimate likely future losses and charge accordingly.

To be sure, the insurance industry does support improving safety. Historically, the insurance industry has spawned activities to create safety standards and help their customers manage risks.[2] Organizations such as the Insurance Institute for Highway Safety (IIHS)[3] are both active and vocal in support of vehicle automation safety. Insurance companies typically have a loss prevention activity to support their clients as well. So these remarks should not be interpreted as indicating industry disregard for safety.

Nonetheless, the economic reality of the situation is that you can get insurance for activities that are objectively dangerous so long as you are willing to pay the required premium. With a multi-billion dollar war chest for AV development and aggressive timelines to deploy, reducing insurance costs is nowhere near the top budget item of concern until well after deployment. Higher insurance premiums are unlikely to drive safety improvements very hard until the industry is operating at significant scale. And even then, other issues discussed regarding risk mitigation incentives will still apply.

Low premiums do not necessarily mean low harm

A crucial aspect of insurance for vehicle crashes is that total cost of insurance bundles together harm to people as well as property damage. For personal policies the contributing costs are broken out into several categories. But if you are tracking business profitability what matters is the total insurance cost, which includes both harm to people and property damage. If the property damage risk far outweighs the risk of harm, there is reduced economic pressure to mitigate harm to people.

An everyday example of divergence between insurance premiums and fatality rates can be seen in motorcycles.

Overall, motorcyclists account for 14% of all traffic fatalities, with those fatalities occurring nearly 27 times more frequently than for passenger cars on a per-mile basis.[4] Even though motorcycles are 27 times more dangerous in terms of risk of death per mile, motorcycle insurance costs about half that of car insurance. This difference is attributed to motorcycle crashes causing much lower equipment damage bills (both to the motorcycle and any other vehicle hit) that more than offset the cost of increased fatalities.[5]

Based on motorcycle insurance alone, it is clear that insurance cost can be a poor prediction of occupant harm because of the influence of property damage on insurance rates. Cheaper insurance does not necessarily mean a vehicle is safer. By the same token, more expensive insurance on an expensive-to-repair vehicle chock-full of crash safety technology does not mean such a high-end vehicle is less safe.

In short, insurance rates are not necessarily predictive of safety.

Insurance premiums will not force acceptable safety

You can perform risk management exclusively by purchasing insurance and doing no risk mitigation whatsoever, so long as you can afford the premiums. People and businesses do precisely that on a regular basis.[6] However, there is economic pressure for sophisticated companies to perform risk mitigation to lower insurance premiums – to a point.

In principle, risk mitigation will lower your insurance premiums, but that might or might not be worth your while. If hypothetically you are spending $1 per mile to run a vehicle and insurance costs $0.05 per mile, the theoretical limit to the benefit of risk mitigation is only 5% of your costs. You might be better off from a purely economic point of view spending management attention on optimizing the other $0.95 per mile of cost. Note that safety does not enter into such an insurance-driven risk management calculation – it is purely about optimizing profits.

Yes, if insurance is 95% of your cost, you have strong incentive to reduce risk. But as insurance cost becomes small compared to other factors, there is less and less pressure to do risk mitigation to further reduce costs. This is especially true if you are in a fast-moving business where things like time to market and ability to scale the fleet quickly are an existential threat to your business vs. a few cents per mile of insurance cost. (If you only have a handful of cars on the road because they don't really work yet, a few cents per mile of insurance cost are simply not on your list of worries.)

In other words, the ability to buy insurance does not mean that an AV is safe, but rather that the insurance company has decided they are willing to get paid a certain amount to cover any potential losses, and the AV maker has decided they can afford to pay that amount while achieving their goals. If you have a company spending more than a million dollars a day on engineering costs, a few dollars extra of insurance cost for test fleets is inconsequential. Perhaps insurance costs will be optimized after deployment, but even then it is economically pressing to do so only when all the much greater business costs have been optimized on a large deployed fleet.

It is unreasonable to expect an AV developer to delay market introduction to improve safety simply to shave a few pennies per mile off their insurance costs. Rather, they will be incentivized to deploy as soon as they can to capture market share – even if they lose money on every mile driven to do so – and worry about incrementally reducing insurance costs later.

Still another consideration is that insurance companies might low-ball quotes to obtain market share. The theory is that if AVs become a big insurance market, it is advantageous for insurance companies to use early policies as loss leaders to in effect “buy” part of the market by establishing early relationships with AV developers. That might mean, for example, that a company could write a policy for an AV tester that is the same rate as for an ordinary vehicle even if the risk might be higher or even largely unknown. Given a fixed payout limit set by a policy cap, the worst-case downside for a crash is limited to that policy cap. The upside is preferential access to a potentially huge future insurance market.

Insurance premiums are further reduced by artificially low insurance policy cap requirements compared to the risk that is likely being taken during testing and early deployment. State laws require insurance maximums to be much lower than reasonably expected jury compensation awards, sometimes no larger than the state insurance minimum requirement for human drivers.[7]

Insurance provides a comparatively weak economic incentive to be “safer.” But as with risk management, that economic incentive runs out of steam when the expected insurance cost becomes small compared to other financial considerations and management imperatives.



[1] Yes, this is a thing even for ships without sails and cannons. 
See:
https://en.wikipedia.org/wiki/Captain_Phillips_(film)

[2] Underwriters Laboratories was founded in the 1890s to help improve fire safety in partnership with the insurance industry. See: https://www.ul.com/about/history

[5]Source: https://policyscout.com/auto-insurance/learn/motorcycle-insurance-vs-car-insurance A possible confounder is number of miles per year driven by motorcycles being less than for cars. But that is unlikely to explain the entire difference here.

[6] If you doubt this, the next time you are at a rental car counter during a quiet shift, take a few minutes to ask the attendant for their worst horror story of damage carelessly or perhaps even intentionally done to a vehicle for which someone purchased the expensive zero deduction insurance waivers.

[7] State insurance requirements for AVs range from state minimums of $25K to $50K up to a high of $5M. Even if a court verdict is higher, the insurance company is only on the hook for the maximum, with the rest of any potential liability falling back on responsible parties – if the plaintiff can manage to collect. For a list of state insurance requirements see:       
https://www.iihs.org/topics/advanced-driver-assistance/autonomous-vehicle-laws