The effect of AV company business models on safety

The business model and exit plan for an AV company can powerfully incentivize behavior that is at odds with public safety and transparency. This is probably not news regarding any private company, but it is especially a problem for AV safety.

An AV developer with a plan to develop, deploy, and long-term sustain their technology should be incentivized to reach at least some level of safety subject to all the ethical issues discussed already in this chapter. If they do not, they will probably not have a viable long-term business. Arguments for a light regulatory touch often make this argument that companies will act in their own long-term best interest. But what if the business incentive model is optimized for something shorter than the “long-term” outcomes?

Short-term aspects of the business objectives and the business structure itself can add pressure that might tend to erode any commitment to acceptable safety. Factors include at least the following, several of which can interact with each other:

·        Accepting money from traditional venture capital sources can commit a company to a five-year timeline to produce products. Thus far we have seen that five-year timelines are far too aggressive to develop and deploy an AV at scale. Re-planning and raising more funding can lengthen the timeline, but there remains risk that funding incentivizes aggressive milestones to show increased functionality and, in particular, remove safety drivers rather than core efforts on safety. Some companies will likely be better at resisting this pressure than others.

·        A business exit plan of an Initial Public Offering (IPO), going public via a Special Purpose Acquisition Company (SPAC), or being bought out by a competitor historically emphasize perceived progress on functionality rather than safety. If the exit plan is to make safety someone else’s problem post-exit, it is more difficult to justify spending resources on safety rather than functionality until the company goes public.[1]

·        The AV industry as a whole takes an aggressively non-regulatory posture, with that policy approach historically enabled by US DOT.[2] This situation forces little, if any, accountability for safety until crashes happen on public roads. There is a tendency for at least some companies seem to treat safety more as a public relations and risk management function than a substantive safety engineering function. Short-term incentives can align with a dysfunctional approach.

·        Founders of AV companies with a primarily research, consumer software, or other non-automotive background might not appreciate what is involved in safety at scale for such systems. They might earnestly – but incorrectly – believe that when bugs are removed that automatically bestows safety, or otherwise have a limited view of the different factors of safety discussed in chapter 4. They might also earnestly believe some of the incorrect talking points discussed in section 4.10 regarding safety myths promoted by the AV industry.

·        The mind-boggling amount of money at stake and potential winnings for participants in this industry would make it difficult for anyone to stay the course in ensuring safety in the face of rich rewards for expediency and ethical compromise. No matter how pure of spirit and well intentioned.

It is impossible to know the motivations, ethical framework, and sincerity of every important player in the AV industry. Many participants, especially rank and file engineers, are sincere in their desire to build AVs and believe they are helping to build a better, safer world. Regardless of that sincerity, it is important to have checks and balances in place to ensure that those good intentions translate into good outcomes for society.

One has to assume that outcomes will align with incentives. Without checks and balances, dangerous incentives can be expected to lead to dangerous outcomes. Checks and balances need to be a combination of internal corporate controls and government regulatory oversight. A profit incentive is insufficient to ensure acceptable safety, especially if it is associated with a relatively short-term business plan.

---

[1] Safety theater money spent to impart an aura of safety is a different matter, and spending on this area can bring good return on investment. But we are talking about real safety here. In the absence of a commitment to conform to industry safety standards it can be difficult to tell the difference without a deep dive into company practices and culture.

[2] The official US Department of Transportation policy still in effect at the time of this writing states: “In this document, NHTSA offers a nonregulatory approach to automated vehicle technology safety.” See page ii of:            https://www.nhtsa.gov/document/automated-driving-systems-20-voluntary-guidance

This is an adapted excerpt (Section 10.5) from my book: How Safe is Safe Enough? Measuring and Predicting Autonomous Vehicle Safety