Safe Autonomy

(This is the expanded version of: Philip Koopman, " Autonomous Vehicle Myths: The Dirty Dozen ," EE Times, Oct. 22, 2021.) Too often, I’ve read documents or listened to panel sessions that rehash misleading or just plain incorrect industry talking points regarding autonomous vehicle standards and regulations. The current industry strategy seems to boil down to “Trust us, we know what’s best,” “Don’t stifle innovation,” and “Humans are bad drivers, so computers will be better.” As far as I can tell, what’s really going on is that automated-vehicle companies are saying what they say both to avoid being regulated and to avoid having to follow their own industry safety standards. That strategy has not yielded long-term safety in other industries that have tried it, however, and I predict that in the long term it will not serve the automotive industry well either. It certainly does not encourage trust.  In this essay, I address the usual industry talking points and provide summary

Once in a while I'm contacted by a city or state Department of Transportation (DOT) to provide advice on safety for "self-driving" car testing. (Generally that means public road testing of SAE Level 3-5 vehicles that are intended for eventual deployment as automated or autonomous capable vehicles,) The good news is that industry standards are maturing. Rather than having to create their own guidelines and requirements as they have in the past, DOTs now have the option of primarily relying upon having AV testers conform to industry-created guidelines and consensus standards. And ... in September 2021 NYC DOT blazed a trail by requiring the self-driving car industry to conform to their own industry consensus testing safety standard (J3018). Kudos to NYC DOT!  (check it out here ( link ); more on that in the details below. The #1 important thing to keep in mind is that testing safety is not about the automation technology -- it is about the ability of the human safety driver

Since this blog was written, I teamed up with William Widen and did an deeper analysis of what Tesla told CA DMV and the relevant CA regulations. Conclusion: Our analysis indicates that Tesla FSD beta is SAE Level 4.  This builds upon the previous blog post below, but the article in the JURIST and associated SSRN article supersede the below, which is preserved only for historical context. https://www.jurist.org/commentary/2021/09/william-widen-philip-koopman-autonomous-vehicles/ (update: Oct 1, 2021) Original article is below. ---------------------------------------------------------------------------- The technical crux of the Tesla "Full Self Driving" naming and marketing dilemma is SAE J3016 Section 8.2  (J3016 is the standard that defines the Levels.) If the design intent for the Tesla FSD feature is to eventually operate without a human driver being required for safety, per SAE J3016 that makes it SAE Level 4 ("L4") -- even if today a human driver is required t

SAE J3016 defines vehicle automation levels, but is not a safety standard (nor does it claim to be). Levels 2 & 3 are especially problematic from a safety point of view. What they define if the standard is followed -- and no more -- is unlikely to provide acceptable safety in practice. To be clear: a vehicle said to be SAE Level 2 or SAE Level 3 might be safe. But if it only does the bare minimum required for J3016 conformance, it is unlikely to be safe. More is needed. (For more on the specifics of SAE J3016 Levels see this user guide (link)  including a detailed discussion of what is and is not required by the SAE Levels.) SAE Level 2 safety SAE Level 2 requires that the driver be responsible for the Object and Event Detection and Response (OEDR). The driving automation might or might not see some objects, and might or might not respond properly, thus requiring continuous driver vigilance. However, it is well known that human drivers do poorly at supervising automation. Paradoxi

The SAE J3016:2021 standard ( https://www.sae.org/standards/content/j3016_202104/ ) defines terminology for automated vehicles including the famous SAE Automation Levels. It is widely referenced in discussions, other standards, and even government regulations. Unfortunately, what is said about J3016 is too often inaccurate, misleading, or just plain incorrect. Misinterpreting the SAE Levels can lead to misunderstandings about what the standard actually says, the technology incorporated into a car, and a driver's expectations. It's important to get statements in standards and regulations right. Moreover, it's important when referring to J3016 to understand that it says what it says, not what some author might want it to say, what might seem optimal for safety, or what other documents state that it says. (While this might seem obvious, perpetuation of misunderstandings is rampant.) You can read the full user guide that explains the standard, its implications and debunks myths

 SAE J3016 Automation Levels are widely used, controversial -- and arguably not the right way to describe automation for anyone other than the engineers designing the systems. In the video below I give a summary of SAE J3016 terminology and the infamous Levels and provide an alternative description approach. I also cover some of the many myths that so many people think are true, but which are not actually part of that standard at all. I have also published a pretty thorough SAE J3016 User Guide that details more precise definitions and myths to help those who need to get things exactly right find the subtleties in the standard that are not apparent after only one (or two, or three) reads. User guide:   https://users.ece.cmu.edu/~koopman/j3016/ Terminology video: