Posts

Showing posts from May, 2022

A gentle introduction to autonomous vehicle safety cases

Image
I recently ran into this readable article about AV safety cases by Thomas & Vandenberg from 2019. While things have changed a bit, it still is a reasonable introduction for anyone asking "what exactly would an AV safety case look like." A real industry-strength safety case is going to be complicated in many ways. In particular, there are many different approaches for breaking down G1 which will significantly affect things. On the other hand all the pieces will need to be there somewhere, so choosing this high level breakdown is more of an architectural choice (for the safety case, not necessarily the system). We do not yet have a consensus on an optimal strategy for building such safety cases, but this is not a bad starting place from safety folks who were previously at Uber ATG. Thomas & Vandenberg, Harnessing Uncertainty in Autonomous Vehicle Safety, Journal of System Safety, Vol. 55, No. 2 (2019) https://doi.org/10.56094/jss.v55i2.46 (Uber ATG also published a  muc

SEAMS Keynote talk: Safety Performance Indicators and Continuous Improvement Feedback

Image
Abstract: Successful autonomous ground vehicles will require a continuous improvement strategy after deployment. Feedback from road testing and deployed operation will be required to ensure enduring safety in the face of newly discovered rare events. Additionally, the operational environment will change over time, requiring the system design to adapt to new conditions. The need for ensuring life critical safety is likely to limit the amount of real time adaptation that can be relied upon. Beyond runtime responses, lifecycle safety approaches will need to incorporate significant field engineering feedback based on safety performance indicator monitoring. A continuous monitoring and improvement approach will require a fundamental shift in the safety world-view for automotive applications. Previously, a useful fiction was maintained that vehicles were safe for their entire lifecycle when deployed, and any safety defect was an unwelcome surprise. This approach too often provoked denial and

ICSE keynote: Autonomous Vehicles and Software Safety Engineering

Image
Abstract: Safety assurance remains a significant hurdle for widespread deployment of autonomous vehicle technology. The emphasis for decades has been on getting the technology to work well enough on everyday situations. However, achieving safety for these life-critical systems requires more. While safety encompasses correct operation for the mundane, it also requires special attention to mitigating the risk presented by rare but high consequence potential loss events. In this talk I'll cover some history of autonomous vehicle development and safety at the Carnegie Mellon National Robotics Engineering Center that led over the years to the development of the ANSI/UL 4600 standard for autonomous vehicle safety. I'll also touch upon activities specific to safety engineering, why a heavy tail distribution of rare events makes ensuring safety so difficult, why brute force road testing won't ensure safety, and the emergence of safety assurance cases as the approach of