I'm presenting a paper on AV safety validation at the 2018 SAE World Congress. Here's the unofficial version of the presentation and a preprint of the paper.
Toward a Framework for Highly Automated Vehicle Safety Validation
Philip Koopman & Michael Wagner
2018 SAE World Congress / SAE 2018-01-1071
Validating the safety of Highly Automated Vehicles (HAVs) is a significant autonomy challenge. HAV safety validation strategies based solely on brute force on-road testing campaigns are unlikely to be viable. While simulations and exercising edge case scenarios can help reduce validation cost, those techniques alone are unlikely to provide a sufficient level of assurance for full-scale deployment without adopting a more nuanced view of validation data collection and safety analysis. Validation approaches can be improved by using higher fidelity testing to explicitly validate the assumptions and simplifications of lower fidelity testing rather than just obtaining sampled replication of lower fidelity results. Disentangling multiple testing goals can help by separating validation processes for requirements, environmental model sufficiency, autonomy correctness, autonomy robustness, and test scenario sufficiency. For autonomy approaches with implicit designs and requirements, such as machine learning training data sets, establishing observability points in the architecture can help ensure that vehicles pass the right tests for the right reason. These principles could improve both efficiency and effectiveness for demonstrating HAV safety as part of a phased validation plan that includes both a "driver test" and lifecycle monitoring as well as explicitly managing validation uncertainty.
Paper Preprint: http://users.ece.cmu.edu/~koopman/pubs/koopman18_av_safety_validation.pdf
Fly-Fix-Fly Antipattern: Brute force mileage accumulation to fix all the problems you see on the road won't get you all the way to bein...
The UK already has a pretty good answer for how to do self-driving car testing safely. US stakeholders could learn something from it. ...
UPDATE 8/17 -- added presentation slides! I'm very pleased to share a publication from our NREC autonomy validation team that explains...
This is the slide version of my op-ed on how to make self-driving car testing safe. The take-away is create a test vehicle with a written ...