ANSI/UL 4600 Version 2 (2022)

Version 2 of ANSI/UL 4600 has just been issued. This standard provides guidance on how to ensure that autonomous vehicles safety cases are created and maintained to ensure acceptable safety for deployment.

Since version 1 of the standard was issued in April 2020, the Standards Technical Panel members (the voting committee) and stakeholders have been involved in suggesting clarifications, upgrades, and other improvements as part of the standard's continuous improvement process. 

Version 1 of the standard included chapters on: terminology, safety cases, risk assessment, interaction with humans, autonomy functions, software/system engineering processes, dependability, data/networking, verification/validation/test, tool qualification/COTS/legacy components, lifecycle concerns, maintenance, metrics, and assessment. The standard is designed to work with other safety standards such as ISO 26262 and ISO 21448 to make sure that all the bases are covered for system-level safety on autonomous road vehicles.

Version 2 has some substantive changes:

• The assessment terminology and role of independent assessment have been changed. This is compatible with the previous approach except that the independent assessor has a more substantive role.
• Self-assessment: the development team creates and vets its own safety case.
• Independent assessment: an independent organization examines both the form and the technical substance of the safety case to ensure it is acceptable.  (This independent organization might be within the same company at an arms-length relationship, or could be an external assessor at the company's option.)
• Significant clean-up of the discussion of safety case terminology and structure. This is overall a significant improvement of the ideas that were already there in version 1, but a lot of work went into this area.
• The terminology section has been substantially rewritten to clean up wording and improve alignment with other standards without substantively changing the intended meaning of terms being used.

Version 2 maintains the same structure as Version 1 with some minor changes to top level requirements. Significant attention has been paid to ensure a smooth transition between versions. While much of the document has been updated, the updates beyond the substantive changes tend to be relatively minor in scope and are more in the nature of clarification and adding helpful detail. The changes significantly improve the standard, but the vast majority do not fundamentally change its nature or general requirements.

Overall there were seven different task groups whose members spent many hours contributing to improving the standard, and there are important changes in each area to improve the standard.  Those task groups drafted proposals to modify content regarding: safety cases, faults/hazard/risks, assessment, sensor requirements, object tracking, safe egress, and terminology. 

Everyone who participated deserves a big "Thank You!"  Also, a special thanks to Deborah Prince and Heather Sakellariou at Underwriters Laboratories for coordinating all the activities and keeping everything on track, especially with the challenges presented by the whole process being done electronically via e-mail and on-line meetings. It has truly been a pleasure to work with a group that is so dedicated to this collaborative effort to ensure the safety of autonomous vehicle technology.

The official kickoff of UL 4600 version 3 is planned for April, with the biggest goal being extending the standard to cover special considerations that apply to autonomous heavy trucks. Anyone interested in contributing to that discussion can request stakeholder status (no participation fee required).

You can find a pointer to a copy of the newly released version 2 including FREE (with free digital account) digital access to a full copy of the standard, a list of committee members, and more here: https://users.ece.cmu.edu/~koopman/ul4600/index.html 

The official news release from the standards organization is here: https://ul.org/standards-and-engagement/presenting-standard-safety-evaluation-autonomous-vehicles/second-edition 

Maturation path for safety & security practices

Brief informal notes from a wrap-up quick position statement talk I did at a workshop today.

Both safety and security have a lot in common in terms of how they are maturing over time. Without getting into a religious debate about the difference between them, I note that their trajectory seems to include the following steps, especially for autonomous systems. I'd argue that each step is in a sense more mature than the previous step.

1. Get the system to work. Safety/security can come later.
2. Get the system to work almost all the time. Conflate this with safety/security even though you're still really just getting it to work in the common cases (safety for a vehicle is "doesn't hit stuff" while security is "doesn't get taken down by the usual continuous stream of automated attacks")
3. Brute force problem fixes:   fly/crash/fix/fly (air) and drive/crash/fix/drive (ground)
4. Create a set of best practices in the nature of a building code ("build your system this way")
• Create a useful fiction that you have completely characterized the requirements and operational environment and that your building code will always work.
• Any failure is an embarrassing piece of bad news that violates the fiction of complete understanding.
5. As system matures, complain about false alarm safety/security shutdowns
• It might feel like this means your system has problems, but in fact you're a lot more safe and secure than systems that operate oblivious to their vulnerabilities
6. Start permitting breaking the building code standard rules by arguing that exceptions still result in equivalent safety/security
7. Evolve to full-up deductive assurance cases to argue safety/security beyond building codes
• Still maintain the fiction of complete knowledge of requirements and environment
8. Start operating in more open environments and admit you didn't really understand requirements, nor environment
• Spend a lot of time chasing down problems that reveal defects in your safety case (safety case does not match environmental assumptions, or might not even match deployed system)
9. Switch to an inductive safety case approach:
• Account for risk from epistemic uncertainty (unknown unknowns)
• Instrument system for failure precursors (e.g., safety performance indicators tied to safety case claims)
• Treat incidents as an opportunity to fix problems before there is a loss event.