Saturday, October 28, 2023

A Snapshot of Cruise Crash Reporting Transparency: July & August 2023

A comparison of California Cruise robotaxi crash reports between the California DMV database and the NHTSA SGO database reveals significant discrepancies in reporting. 31 crashes reported to NHTSA do not appear in the California DMV database. This includes seven unreported injury crashes. Of special note is the Cruise crash with a fire truck that caused serious injury to an occupant of the Cruise robotaxi does not appear as a California DMV crash report. To be sure, Cruise might not be legally required to file these reports, but the situation reveals an apparent lack of transparency.

Comparison Results:

39 crashes were identified across both databases for the date-of-crash months of July 2023 through August 2023. The comparison was performed on October 28, 2023, so there was adequate time for all such crashes to have been reported.

Each database was missing one or more crashes found in the other database:

  • 39 crashes in the NHTSA base, including 8 also found in the CA DMV database.
  • 31 crashes reported to NHTSA were not in the California DMV database
  • The California DMV database was in particular missing SEVEN (7) crash reports which indicated an injury had occurred or might have occurred.
  1. NHTSA 30412-5968: Other car ran a red light striking Cruise; passenger of other vehicle treated on scene for minor injury.
  2. NHTSA 30412-5982: Other car ran into Cruise; passenger of other vehicle transported by EMS for further evaluation. Possible injury ("unknown" injury status).
  3. NHTSA 30412-6144: Cruise crash with fire truck; serious injury reported to passenger
  4. NHTSA 30412-6145: Cruise reversing contacted cyclist; minor injury reported to cyclist
  5. NHTSA 30412-6167: Cruise rear-ended after braking; minor injury reported to other vehicle driver
  6. NHTSA 30412-6175: Cruise hit pedestrian crossing in front of it (said to be crossing against light); moderate injury to pedestrian
  7. NHTSA 30412-6270: Cruise hit from behind after stopping to yield to pedestrian in crosswalk; minor injury to passengers inside AV
Two crashes involved non-motorists:
  • 30412-6145 with a cyclist
  • 30412-6175 with a pedestrian

        Why the Disparity?

        The thing that makes this complicated is that CA DMV does not require reporting crashes for "deployment" operation -- just for "testing" operation. Apparently when the regulations were written they did not anticipate that companies would "deploy" immature technology, but that is exactly what has happened.

        It is difficult from available information to check how Cruise is determining which crashes must be reported to California DMV (testing) and which do not have to be reported (deployment). In practice it might boil down to a management decision which ones they want to report, although there might be some less arbitrary internal decision criterion in use.

        CA DMV should require all companies to provide them with unredacted copies of all NHTSA SGO reports to provide improved transparency. For the foreseeable future, making a distinction between "testing" and "deployment" with no driver in the vehicle serves no useful purpose, and impairs transparency. If there is no driver it is a deployment, and should be held to the standards of a production vehicle, including reporting both crashes and driving behavior that puts other road users at undue risk. This is true for all companies, not just Cruise.

        Other notes:

        • CA DMV reports have the street names, yet Cruise redacts this same information from reports filed with NHTSA claiming it is "confidential business information." It is difficult to understand how information publicly reported by California can be classified as "confidential."
        • The NHTSA database does not have the date of the crash, although the California database has that information.
        • Crashes considered were for reported incident dates of July & August 2023, considering only uncrewed (no safety driver) operation.
        • It is our understanding that Cruise is not required to report all crashes that occur during deployment to California DMV. So it is possible that these reporting inconsistencies are still in accordance with applicable regulations.
        • All crashes on this spreadsheet in the NHTSA database list the "driver/operator type" as "Remote (Commercial / Test)" so it is not possible to distinguish whether the vehicle was considered in commercial service at the time of the crash. 
        • At the time of this posting the tragic Oct. 2nd severe injury crash that involved a Cruise robotaxi dragging a pedestrian who had been trapped under the vehicle has also not been reported, while another crash on Oct 6th has. There is nothing on the Oct 6th CA DMV form to indicate that the reported crash was specific to a testing permit vs. deployment permit.

        Review status: 

        This data has not been peer reviewed. Corrections/additions/clarifications are welcome to improve accuracy. The data analysis results are included below.

        Google Spreadsheet link:  https://docs.google.com/spreadsheets/d/1o9WWzMpiuum-QHZk9goY68gnBZuRC1InxUSMa7-h4DU/edit?usp=sharing

        Data sources: 






        Updated 10/30/2023 to incorporate three more crash reports found in a wider search of the SGO database. All CA DMV crash reports have now been identified in the SGO database.






        Friday, October 27, 2023

        The Cruise Safety Stand-Down -- What Happens Next?

        Cruise has announced a fleet-wide safety stand-down. This involves suspending driverless operations in all cities, reverting to operation only with in-vehicle safety drivers.

        I'm glad to see this step taken. But it is crucial to realize that this is the first step in what it likely to prove a long journey. The question is, what should happen next?


        Loss of public trust is an issue as they say. And perhaps there was an imminent ban in another state forcing their hand to be proactive. But the core issues almost certainly run deeper than mismanaging disclosure of the details of a tragic mishap and doing damage control with regulatory trust. 

        The real issues will be found to have their roots in the culture of the company. Earnest, smart employees with the best of intentions can be ineffective at achieving acceptable safety if the corporate culture undermines their official company slogan of "safety first, always."

        This is the time to ask the hard questions. The answers might be even harder, but they need to be understood for Cruise to survive long term. It is questionable whether they could survive a ban in another state. But escaping that via a stand-down only to implement a quick fix won't be enough. If we see business as usual restored in the next few weeks, that is almost certainly a shallow fix. It will simply be a matter of time before a future adverse situation happens from which there will be no recovery. 

        This is the moment for Cruise to decide to lean into safety.

        The details are too much for a post like this, but the topics alone indicate the scope of what has to be considered:
        • Safety engineering -- Have they effectively identified and mitigated risks?
        • Operational safety -- Safety procedures, inspections, maintenance, management of Operational Design Domain limits responsive to known issues, field data feedback, etc.  This includes ensuring their Safety Management System (SMS) is effective.
        • System engineering -- Do the different pieces work together effectively? This includes all the way from software in 3rd party components to vehicle integration to ability of remote operators to effectively manage gaps in capabilities ... and more
        • Public messaging and regulatory interface -- Building genuine trust, starting with more transparency. Stop the blame game; accept accountability. Own it.
        • Investor expectations -- Determine a scaling plan that is sustainable, and figure out how to fund it in the likely case it is longer than what was previously promised
        • Definition of acceptable safety -- More concrete than seeing how it turns out based on crash data, with continual measurement of predictive metrics
        • Safety culture -- Which underlies all of the above, and needs to start at the top.
        And I'm sure there are more; this is just a start.

        Near-term, the point of a safety stand-down is to stabilize a situation during uncertainty. The even more important part comes next: the plan to move forward. It will take weeks to take stock and create a plan, with the first days simply used to organize how that is going to happen. And months to execute that plan. Fortunately for Cruise there is an existing playbook that can be adapted from Uber ATG's experience with their testing fatality in 2018. Cruise should already have someone digging into that for initial ideas.

        An NTSB-style investigation into this mishap could be productive. I think such an investigation would be likely to bring to light new issues that will be a challenge to the whole industry involving expectations for defensive driving behaviors and post-crash safety. If NTSB is unable to take that on, Cruise should find an independent organization who can do something close. But such an investigation is not the fix, and cultural improvements at Cruise should not wait for one to conclude. However, an independent investigation can be the focal point for deeper understanding of the problems that need to be addressed.

        ------------------------------------------------

        Philip Koopman is a professor at Carnegie Mellon University in Pittsburgh Pennsylvania, USA, who has been working on self-driving car safety for more than 25 years.   https://users.ece.cmu.edu/~koopman/

        Wednesday, October 25, 2023

        My Talk at a SF Transportation Authority Board meeting

        On October 24th I had the privilege of testifying at a San Francisco CTA board meeting regarding autonomous vehicle safety. The session started with SFCTA and SFMTA describing the challenges they have had with robotaxis, my talk, and then a series of Q&As about additional relevant topics. The robotaxi companies were invited to participate but declined. The news of the CA DMV suspension of Cruise permits broke as the hearing was starting, adding some additional context to the hearing.

        We have all heard the hype and marketing from the robotaxi companies. This event gave air time to the challenges faced by cities that must be resolved for this technology to succeed at scale.

        • Information about the event:  (link)
        • Full hearing video:  (link
          • Start at 33:48 (click on item 11 in the index to fast forward to that point) for the full segment including SFCTA and SFDMV.
          • My talk starts at 1:10:25
        • My talk only, live recording:  Youtube | Archive.org
        • Slides for my talk (link to acrobat)
        The robotaxi companies were invited, but declined to participate in this event. You can see what they have to say from the transcript of a CPUC status conference held on August 7, 2023: https://docs.cpuc.ca.gov/PublishedDocs/Efile/G000/M517/K407/517407815.PDF

        Direct link to YouTube talk:





        Saturday, October 21, 2023

        Safety Analysis of Two Cruise Robotaxi Pedestrian Injuries

        Cruise has now had two pedestrian injuries in San Francisco, with the more severe one being complicated because it involved a pedestrian first hit by another vehicle.  NHTSA has launched an investigation based on those injuries and at least two other public video reports of close encounters. This makes available the relevant crash reports, so we have more direct information about what happened. The question asked in this piece is what can be done to avoid similar crashes in the future.


        On a numbers basis, two pedestrian injuries in a span of fewer than six weeks for a fleet of a couple hundred vehicles in San Francisco is a concern, so this is worth some analysis based on available information.

        • First injury: Aug. 26, 2023.  A pedestrian stepped off the curb into a crosswalk right in front of a Cruise vehicle at the change of a traffic light. The Cruise swerved, then braked. Impact at 1.4 mph. Pedestrian transported by EMS.
        • Second injury: Oct 2, 2023. A pedestrian crosses on the opposite side of a cross-street in front of the Cruise vehicle and another vehicle next to it. Both vehicles proceeded through the intersection as a pedestrian was in a crosswalk across their paths. The other vehicle struck the pedestrian at an undisclosed speed, who was then run over by the Cruise vehicle and trapped under it with severe injuries.
        In both cases the injuries were severe enough to require transport. For the second crash the pedestrian was almost completely underneath the rear of the vehicle.  (It is worth noting these descriptions are written 100% by Cruise. The reader should assume the most favorable-to-Cruise possible interpretation of events has been presented. If something obviously relevant is omitted, such as the impact speed for the second injury, one is justified in assuming it would be unfavorable to Cruise if disclosed.)

        Cruise, predictably, blames others for both crashes, although in both cases without review of the video it is difficult to be sure that is really true. However, we set blame aside and instead ask the question: what can be done to avoid the next pedestrian injury in similar circumstances.

        First Pedestrian Crash


        For the first crash, the question is whether a reasonable human driver would have had contextual clues that this pedestrian was about to enter the crosswalk even though the light had changed. For example, were they running to catch a bus pulling up to a stop across the street?  Were they "distracted walking?" Or were they at a complete stop on the curb and literally jumped out into the street? Opportunities for improvement include asking these questions:
        • Were there obvious contextual clues that the pedestrian would attempt a last second crossing? What are common cases, and are they covered by the Cruise AV design?
        • Why did the vehicle swerve before stopping instead of doing both at once?
        • Could/should the Cruise vehicle have followed a less aggressive acceleration profile given the likely risk of a pedestrian entry into the crosswalk in that type of circumstance?

        Second Pedestrian Crash


        For the second crash, things are more complicated. Let's break down the sequence, taking into account the initial setup sketched below (note that both vehicles are in the middle of an intersection, but the sketch tool I used did not make this easy to represent):

        1. There are two vehicles starting through an intersection, side by side, with two lanes in that direction of travel. From a top view the other, human driven, dark-colored vehicle is on the left (faster lane) and the lighter-colored Cruise is on the right (curb lane).
        2. A pedestrian is walking across the far side of the intersection in the crosswalk. At the same time, both vehicles accelerate into the intersection. The most likely situation is the Cruise vehicle was a bit behind the other vehicle (although this is an educated guess based on the description of the events).
        3. Cruise says the pedestrian entered the crosswalk after the light changed, crossed in front of the Cruise vehicle, then stopped in the other vehicle's lane. The other driver presumably thought the pedestrian would clear the travel lane in time, and did not slow down.
        4. The other vehicle hit the pedestrian. Cruise says the pedestrian was deflected back into the Cruise vehicle's lane.
        5. The Cruise vehicle "braked aggressively" in response to a surprise pedestrian appearing in its lane, but hit the pedestrian shortly after.
        6. The Cruise vehicle had sufficient forward speed that it ran over the pedestrian and came to a stop with the pedestrian trapped under the rear axle. Both of the pedestrian's feet protruded from under the vehicle by the left rear tire, with that tire on top of one leg. (Photo link below.)
        7. The pedestrian was severely injured by a combination of the two vehicle strikes. Information about the ultimate outcome for that pedestrian is not currently available, although we hope that a recover is quick and as complete as possible.

        California Rules of the Road have an interesting requirement for crosswalks:

        "(c) The driver of a vehicle approaching a pedestrian within any marked or unmarked crosswalk shall exercise all due care and shall reduce the speed of the vehicle or take any other action relating to the operation of the vehicle as necessary to safeguard the safety of the pedestrian."  (emphasis added)

        It is interesting to ask if the Cruise vehicle actually exhibited "all due care."  It likely did not reduce speed from its normal green light acceleration, or Cruise would have taken credit for having done so.  (If they want to provide more details I will gladly update this statement.)

        Of note is the Cruise position that their vehicle stopped as quickly as possible once the pedestrian was in their lane, in effect claiming the collision was unavoidable. But that position is not necessarily true in the larger context, especially if one learns from this crash for the next potential pedestrian crosswalk collision. The question is when the Cruise AV could have stopped. There are at least three possible decision points for stopping to avoid this collision with the pedestrian, and the Cruise vehicle appears not to have exercised the first two:

        • The light changes green, but there is a pedestrian still in the crosswalk in the Cruise vehicle's direction of travel in front of the Cruise vehicle. Did it slow down?  Or execute a normal acceleration because it predicted the pedestrian would be clear by the time it got there?   A prudent human driver would have waited, or more likely crept forward while waiting to signal cars behind it not to honk for failing to recognize a green light.
        • The pedestrian clears the Cruise lane, but the Cruise vehicle clearly sees the pedestrian about to be hit by the adjacent vehicle. The Cruise vehicle could have (I would argue should have) stopped to avoid being close to an injury event. Expecting it to predict a pedestrian collision trajectory is asking a lot -- but it should have stopped precisely because it cannot predict what will happen after such a collision. Safety demands not going fast past a pedestrian who is about to be hit by another vehicle in an adjacent lane. But this is precisely what the Cruise vehicle did.
        • The pedestrian lands in the Cruise lane and the Cruise vehicle has not slowed down yet. By then it is too late, and it runs over the pedestrian.  This could likely have been avoided by a prudent driving strategy that addresses the previous two decision points.

        The Redacted Confidential Business Information

        (This section added October 25, 2023 based on new information.)

        California DMV issued an order suspending the driverless operating permits for Cruise robotaxis on October 24, 2023 as a response to the circumstances of this second crash.  Link to order here.

        This order brought to light that after the vehicle had stopped post-crash, it started movement again with the pedestrian still under the vehicle, dragging that victim about 20 feet at a speed up to 7 mph, which was said to contribute to severe injuries. This strongly suggests the vehicle did not account for a pedestrian being trapped underneath it when deciding to move. (It is possible a remote operator was unaware of the trapped pedestrian and remotely commanded a pull-to-side maneuver. We'll have to see what is revealed during any investigation.)

        Cruise also published a blog post with additional information that day. A straightforward update to the crash report is to add at the end as at least part of the "redacted confidential business information" the following (quoted from the Cruise blog post): 
        "The AV detected a collision, bringing the vehicle to a stop; then attempted to pull over to avoid causing further road safety issues, pulling the individual forward approximately 20 feet."

        This certainly makes Cruise look bad, but that is not an acceptable reason for a redcation. It is difficult to understand how this can reasonably be characterized as "confidential business information" in a mandatory crash report.

        Calling Emergency Services

        Also crucial for practical safety, but barely talked about, is notification of emergency services ("call 911"). News reports indicate that a passer-by called 911, not Cruise. In fact, in neither collision report do they take credit for notifying emergency services. This is a glaring omission that needs to be addressed.

        Consider: they had a vehicle tire on top of a pedestrian's leg and did not call 911. (Again, if this is incorrect I will update this statement when I get that information.) That's a HUGE problem nobody is talking about. A human driver would have realized they just ran someone over and either called 911 or asked someone to do so. If there had been no passer-by, how many minutes would that pedestrian have been trapped under the car before help was summoned?

        The Cruise AV and its support team need to realize an injury has happened and take immediate action. It would be no surprise if the remote operators had no idea what the vehicle had run over. By the time they download and review video logs (or whatever) that pedestrian has been trapped under the vehicle for a while. That's not acceptable. They need to be able to do better.

        Cruise Safety Record

        The first pedestrian injury happened just over two weeks after the August 10th California PUC meeting that granted operating permits to Cruise. That report was overshadowed by the crash apparently due to failure to yield to a fire truck on August 17th. That night also saw another injury involving a collision to a different vehicle driver.  So we are seeing a steady stream of injuries.

        Cruise blames crashes on other parties to the maximum degree possible, and ignores injuries where it is less than 50% at fault (there have been others; notably a very ill-advised left turn maneuver by a Cruise robotaxi that resulted in multiple injuries).  Safety is not achieved by blaming others. If Cruise vehicles are crashing and injuring people more often than other vehicles, then that is an increased rate of injury regardless of blame.

        A company with a responsible safety culture would be asking what they can do to reduce the risk of future injuries -- regardless of blame. We will have to wait to see the outcome of this NHTSA investigation, and whether Cruise proactively improves safety or waits for NHTSA to force the issue.

        As a note to likely responses to this analysis: comparisons to human driver errors are not productive. Indeed, another driver hit the pedestrian first in the second crash. But another driver being negligent does not forgive imprudent driving behavior from a robotaxi that is being relentlessly touted as safer than human drivers. They should be continuously improving, and our hope is that this analysis highlights areas that they and other robotaxi companies need to improve.

        Supporting Information


        Thursday, October 5, 2023

        Defining a Computer Driver for Automated Vehicle Accountability

        This 25-minute video talks about the concept of a Computer Driver to resolve the ambiguity and industry tapdancing we're seeing about who might be negligent for a crash when automation is driving the car. This creates a framework for both SAE Level 2 and Level 3 features that makes it explicit which of the computer driver or the human driver has the duty of care for safety at any given time.

        Topics:

        • Why product liability is not enough to get the job done
        • Tort law for engineers
        • Assigning a duty of care to promote accountability
        • Implications of defining a computer driver
        • Duty of care for conventional, fully autonomous, and testing features
        • The awkward middle: transferring the duty of care for supervisory mode
        • The urgency of defining a computer driver while we wait for longer-term regulations
        Materials:

        YouTube Video