Our AV investigations led us to several key findings and lessons learned, some of which were common across most of the investigations. As a result of these investigations, we also issued several new safety recommendations to a variety of organizations, including manufacturers, government, and academic.
The Need for Continuous Driver Monitoring
Vehicles with partial automation capabilities are the highest level of automation that the public can purchase today. The partial automation systems are intended to assist drivers. Although they can provide both longitudinal (acceleration and deceleration) and lateral (steering) control, they have limited perceptual capabilities and require continuous driver monitoring.
When using partial vehicle automation, the driver’s primary role is that of an automation (or safety) monitor. But humans have significant limitations related to staying attentive or focused. A driver is highly susceptible to over-reliance on the automation. This results in a driver disengaging -- allowing their mind to stray and doing some non-driver related activity-- from the primary task of monitoring safety.
So, when a partial automation system fails to detect a hazard while the driver is disengaged a crash is likely to occur. And in a highway environment, crashes develop and occur within mere seconds.
Primary safety issues:
Our investigations revealed several recurring issues, each being critical to safe operation of the vehicle or postcrash analysis. We determined the following:
- Partial automation systems have considerable limitations in detecting hazards, as well as in maintaining an appropriate travel path.
- The investigations also showed that drivers can become disengaged from the driving task for an extended period, and that the predominant method of ensuring driver engagement—monitoring driver’s interaction with the steering wheel—is inadequate.
- There are no performance standards for the operation of partial automation systems or for driver monitoring systems.
- There are no requirements for recording of parameters pertaining to operation of driving systems with various levels of automation.
NTSB recommendations:
In our Williston report, we recommended to manufacturers:
- Implement a system-based safeguard that would prevent partial automation systems from being used in conditions for which they were not designed. (H-17-41; H-17-43)
- Develop and implement an effective method of monitoring driver’s engagement. (H-17-42).
In our Williston report (2017) and Mountain View report (2020), we recommended that federal agencies:
- Define and require capture of the parameters needed to understand the operation of driving systems with various levels of automation (H-17-37; H-17-39; H-17-40)
- Verify that vehicle manufacturers have implemented a system-based safeguard that would prevent partial automation systems from being used in conditions for which they were not designed (H-17-38)
- Develop a performance standard for driver monitoring systems and mandate their implementation (H-20-3; H-20-4)
- Evaluate Tesla Autopilot vehicles for safety defects (H-20-2).
Testing of vehicles with higher levels of automation
Dozens of technology companies and vehicle manufacturers are currently conducting testing of automated vehicles on public roads. Some of these vehicles are built ground-up as automated vehicles without traditional vehicle controls (e.g., steering wheel). These vehicles typically require exemption by NHTSA to be able to operate on public roads, and only within testing capacity. Other test vehicles are traditional vehicles equipped with additional sensors, cameras and computing equipment. While these traditional test vehicles typically do not require an exemption by NHTSA, they may require an approval by the state in which testing is being conducted.
Since the challenge of full vehicle automation has not yet been solved, the primary safety objective of the testing should be to identify risks and develop mitigation strategies to avoid crashes while testing on public roads.
Primary safety issues:
In our investigations we determined the following:
- Testing of developmental automated driving systems exposes its functional limitations, particularly in detecting hazards and predicting movement of various road users.
- The extent to which these limitations pose a safety risk depends on safety redundancies and risk mitigation strategies implemented during system development and testing.
- Safety drivers can be affected by automation complacency, which can lead to distraction and failure to monitor the environment and the operation of the automated system.
- There are no federal safety risk management requirements for testing of automated vehicles on public roads.
- The safety self-assessment reports that some automation developers have submitted to NHTSA have a very limited benefit. Those reports are voluntary, and NHTSA does not provide any evaluation; as a result, many of submitted reports lack meaningful safety or technical information.
- Due to lack of federal safety standards or meaningful testing protocols, some states have developed specific risk management-focused requirements that developers have to meet prior to testing on public roads. However, many other states lack such requirements.
NTSB recommendations:
In our Tempe report, we recommended to:
The automation developer (Uber ATG, which has since been acquired by another developer)
- Complete the implementation of a safety management system (SMS), which would have addressed all the issues that presented a safety risk during testing of automated vehicles on public roads (H-19-52).
NHTSA
- Require automation developers to submit a safety self-assessment report prior to testing (H-19-47)
- Evaluate the reports to determine whether they include appropriate safeguards for testing of automated vehicles on public roads (H-19-48).
Arizona and other states
- Require automation developers to submit an application for automated vehicle testing that details a plan for safety risk management and establishes countermeasures to prevent crashes or mitigate their severity (H-19-49; H-19-51)
- Establish a task group of experts to evaluate testing applications, before granting testing permit (H-19-50; H-19-51).
