Good morning! Thank you to Greg Woo, Kathy Abbott, Alex Landsburg, and of course, my NTSB colleague, Bill Tuccio, for the opportunity to speak at this workshop. My name is Bella Dinh-Zarr and I am a Board Member at the NTSB (National Transportation Safety Board). It is a pleasure to be here with all of you - people who value good data.
We all know that data is at the core of our work in transportation. It is integral to the work at TRB and to each of our individual organizations, so your discussions today will be an important step in shaping the direction of its collection and its proper use. Of course, there are diverse types of data in transportation and, in fact, I started my career in injury prevention and transportation safety where I worked with very different types of data than what we often now use at the NTSB and probably different from what some of you use.
In my past life, I conducted systematic reviews and meta-analyses using injury severity and fatality data that was usually collected by medical personnel or law enforcement officers. Unlike many of you, I did not have to validate the conversion of zeros and ones from electronic devices, and I did not often have to analyze qualitative narratives such as we do with voluntary aviation data like ASRS (Aviation Safety Reporting System) and ASAP (Aviation Safety Action Program), but we all know that every type of data presents its own challenges so perhaps part of your discussions today will address the unique technical and structural issues for different sources of data and how those data can be used most wisely.
Today’s workshop will focus specifically on accident data, operationally reported data, and voluntary safety data – some of which will overlap. But ultimately, as the title of this workshop indicates, no matter what type of data or its source, we want to determine how best to use the information we obtain to prevent deaths and injuries. This morning, I will not attempt to do a deep dive into the many areas of data that deserve discussion, not only because there is not enough time, but because you are the experts in each of your areas. What I would like to do is to give some examples of the situations we encounter with data – or lack of it – at the NTSB, in the different modes. This will, I hope, spark continued conversations among you – the experts, the people who care about data – about how it can and should be collected, how we can better use data for prevention, and how we all can and must continue to speak out in support of better, more robust data at every opportunity.
You may have heard of the DIKW Pyramid. Bill Tuccio reminded me of it the other day. DIKW stands for data, information, knowledge, and wisdom. Much of what we do at the NTSB, and certainly what many of you do at your institutions is work hard to analyze and convert the data we have into information and knowledge. If we are very good and somewhat lucky, we sometimes end up with more wisdom for the field as well.
Some of my past injury prevention work was with the CDC (the Centers for Disease Control and Prevention) and was viewed through a public health lens, so it was called injury epidemiology. Epidemiology, as many of you know, is the study of the determinants and distribution of health-related events in specified populations and the application of this study to the control of these health problems.
We do not often use the term epidemiology in our transportation safety studies, but epidemiology is what we are doing. As one doctor said to me, “Bella, you work on epidemics, too, except instead of a disease outbreak, it is an outbreak of kinetic energy!” Our work fits well into the DIKW Pyramid also. We start with data and try to convert it into information that will, we hope, provide us with the knowledge to control – or prevent – future injuries and deaths. We want to gain knowledge to predict potential risk and reduce it. As Bill and I have discussed, we only wish there were a “black box” (pun intended) that has data coming in one side and prevention magically coming out the other side. But predictive data analysis depends on hard work and input by experts like you. Any effort to prevent an accident is an effort to predict the future using expertise like you possess in this room. In public health, we often use the word “control” such as injury control or disease control, and I think it applies to transportation as well. We want to know what may happen, so we can control it. We not only want to predict the future, we want to control it. We are trying to gain knowledge, and perhaps even eventually wisdom, to “control” the future in a positive manner, by preventing future accidents or at least preventing the deaths and injuries associated with those future accidents.
When you think of the NTSB, you probably think of us in our dark blue uniforms with the bright yellow letters on the back at the scene of transportation disasters. Although our work continues long after we are off-scene, it is true that our first encounter with the many different situations involving data is at the scene. In fact, even before we arrive on scene, people in our Research and Engineering Office, like Bill, are hard at work data mining to provide historical context. Since this is such a diverse group, I thought I would try to give a brief example from some of our recent investigations in three different modes: aviation, maritime, and highway.
During my brief stint as Acting Chairman last year, I chaired a board meeting about an Airbus helicopter accident. The Helicopter Emergency Medical Services (HEMS) chopper lifted off from a medical center helipad in Frisco, Colorado, began spinning counterclockwise, gaining about 100 feet of altitude before plunging downward, impacting the ground 360 feet away. The pilot was fatally injured, and the two flight nurses were seriously injured. The helicopter was destroyed by impact forces and a post-crash fire. The probable cause was a preflight system check which depleted hydraulic pressure in the tail rotor and the lack of an alert in the cockpit that could have alerted the pilot of low hydraulic pressure resulting in high pedal loads and a loss of control after takeoff. Contributing factors were the pilot not performing a hover check and fuel system which was not crash resistant.
Although adherence to flying procedures (such as performing a hover check) could have prevented this crash, we know that people make mistakes, which is why we need to use the information we have to enhance systems that can help mitigate human error. In this case, there was flight data monitoring (FDM), but the data were not being used. Parametric data was archived but had problems because the quality of the data was poor, with dropouts and fluctuations of GPS (Global Positioning System) information and pitch and roll parameters as well as incorrect altitude data among other issues. These quality problems may have been due to the intermittent GPS signal and because the pitch parameter was not calibrated. Video/audio data were not archived nor were they used in the FDM. This aviation example demonstrates the challenges we often encounter with accident data, but the poor data quality we encountered would have been identified had there been a vibrant FDM program, operationally using the data day in and day out.
El Faro was a 790-foot cargo vessel that set sail from Jacksonville, Florida, to San Juan, Puerto Rico, in September 2015. It sank about 34 hours later near the eye of Hurricane Joaquin. All 33 crewmembers aboard the ship perished. I was the Board Member on duty and the NTSB launched an investigation as soon as the sinking was confirmed. With assistance from the Navy and Coast Guard, the wreckage and debris field was located more than 15,000 feet under the surface of the ocean. The probable cause was the Captain’s insufficient actions related to the hurricane along with contributing factors such as poor safety management, loss of propulsion due to low lube oil, cargo hold flooding, and lack of appropriate life rafts, among other issues.
After many months and much work, the VDR was recovered and our investigators spent hundreds of hours transcribing and analyzing the 26 hours of recording which were instrumental to the investigation. The investigation produced more than 70 findings and more than 50 proposed recommendations. Some of the problems we encountered with El Faro data were:
Voyage data recorder (VDR) audio quality
Inadequate performance testing for VDRs
Global maritime distress and safety system (GMDSS) user entered position errors, and
Incomplete recordings, including lack of certain:
Calls to/from engine room
Calls to/from the captain’s stateroom
Calls to/from other portions of the ship
External (VHF) communications recorded only from ambient sources
Despite these issues, without the VDR, we would not have been able to conduct such a thorough investigation. Yet, perhaps, we could have uncovered even more meaningful insights if the recordings were better - if we could have heard both sides of conversations, for example.
TESLA – Willison, FL
Last May, a 2015 Tesla Model S, was traveling eastbound on a divided highway near Williston, Florida. A truck traveling westbound was making a left turn across the eastbound lanes. The Tesla’s automation did not detect – nor was it designed to detect – the crossing vehicle. The Tesla struck the side of the semitrailer, crossed underneath, shearing off the car’s roof. Sadly, the car driver died. The probable cause was the truck driver’s failure to yield of the right of way, combined with the car driver’s inattention due to overreliance on vehicle automation. A contributing factor was the vehicle’s operational design, which permitted prolonged disengagement from the driving task.
The Tesla, considered a Level 2 automated vehicle, recorded a large number of parameters. Some of these data – such as vehicle speed – were recorded at a set rate of once per second. Other data – such as the steering wheel hands-on-state – were recorded in response to a change in the state of that parameter.
The camera system used to support Autopilot operation was designed to capture a set of images in response to a Forward Collision Warning or Automatic Emergency Braking event.
Data stored on the vehicle is periodically uplinked to Tesla servers via WiFi. Any data recorded after the most recent up-link event will be resident only on the vehicle. In all cases, the identification and conversion of this data to information usable by safety investigators – such as speed, steering angle, lead vehicle distance – had to be obtained via a query of the data on Tesla servers using proprietary manufacturer software. Fortunately, they were cooperative and knowledgeable and the data survived, but the data could have been lost.
We are in an environment of continuous data collection by automobile manufacturers and operators. Perhaps we should ask what data are necessary for the broader safety community? How should audio and video data be monitored? How can we balance safety benefits versus privacy? What about exposure data which is so often overlooked and often only obtainable from manufacturers for system operational states?
These are all critical issues, but hardly new ones. In fact, I recently saw this Safety Study written 15 years ago by NTSB’s Dr. Deb Bruce, highlighting some of these same problems.
The NTSB is unique because we are independent, and we do not have any regulatory or enforcement authority, so we rely on our powers of persuasion. Fortunately, most people feel we wear the “white hats” and because they believe we make decisions based on their safety impact, people often want to do what we ask. In fact, we have issued over 14,000 safety recommendations to over 2,300 recipients over the years – and approximately 80% of our recommendations have been adopted.
However, we cannot do this without your help and your expertise and your support. Data and information and knowledge cannot lead to wisdom in our field without your expertise and your championing the importance of data.
I know you will be discussing many topics today with your fellow data experts, but throughout it all, I hope you never forget that your scientific and technical work gives legitimacy and power to what you say to the public. Your real-world examples give power to the data to overcome incorrect and misleading information that sometimes becomes the norm. As researchers and as those who believe in the importance of data, it is all of our jobs to tell the story of good data, to make people feel something about good data.
I know researchers do not usually talk about feelings, but it is worth remembering what the great writer Maya Angelou said: “I've learned that people will forget what you said, people will forget what you did, but people will never forget how you made them feel.” We can make people feel something about data and information and knowledge, using our own examples and stories from our work, and perhaps more people will be inspired to support worthy, evidence-based safety efforts. In addition, what more noble goal can there be than to use our combined knowledge – and yes, our wisdom – to save lives and prevent injuries?
I hope to see you at our NTSB session on Wednesday at 8:00 a.m. in Salon B of the Convention Center where our very capable staff will provide much more in-depth presentations on some of our studies and investigations, including the Frisco helicopter accident and Tesla crash.
I wish a very productive discussion today. You come from diverse organizations and I am looking forward to hearing your discussions. I know you are in good hands with my colleague Dr. Bill Tuccio and the other organizers of this workshop. Thank you again for having me here today. In addition, thank you for your work to advance transportation safety.