Skip Ribbon Commands
Skip to main content
Safety Recommendation Details

Safety Recommendation A-05-002
Details
Synopsis: On July 13, 2003, about 1530 eastern daylight time, Air Sunshine, Inc. (doing business as Tropical Aviation Services, Inc.), flight 527, a Cessna 402C, N314AB, was ditched in the Atlantic Ocean about 7.35 nautical miles west-northwest of Treasure Cay Airport, Treasure Cay, Great Abaco Island, Bahamas, after the in-flight failure of the right engine. Two of the nine passengers sustained no injuries, five passengers and the pilot sustained minor injuries, and one adult and one child passenger died after they evacuated the airplane. The airplane sustained substantial damage. The airplane was being operated under the provisions of 14 Code of Federal Regulations (CFR) Part 135 as a scheduled international passenger commuter flight.
Recommendation: TO THE FEDERAL AVIATION ADMINISTRATION: Conduct a study to determine whether the number of flight checks a pilot can fail should be limited and whether the existing system of providing additional training after a notice of disapproval is adequate for pilots who have failed multiple flight checks. On the basis of the findings of the study, establish a flight check failure limit and modify the recheck training requirements, if necessary.
Original recommendation transmittal letter: PDF
Overall Status: Closed - Acceptable Action
Mode: Aviation
Location: Treasure Cay, Bahamas
Is Reiterated: No
Is Hazmat: No
Is NPRM: No
Accident #: MIA03FA141
Accident Reports: In-Flight Engine Failure and Subsequent Ditching Air Sunshine, Inc., Flight 527, Cessna 402C, N314AB, About 7.35 Nautical Miles West-Northwest of Treasure Cay Airport
Report #: AAR-04-03
Accident Date: 7/13/2003
Issue Date: 1/27/2005
Date Closed: 3/27/2014
Addressee(s) and Addressee Status: FAA (Closed - Acceptable Action)
Keyword(s): Training and Education

Safety Recommendation History
From: NTSB
To: FAA
Date: 3/27/2014
Response: We classified this recommendation “Closed—Unacceptable Action” on September 13, 2012, because, in the 7 1/2 years since the recommendation had been issued, you had not supplied us with the results of any study to determine whether the number of flight checks a pilot can fail should be limited or whether the existing system of providing additional training after a notice of disapproval was adequate. Although FAA and NTSB staff had discussed such a study in early 2006, we were not able to obtain a copy of the study or its results. In our September 13, 2012, letter, we indicated that this recommendation would have been satisfied had you simply provided (1) a description of the study that you had conducted and (2) any planned actions based on the study’s results (including the determination that no further actions were necessary). We invited you to provide us with additional information describing the actions you had completed in response to this recommendation, if there were any, and indicated that, if we received such information, we would reconsider the classification of Safety Recommendation A-05-2. In your current letter, you supplied a copy of the results of the study, completed in April 2006. You stated that the results do not support the establishment of a numerical limit of disapprovals, and you therefore plan no further action. Based on the April 2006 study, and your determination that no further action is warranted, Safety Recommendation A-05-2 is classified CLOSED—ACCEPTABLE ACTION.

From: FAA
To: NTSB
Date: 1/21/2014
Response: -From Michael P. Huerta, Administrator: Even though the Board 's letter, dated September 13. 2012, classified this recommendation as closed-unacceptable, the Federal Aviation Administration (FAA) is providing additional infom1ation for the Board to reconsider the classification. In its last letter to the FAA, the Board stated that this recommendation would have been satisfied had the FAA provided: 1) A description of the study that it conducted; and 2) Any planned actions based on the study's results (including the determination that no further actions were necessary). The FAA appreciates the Board’s suggestions for a satisfactory response. A copy of the results of our study of this issue, completed in April 2006, is enclosed. Unfortunately, due to other safety critical work, we were unable to update the Board until now. When reviewing the study, please note that all data, cost estimates, and other information are current as of April 2006. Based on the study, we do not believe that implementation of this recommendation would result in any safety improvement in aviation. According to the study, which compared two databases, the data does not show that an increased number of disapproval results in a less competent pilot. Therefore, the compiled data does not support the establishment of a numerical limit of disapprovals. As previously noted, I consider our actions complete on this recommendation. However, based on the infom1ation provided in this letter, I request that the Board reconsider its classification.

From: NTSB
To: FAA
Date: 9/13/2012
Response: Staff from the FAA and the NTSB held teleconferences on January 11, 2006, and February 8, 2006, to discuss details of the type of study recommended. The FAA staff subsequently sent NTSB staff an e-mail, describing a study conducted during March 2006, which examined data from a 10-year period (October 1995 to October 2005) during which there were 163,846 disapprovals and 4,775 pilots with three or more flight check failures. On November 3, 2006, we indicated that we looked forward to receiving a report of this study describing the data analyzed, the results of the study, and any actions the FAA planned to take in response. The NTSB is disappointed that, in the intervening 5 1/2 years, the FAA has neither issued a report on this study nor indicated any planned actions based on the study’s results. This recommendation would have been satisfied had the FAA simply provided (1) a description of the study that it conducted and (2) any planned actions based on the study’s results (including the determination that no further actions were necessary). However, the FAA has not done so, and the NTSB is not aware of any plans by the FAA to complete this work; consequently, Safety Recommendation A-05-2 is classified CLOSED—UNACCEPTABLE ACTION. We invite the FAA to provide us with additional information describing the actions that it has completed in response to this recommendation, if there are any. If we receive such information in a timely manner, we will reconsider the classification of Safety Recommendation A-05-2.

From: FAA
To: NTSB
Date: 1/21/2010
Response: Letter Mail Controlled 1/25/2010 10:00:08 AM MC# 2100025 - From Tony Fazio, Office of Accident Investigation and Prevention, AVP 420: Study was conducted (March ’06). Awaiting report of results and next steps by FAA to address the flight check failure issue.

From: NTSB
To: FAA
Date: 11/3/2006
Response: The Safety Board recently issued Risk Factors Associated with Weather-Related General Aviation Accidents (NTSB/SS-05/1), a safety study that examined factors associated with accidents that occur in weather conditions characterized by instrument meteorological conditions or poor visibility. Among the findings of this study was the following: "Knowledge and practical test failures are both associated with a higher risk of a pilot being involved in a weather-related general aviation accident." (emphasis added). Staff from the FAA and the Safety Board held teleconferences on January 11, 2006, and February 8, 2006, to discuss details of the type of study recommended. The Board is aware that during March 2006, the FAA examined data from a 10-year period (October 1995 to October 2005) during which there were 163,846 disapprovals, and 4,775 pilots with 3 or more flight check failures. The Board looks forward to receiving a report of this study describing the data analyzed and the results found. The Board notes the concern expressed in the FAA's September 9, 2005, letter that if a hard limit on the number of test failures is established, examiners may be reluctant to find an applicant unsatisfactory, which could lead to an applicant's passing flight checks who otherwise would not. The study conducted by the FAA is responsive to this recommendation. Pending receipt of a report describing the study, a statement from the FAA on the conclusions drawn from the study results, and appropriate action in response to those findings, Safety Recommendation A-05-2 is classified OPEN -- ACCEPTABLE RESPONSE.

From: FAA
To: NTSB
Date: 3/24/2006
Response: -From Glenn White, FAA Office of Accident Investigation, 3/24/06 (email to J. Marcus, NTSB) dated 3/24/2006: Barry Farnsworth's group has run the last three years of AIDS records against the FAA's activity file. This group has previously run ten years of our activity files and looked at the number of disapprovals, with the following results: Here's some stats for Activity file records 10/09/1995 thru 10/31/2005: TOTAL ACTIVITY DISAPPROVALS: 163,847 TOTAL US PILOT DISAPPROVALS: 128,996 TOTAL US PILOTS W/3 OR MORE: 4,775 TOTAL US PILOT FILE RECORDS: 16,175 Disapprovals by Total Number for Same Pilot: Number Pilots ------ ------ 10 4 9 2 8 3 7 32 6 78 5 268 4 897 3 3,491 ----- Total: 4,775 So, to answer the question posed by the NTSB as to whether pilots with more disapprovals have a higher likelihood of having accidents (ending up in the AIDS database), we can point out the following facts, based upon the last three years of total records in the AIDS database: 1. Over the last ten years, there are four pilots with ten disapprovals. NONE of these are in the AIDS database. 2. Over the last ten years, there are two pilots with nine disapprovals, NEITHER of which is in the AIDS database. 3. Over the last ten years, there are three pilots with eight disapprovals. NONE of these are in the AIDS database. 4. Airmen with more than one disapproval comprise 4% of AIDS records 5. Airmen with one disapproval or less comprise 96% of the AIDS records. There are eight pilots with five disapprovals in the three-year AIDS records. So if you wish to view the numbers in this fashion, an airman with five disapprovals over a ten year period, based on these percentages, has a 2% chance of having an AIDS record in the last three years - for this particular sample at this particular time. If we extrapolate the AIDS records to ten years, the chance of having an AIDS record in the last ten years would be .94% There are presently about 609,000 active US pilots. There are 8,505 AIDS records for the last three years. The chance of any of those pilots being in AIDS is about 1.4 percent, so all the analysis done on AIDS records represents a sample of less than two percent of active US pilots. Jeff, I believe we have gone as far as we can go especially in respect to the utilization of our resources. We could do a lengthy project involving a large amount of man-hours, but the result will be approximately the same. I talked to Hop Potter this morning and he is requesting if your group has any safety data to support something other than our evaluation, please make it available to us as we are all on the same safety team. We really want to do the right thing, and I believe we have accomplished that. As in this case, we keep coming up with the same approximate answer no matter how much statistical analysis we perform. Please give me a call to discuss the FAA's findings. I truly believe we have met with the intent of this recommendation and have performed the analysis that your group has requested.

From: FAA
To: NTSB
Date: 9/9/2005
Response: Letter Mail Controlled 9/20/2005 1:51:32 PM MC# 2050440 - From Marion C. Blakey, Administrator, FAA: The FAA conducted a study in 2004 to determine if there is a correlation between flight test failures and the airman being cited in an FAA enforcement action. A review of a total of 15,024 disapprovals against the FAA Enforcement Information System showed a very low correlation, less than one percent. While any single case of involvement in an FAA enforcement action may have little significance, taken collectively involvement in enforcement can be an indicator of lack of qualities that make up a safe pilot, including required skills, knowledge, or compliance disposition. This study showed a negligible correlation between test failures and enforcement. We would be pleased to provide further details on this study to the Board. An additional concern that has been raised by training experts and FAA inspectors as regards establishment of a hard limit on the number of test failures, is that, particularly as this limit is approached, examiners will be extremely reluctant to find an applicant unsatisfactory. This could result in applicants passing flight checks who otherwise would not, with net negative safety consequences. As discussed in response to Safety Recommendation A-05-1, there are system in place to enable 14 CFR Parts 121 and 135 operators to take additional steps to screen applicants. The amendment to AC 120-68 will strengthen the use of letters of consent to authorize FAA to release records of Notices of Disapproval for flight checks to prospective employers. The cost of retraining and other economic considerations also tend to screen out pilots who are consistently poor performers. I believe that the FAA has satisfactorily responded to this safety recommendation, and I look forward to your response.