DCA09IA014
DCA09IA014

HISTORY OF FLIGHT

On November 26, 2008, about 1930 coordinated universal time, a Boeing B777-232ER (B777), N862DA, operated by Delta Air Lines as flight 18, experienced an uncommanded thrust reduction (or “rollback”) of the right Rolls-Royce RB211 Trent 895 engine during cruise flight at flight level (FL) 390 (about 39,000 feet). Following the rollback, the flight crew descended to FL310 and executed applicable flight manual procedures, after which, the engine recovered and responded normally. The flight continued to Atlanta-Hartsfield International Airport (ATL), Atlanta, Georgia, where it landed without further incident. The airplane was not damaged, and no injuries were sustained by the flight crew or passengers. The flight was a regularly scheduled passenger flight from Shanghai Pudong International Airport (ZSPD), Shanghai, People’s Republic of China, to ATL, operating under the provisions of 14 Code of Federal Regulations (CFR) Part 121. Visual meteorological conditions prevailed for the flight, and an instrument rules flight plan had been filed.

There were no reported anomalies in flight from the departure from Shanghai through the initial portion of cruise. While crossing the northern Pacific Ocean, just south of the Aleutian Islands, the airplane passed a low pressure area with some light to moderate turbulence. Shortly after entering U.S. airspace in the vicinity of Seattle, Washington, air traffic control cleared the flight to climb to FL370. The flight crew executed the climb using VNAV (vertical navigation) mode and the CLB (maximum climb) power setting. About 15 minutes after reaching FL370, the en-route flight crew was relieved by the incident flight crew. At that time, the airplane was in clear weather over an undercast layer, and all flight deck indications were reported as normal. Shortly after the flight crews changed, the flight was cleared to climb to FL390. The incident flight crew executed the climb using the VNAV and CLB settings, and flight deck indications were reported as normal.

The first officer, who was the pilot flying, stated that, about 30 minutes after reaching FL390, he noted that the right engine pressure ratio (EPR) indication was “fluctuating” and that he saw corresponding fluctuations on the N1 and exhaust gas temperature (EGT) indications. He reported that his attention was drawn by the message, “ENG RESP,” displayed on the engine indicating and crew alerting system (EICAS); however, no message with this text exists on the B777. The flight crew reported that they were aware of a January 2008 accident involving a B777 at London Heathrow Airport that experienced engine rollbacks and, therefore, they suspected that “engine icing” was causing the indications. (Air Accidents Investigations Branch [AAIB] AAR 1/2010 “Report on the Accident to Boeing B777-236ER, G-YMMM, at London Heathrow Airport on 17 January 2008” is available on-line at http://www.aaib.gov.uk/sites/aaib/publications/formal_reports/1_2010_g_ymmm.cfm.)

The ENG RESPONSE (engine response) non normal checklist in the Delta B777 Operations Manual Quick Reference Handbook (QRH) had been revised on October 31, 2008, as a result of recommendations issued by the AAIB during the investigation of the G YMMM accident. The revised U (unannunciated) ENG RESPONSE checklist stated that it was to be used when “One or both engine(s) did not reach commanded thrust or rolls back after performing the Cold Fuel Operations supplementary procedure or after operating at high thrust settings. Note: The objective of this procedure is to clear ice from the fuel system by reducing engine fuel flow while descending, then checking for proper engine response.” The first item on the checklist was to descend with the thrust levers closed.

The flight crew reported that they had some questions about the applicability of the U ENG RESPONSE checklist to their situation because they were more than 3 hours from the top of the descent and had not yet performed the Cold Fuel Operations procedure, nor had they recently operated either engine at a high thrust setting. However, they retarded both thrust levers and executed a descent to FL310. The right engine recovered and responded normally for the remainder of the flight. The flight crew performed the rest of the U ENG RESPONSE checklist items after reaching FL310. The flight continued to ATL, where it landed without further incident.

Review of the DFDR (digital flight data recorder) and QAR (quick access recorder) information indicated that about 45 minutes after reaching FL390, the right engine EPR dropped from approximately 1.25 to 1.1. The data showed that the measured fuel flow for the right engine reduced to approximately 5000 pounds per hour, consistent with the reduced thrust of the engine. All the engine parameters, including rotor speeds, burner pressure, and stator vane position, were consistent with the measured fuel flow, indicating that the engine was operating as would be expected for the measured amount of fuel flow. Recorded data showed that the engine control system attempted to increase fuel flow in response to autothrottle commands by driving the fuel metering valve (FMV) fully open, but this did not result in any increase in the fuel flow.

The airplane’s Thrust Asymmetry Compensation (TAC) system automatically adjusted the rudder to compensate for the asymmetric thrust, caused by the reduced right engine power. The rollback condition persisted for about 23 minutes between the first indication of fuel flow reduction on the FDR data and the execution of the descent by the flight crew. The engine control system on the Rolls-Royce Trent 895 powered B777 is designed to display an ENG THRUST (sometimes termed “EPR shortfall”) caution message on the EICAS alerting the flight crew of a significant disagreement between the actual measured engine thrust and the commanded thrust. Examination of recorded engine controls data revealed that this message was not displayed on the EICAS (see Tests and Research section below.)

FDR and QAR data further indicated that about 55 minutes prior to the rollback, the oil temperature of the right engine began to rise, eventually increasing by about 31º C relative to that expected based on the previous portions of the flight. During the rollback period, the oil temperature decreased, but it still remained at a higher than expected value. Following the recovery, the oil temperature returned to a value consistent with the previous portions of the flight. Note that a consistent small difference in the oil temperature between the two engines was evident throughout the nominal portions of the flight, which was not considered abnormal, due to variances in manufacturing tolerances.

Recorded data indicated that, during the hour before the rollback, the outside (static) air temperature reached a minimum of -62º C and increased to about -57º C following the step climb to FL390. The total air temperature (due to the speed of the airplane) was about -25º C before the rollback, and it decreased slightly during the incident as the airspeed decreased. Main tank fuel temperatures went below 0° C about 3 hours into the flight and then progressively reduced to a minimum of -23° C. The airplane made four step climbs at fuel flows in excess of 11,000 pph before the restriction occurred. The third and fourth step climbs both occurred at fuel temperatures below 0° C. The third occurred shortly after the fuel temperature went below 0° C, and the fourth occurred just over 3 hours later when the fuel temperature was approaching -15° C. About 3 hours later, the airplane carried out a further step climb, with a maximum fuel flow of just over 11,000 pph. It was during this engine acceleration that the engine oil temperature was observed to rise, followed shortly thereafter by the rollback. Fuel temperature at the time of the rollback was -22° C. The AAIB report of the G-YMMM accident includes an appendix discussing data mining examining fuel temperatures and fuel flow on long-duration B777 flights.

PERSONNEL INFORMATION:

The captain, age 52, held an airline transport pilot (ATP) certificate, multi-engine land, and was type rated in the B777, B767, B757 and DC-9. He held a current Federal Aviation Administration (FAA) first-class medical certificate with no limitations. He had accumulated 13,409 total flight hours, 5,793 hours of which were as pilot-in-command and 87 hours of which were in the B777. All of his B777 time was accumulated in the 90 days before the accident. His last flight review was on September 25, 2008, in the B777.

The first officer, age 54, held an ATP certificate, multi-engine land, with type ratings in the B777, B767, B757 and TBM 700. He held a current FAA first-class medical certificate with no limitations. He had accumulated 10,412 total flight hours, 1,896 hours of which were in the B777. He had accumulated 117 hours in the B777 in the 90 days before the accident.

AIRCRAFT INFORMATION:

N862DA, a B777-232ER, manufacturer serial number 29734, company ship number 7003, was delivered new to Delta Air Lines in December 1999. At the time of the incident, the airplane had accumulated about 41,222 total flight hours and 5,670 flight cycles. The airplane was equipped with two Rolls-Royce RB211 Trent 895 engines. The left engine, serial number 51193, was manufactured in August 1999 and had 38,052 total flight hours. The right engine, serial number 51150, was manufactured in February 1999 and had 33,773 total flight hours. The fuel-oil heat exchangers (FOHE) were manufactured for Rolls-Royce by Sumitomo Precision Products of Japan. The FOHE on the left engine, serial number 366, was installed in March 2008, and the FOHE on the right engine, serial number 504, was installed in November 2007.

The airframe fuel supply system stores and delivers fuel to each of the engines. Fuel is stored in three fuel tanks: a center tank, a left main wing tank, and a right main wing tank. The center tank contains two (left and right) electrically driven override/jettison pumps, and each main fuel tank contains two electrically driven boost pumps, identified as forward and aft. Fuel pump design and normal operating procedure is for all of the pumps in the tanks, with fuel loaded, to be operating. The outlet port of each fuel boost pump contains a check valve and is connected to the fuel feed manifold, which runs across the airplane and connects to the engine fuel feed lines. The manifold is split between the left and right system by two crossfeed valves. When the crossfeed valves are closed and the center tank is the source of the fuel, the left override/jettison pump feeds the left engine and the right override/jettison pump feeds the right engine. Since the center tank pumps operate at a higher pressure than the main fuel tank boost pumps, the main tank boost pump outlet check valves will be closed so that the center tank delivers the fuel to the engines. When the center tank is nearly empty, the center tank fuel pumps are switched off and the main tank boost pumps continue the supply of fuel to the engines.

The Trent 800 series engine is fitted with a main engine-driven fuel pump, which has low-pressure (LP) and high-pressure (HP) stages. Fuel delivered from the LP stage passes through an FOHE before it is delivered to the HP stage. The FOHE is designed to cool hot engine oil by transferring heat to the cold fuel entering from the fuel tanks and to warm the fuel to prevent ice from clogging the fuel filter and sensitive components downstream. The design of the FOHE at the time of the incident passed fuel through a matrix comprising 1,180 tubes about 2 millimeters in diameter and the engine oil circulated around the outside of these tubes. The matrix of tubes was mounted into a casing with an upper and lower face plate to separate the fuel from the oil supply. The inlets to the fuel tubes protruded a few millimeters above the inlet face plate.

METEOROLOGICAL INFORMATION:

At the time of the rollback, the airplane was in visual meteorological conditions above an undercast layer. The flight crew reported no unusual weather phenomena during the cruise portion of the flight. Upper air data indicated a 165-knot jet stream off the northwest Pacific coast and a short wave trough along the route of flight. The airplane passed through a low pressure system south-southwest of the Gulf of Alaska resulting in light to moderate turbulence aloft. During the portion of the flight over the northwestern United States, the airplane passed north of an upper level low pressure system, with the tropopause sloping from about 37,000 to 34,000 feet. Recorded air temperatures during the period preceding the rollback indicated a static air temperature of about -57º C at FL390 with a slightly colder air mass below, corresponding with the tropopause level.

FLIGHT RECORDERS:

The airplane was equipped with a L3 Communications model 2100 digital flight data recorder (DFDR) and a Penny and Giles quick access recorder (QAR). Following the incident, Delta Air Lines removed the DFDR and QAR from the airplane. Delta Air Lines downloaded the DFDR and transmitted the data to Boeing. The airline also forwarded the QAR optical data disk to Boeing for download. The initial indications of this incident are not normally reportable to the National Transportation Safety Board (NTSB) per 49 CFR Part 830.5, and the data downloads were coordinated with NTSB staff, when it appeared that the incident had similarities to the G-YMMM accident. The NTSB’s Vehicle Recorder Division received the electronic data files for the DFDR and QAR from Boeing.

Details of the recorder data and plots of selected parameters are in the NTSB public docket. DFDR and QAR information was used as the basis of the performance studies and other sections of the report.

The airplane was equipped with a cockpit voice recorder; however, the data were overwritten during the continuation of the flight to Atlanta and, therefore, no incident information was available.

TESTS AND RESEARCH:

Postincident examination of the airplane’s engine fuel supply systems and re-examination of the systems in February 2009 following a flight from Shanghai to Atlanta found no evidence of preincident component failures or mechanical restriction in the fuel system. However, during the engine examination after the incident physical evidence similar to that seen on the HP pumps removed from both engines of G-YMMM was seen in the HP stage of the right engine fuel pump. Specifically, markings consistent with cavitation were observed at the outlet ports of the incident engine pump HP stage. Cavitation is the result of the vacuum created when the discharge capacity of the pump exceeds the replacement capacity to the pump inlet. A restriction in fuel flow upstream can result in the HP pump lowering its inlet pressure to the point at which a large quantity of air and vapor bubbles are formed, causing a mixture of fluid and gas bubbles to fill the gear teeth. As these gear teeth open to the HP outlet port of the pump, the gas bubbles collapse and the HP fuel in the outlet port rapidly fills the available space. Both of these actions result in shock waves being formed that cause damage to the outlet port and the delivery side of the bearing faces. (A detailed description of this effect is included in the AAIB accident report on G-YMMM.)

Fuel samples from the incident and the re-examination flights were taken and tested for water, contaminants, and other chemical characteristics. Additionally, records of fuel sample testing were received from Shanghai. No anomalies in the water content or chemical makeup of the fuel were found. A slight amount of bacterial growth was found in the wing tanks and fuel filters, but this was not considered significant.

In support of the AAIB investigation into the G-YMMM accident, Rolls-Royce and Boeing conducted extensive research and tests to understand the nature and location of the fuel flow restriction. The possibility that ice was in the fuel was considered because ice can form from water molecules normally present in jet fuel. Research demonstrated that water molecules entrained in jet fuel will begin to form ice crystals as the fuel cools below 0° C. (The fuel temperature at the time of the rollback was -22° C.) This research also determined that ice crystals suspended in fuel can begin to adhere to their surroundings and accrete at fuel temperatures between -5° and -20° C, this was termed the “sticky range” in testing, and is not an uncommon fuel temperature range for long-haul flights to encounter.

Tests using B777 components in a cold fuel laboratory showed that ice crystals can form from normal concentrations of dissolved and entrained water present in aviation turbine fuel and that, under certain conditions, high concentrations of ice can accumulate on the inside of the fuel delivery pipes of the fuel feed system. This ice can be released during higher fuel flows, and can enter the engine fuel system and collect in a mass on the FOHE inlet face. The testing demonstrated that, after high fuel flows, the FOHE inlet face can be presented with ice concentrations of the order of 100 times more than certification requirements, which is sufficient to restrict fuel flow so that the thrust demanded could not be supported and a rollback could occur.

Testing also showed that reducing the volume of the fuel flow through the FOHE can clear ice from the face of the FOHE within a few seconds. Retarding the throttles reduces the fuel flow and the cooling effect on the tubes such that conducted heat from the oil raises the temperature of the tubes and then melts the ice. Testing showed that temporarily reducing fuel flows by moving the power levers to minimum idle melted ice that had accumulated on the face of FOHE.

As noted above, the incident flight crew reported being alerted to the rollback condition by the appearance of the “ENGINE RESP” EICAS message, but no message with this text exists on the B777. Therefore, the investigation examined the possibility that the “ENG THRUST R” message might actually have been displayed. An examination of the incident data and comparison to testing in support of the G-YMMM investigation indicated that, during the rollback event, the engine control system commanded the FMV to a more open position, attempting to maintain the commanded EPR. As noted above, the fuel flow did not increase as expected and the engine continued to run with restricted fuel flow, but the engine parameters fluctuated due to the erratic flow from the cavitating HP pump. These fluctuations triggered the surge and stall recovery logic, moving the FMV to a less open position, briefly sufficient to cause a momentary dip in the actual fuel flow below the level of the FOHE restriction. The surge and stall recovery logic was then cleared, and the engine control drove the FMV back to its fully open position. This cycle then repeated about every 25 to 30 seconds for the entire 23-minute duration of the rollback. Therefore, Boeing concluded that the “ENG THRUST R” message would not be set because the engine continued to run in an intermediate state above idle and because the cyclical “dips” periodically accelerated toward the command.

ORGANIZATIONAL AND MANAGEMENT INFORMATION:

Delta Air Lines is headquartered in Atlanta, Georgia, and provides service to 311 destinations in 52 countries. Delta was founded in 1929 and purchased Northwest Airlines in 2008. At the time of the incident, the integration process with Northwest was still in effect. Delta and Northwest operate hubs at Atlanta, Georgia; Cincinnati, Ohio; Detroit, Michigan; Memphis, Tennessee; Minneapolis/St. Paul, Minnesota; New York City, New York (John F. Kennedy International Airport); and Salt Lake City, Utah. At the time of the incident, Delta operated eight B777-200ERs with Rolls-Royce RB211 Trent 895 engines. Delta initiated service to Shanghai with the B777-200ER on March 30, 2007.

ADDITIONAL INFORMATION:

In response to recommendations made by the AAIB during its investigation of the G-YMMM accident, in September 2008, Boeing issued a flight crew operations manual bulletin addressing the prevention of long-term ice accumulation in the Trent 800 powered B777 aircraft fuel system during extreme cold operations. The bulletin instructed flight crews to follow specific refueling instructions before long-range flights when the ground fuel temperature was below 0° C. The bulletin also included a cold fuel operations supplementary procedure that specified that flight crews should perform an ice accumulation clearing procedure within 3 hours of the top of the descent if the fuel temperature was below 10° C by briefly increasing the thrust of each engine to maximum climb thrust. Finally, based on the results of the Boeing/Rolls-Royce tests discussed above, Boeing added an engine response non normal procedure to its airplane flight manual that instructed flight crews to temporarily reduce fuel flows by moving the power levers to minimum idle to clear ice from the FOHEs if the engines did not reach commanded thrust after performing the cold fuel operations supplementary ice clearing procedure or after operating at a high thrust setting. The manual changes were mandated by the FAA on September 29, 2008, with the issuance of AD 2008 19 04.

Following the Delta incident, Boeing further revised its engine response non normal procedure to require that the cold fuel operations supplemental procedure be performed within 2 hours of the top of the descent, rather than the 3 hours previously specified, to reduce the time available after the procedure for the buildup of ice. In addition, flight crews were required to retard the throttles to minimum idle speed for 30 seconds during initial descent. The checklist was modified to state that it is to be accomplished if an engine fails to reach commanded thrust and fuel system icing is suspected to prevent flight crews from being confused about the applicability of the engine response non normal procedure (as happened to the incident flight crew) and to ensure that the procedure is used when needed. In response to the revised operational requirements, the FAA issued AD 2009-05-11, which superseded AD 2008-19-04.

In March 2009, the NTSB issued Safety Recommendations A-09-17 through -20, which asked that the Trent 800 series FOHE be redesigned and that the European Aviation Safety Agency (EASA) and FAA require installation of the redesigned FOHE. In addition, the AAIB issued recommendations to review the design of the fuel system to develop changes that prevent ice from causing a restriction to the fuel flow at the FOHE and to require EASA and the FAA to mandate the changes. Recommendations were also made to EASA and FAA relating to certification requirements, feasibility of the use of a fuel system icing inhibitor, and research into ice formation in aviation turbine fuels and release mechanisms of accreted ice.

In 2009, Rolls-Royce developed and obtained certification of a redesigned FOHE with flush-face fuel inlet tubes, which was tested to be much more tolerant to ice accumulation at the inlet face. In July 2009, EASA issued AD 2009-0142, which required the installation of the redesigned FOHE on all Trent 800 engines. The FOHEs on both engines of N862DA were removed and replaced with a new design during the summer of 2009 and Delta completed the installation of the redesigned FOHE on all of the Trent 895 engines on its airplanes in December 2009. The fleetwide retrofit with the redesigned FOHE was completed in September 2010. Rolls-Royce also issued service bulletins to modify the FOHE fitted to the Trent 500 and 700 engines, and EASA modified AD 2009-0142 to include these models.

Federal regulations require that manufacturers demonstrate that an engine performs properly under any anticipated operating condition. Specifically, 14 CFR 21.21(b)(2) states, “…no feature or characteristic makes [a condition] unsafe for the category in which certification is requested.” Following the Delta and G-YMMM events, the FAA reviewed previously accepted methods of compliance with respect to fuel system icing and found that neither the FAA nor applicants anticipated ice accumulating in the aircraft and engine fuel feed system upstream of the FOHE or that, under certain conditions, the ice could release and cause a restriction at the FOHE that would limit fuel flow to the engines. Therefore, the FAA and EASA provided applicants revised acceptable methods of compliance to those applicable regulations (via issue paper and certification review item, respectively) to ensure that future designs are tolerant to the threat of ice concentrations. The FAA and EASA require applicants to address the airplane and engine fuel feed system icing threat by considering the effect of transient fuel icing conditions on engine operability likely to be encountered in service and the possibility for blockage of fuel system components. The airplane and engine fuel system must either be designed to prevent a likely accumulation of ice from being released into the engine or be designed so no loss of engine thrust occurs due to the release of accumulated ice.

Use your browsers 'back' function to return to synopsis
Return to Query Page