UNITED STATES OF AMERICA NATIONAL TRANSPORTATION SAFETY BOARD WASHINGTON, D.C. ********************************************************** IN THE MATTER OF THE INVESTIGATION OF * AMERICAN AIRLINES, INC., FLIGHT 1420, * Docket Number McDONNELL DOUGLAS MD-82, N215AA * SA-519 LITTLE ROCK, ARKANSAS, JUNE 1, 1999 * *********************************************************** Arkansas Excelsior Hotel Bill Clinton Ballroom Three Statehouse Plaza Little Rock, Arkansas 72201 Thursday, January 27, 2000 8:30 a.m. Board of Inquiry HONORABLE JIM HALL, Chairman Board of Inquiry THOMAS HAUETER, Deputy Director Office of Aviation Safety JOHN CLARK, Deputy Director Office of Research and Engineering BARRY SWEEDLER, Director Office of Safety Recommendations and Accomplishments BEN BERMAN, Hearing Officer Office of Aviation Safety Technical Panel GREGORY SALOTTOLO GREGORY FEITH EVAN BYRNE MARK GEORGE CHARLES PEREIRA LAWRENCE ROMAN DAVID TEW DONALD EICK Public Information Officer PAUL SCHLAMM Office of Public Affairs Washington, D.C. Parties to the Hearing LYLE STREETER, Air Safety Investigator Accident Investigation Division, AAI-100 Federal Aviation Administration RONALD J. HINDERBERGER, Director Airplane Safety Boeing Commercial Airplane Group ROBERT W. BAKER, Vice Chairman American Airlines, Inc. CAPTAIN CHRIS D. ZWINGLE Special Assistant to Chairman National Safety and Training Committee Allied Pilots Association KATHY LORD-JONES National Safety Coordinator Association of Professional Flight Attendants ROBERT KUESSNER National Weather Service DEBORAH H. SCHWARTZ, A.A.E. Airport Manager Little Rock National Airport J.T. CANTRELL, Training Chief Little Rock Fire Department I N D E X WITNESS: PAGE: Operations David C. Gilliom, Manager 374 Flight Standards Division - Western Pacific Region Interview by David Tew Robert Baker, Vice Chairman 443 American Airlines Interview by Evan Byrne/Gregory Feith Meteorology George Wilken, Science and Operations Officer 542 National Weather Service North Little Rock Forecast Office Interview by Donald Eick Dr. Wes Wilson 516 MIT Lincoln Labs Interview by Donald Eick Dale A. Rhoda 677 MIT Lincoln Labs Interview by Donald Eick Rainer Dombrowsky, Chief 729 Surface Observation Branch National Weather Service, Headquarters P R O C E E D I N G S 8:30 a.m. CHAIRMAN HALL: We will reconvene this public hearing of the National Transportation Safety Board. This hearing is being held in connection with the Accident Investigation involving American Airlines Flight 1420, a McDonnell Douglas MD-82, Registration N215AA, that had an accident here in Little Rock on June 1, 1999. I would ask our hearing officer, Mr. Berman, if he would call the next witness. MR. BERMAN: I call Mr. David C. Gilliom. Whereupon, DAVID C. GILLIOM having been first duly affirmed, was called as a witness herein and was examined and testified as follows: INTERVIEW BY BOARD OF INQUIRY BY MR. BERMAN: Q Good morning, sir. Would you please state your full name and business address for the record? I'm sorry. Flip the switch on your mike. MR. BERMAN: Stand by one second. CHAIRMAN HALL: We've got assistance coming here. MR. BERMAN: Go ahead. THE WITNESS: Thanks. I'm David C. Gilliom. I work for the Federal Aviation Administration, and my business address is the Western Pacific Regional Office in Los Angeles, California. MR. BERMAN: Thank you. BY MR. BERMAN: Q And what is your -- your present position at the Federal Aviation Administration? A I'm the Manager of the Flight Standards Division for the Western Pacific Region. Q How long have you held that position? A For eight years. Q Would you please describe your duties and responsibilities as Manager of Flight Standards Division? A Yes. I would first distinguish -- excuse me -- that the field divisions are responsible for the operational aspects of our programs, and I bring that out because it's obviously distinct from what's done in Washington, which is the policy and rulemaking piece of it. So, as an implementing body, I oversee the functions of 17 field offices, and I ensure that those offices comply with internal guidance, and that the resources that are distributed in my region are used in accordance with agency priorities. I'd also point out that I chair a national committee. So, this is slightly different from the implementing role, and that committee has provided sponsorship of the ATOS System. Q Thank you. And could you please summarize your education, training and experience that you used to qualify for your position? A I've been in aviation for 32 years, with the FAA for 22 years. I have a military background, a concurrent career, with the Air Force, 25 years on active duty and as a reservist, hold an airline transport pilot certificate with 727-737 ratings, although I haven't been an active pilot for eight years. I also have a Master's degree in Business Administration. Q Thank you very much. MR. BERMAN: Go ahead, Captain Tew. INTERVIEW BY TECHNICAL PANEL BY MR. TEW: Q Good morning, Mr. Gilliom. A Good morning. Q Could you tell us what your role was in the development of ATOS? A Yes. I have participated in the development of ATOS since its inception, primarily in the oversight of the activities that occurred to develop the nuts and bolts of the system. I was involved with the 90-day safety review in 1996. Some of the recommendations that came from that review led to our internal look that eventually led to the decision that we needed to move forward from where we were in relation to our regulatory oversight systems. MR. TEW: Okay. For the audience's benefit, I should point out, we did yesterday, we'll do it again today, ATOS is the FAA's Air Transportation Oversight System. It's the new oversight system that is in place, and that's what we're discussing when we say ATOS. CHAIRMAN HALL: And it might help the audience if we had an explanation of what precipitated the 90-day review in 1996, you know, and your look at the system. THE WITNESS: The 90-day safety review was directed by at that time Administrator Hensen following the ValuJet accident, in which Flight Standards was under certainly some scrutiny regarding its current oversight system, and as a result of that review, there -- which was quite brief. It was advertised as a 90-day review. Actually, the primary work took place in about 45 days. So, it was very compressed and obviously not enough time to do a thorough system development. But as a result of that review, there were several recommendations that involved changes to the oversight of Flight Standards, and also changes within our own internal systems. It was an opportunity to look inward, to make sure that our own safety culture was an adequate one to support the activities that we're responsible for. Many of those recommendations were further developed as we worked with Sandia National Laboratory subsequent to the review to develop the Air Transportation Oversight System. We chose Sandia as a consultant because of their eminent expertise in the systems safety arena. CHAIRMAN HALL: Not because of the weather? THE WITNESS: Well, obviously not because of the weather. That's correct. BY MR. TEW: Q Could you tell us how much ATOS has cost to develop? A How much it has cost? Q Yes. A Oh, I don't have precise figures in terms of developing a system of its magnitude. The actual nuts and bolts of it have not been that expensive in terms of -- of buying outside consultancy help and facilitation help and documentation help, the kinds of things that the agency's not prepared to do internally. Much of the manpower to develop it came from our own resources. In fact, the subject matter experts that we used were, for the most part, line inspectors. We wanted to use their expertise in developing the new system. So, I can give you a rough estimate. I would guess from 1996 to '98, which is the initial development period, we spent roughly $2 million on development costs outside of our own internal investment, and again I'm talking about the costs of salaries for the people who were involved with the development and travel. But money that was spent on contractors, in the neighborhood of $2 million, and we still have expenditures on-going because there's extensive development continuing with ATOS as -- as we further develop the sophistication of the system. Q Was the ATOS System intended to replace the old PTRS System? A It definitely is intended to replace that system for Part 121 air transportation air carriers. It's not implemented to that extent. I'm not sure if it's been made clear that the system has only been implemented for 10 existing carriers. There's still a 140 121 air carriers that are operating under the National Program Guidelines and PTRS. Q At these 10 carriers, did the PTRS System stop when ATOS was implemented or is still being run in parallel? A It stopped to the extent that we use PTRS to record surveillance activities. There are still certificate management functions that occur that are reported through the PTRS activity, and I would say that there has been some transitional problems in terms of inspectors getting used to the new system. So, some functions that are borderline between certificate management and surveillance, and I would mention specifically initial operating experience, in some cases, that's still been entered in the PTRS Recordkeeping System. CHAIRMAN HALL: We might again observe for the individuals that are observing and keep them informed, this system was evaluated initially by the Board as a result of our accident that occurred in Miami with FineAir. It was the Board's recommendation at that time that the system be extended from the major carriers to all the Part 121 fleet. THE WITNESS: Yes, sir, and we have developed a plan to do that. At this point, we're somewhat reluctant to move forward until we've finished complete development of some of the process modules in the system and also have the resources available to move forward. CHAIRMAN HALL: Might finish staffing the ATOS for American first, and then we'll move forward. THE WITNESS: Well, that's certainly an important issue, and staffing is -- is one of the things that this system, once it's developed fully, will help us be very clear about what the needs are, and in fact, I think much of that is in place for American today. There's a new Comprehensive Surveillance Plan, as was mentioned yesterday, that was developed just last week for American. That plan is very quantifiable and should identify resource shortages, particularly in the geographic inspector arena, and there are policies and procedures to identify those needs. CHAIRMAN HALL: Dave, I don't want to be interrupting, but I would ask, what was wrong with the old system? Why do we need a new system? Just simply put, what were the deficiencies that you found, and what are the things you're trying to give attention to with this new system? THE WITNESS: Well, quite simply, the old system, which was very good and certainly established an excellent record of safety for the industry and FAA, had reached a plateau. There was really no way to make further safety gains with it. It was a system that was based on professional expertise of individual inspectors. It really wasn't a system so much. It was designed to inspect in safety. So, we attempted to do that by observing as many activities and surveillances we had resources to do. Actually, the data that was produced in terms of taking actions on that data, the predominant actions we took were enforcements, and I believe it was mentioned yesterday that that approach has limited effectiveness. The change to the ATOS model is an effort to move away from a system that's totally reliant on numbers of inspectors to again attempt to inspect in quality to a system that is focused on data and risk management and uses system safety principles not only to develop the FAA's system but to look at the air carriers' operating systems to ensure that those systems also are built around good system safety concepts. CHAIRMAN HALL: Wasn't it correct, David, that as a result of ValuJet, maybe there was a feeling that there weren't enough resources for -- that the major carriers basically had their safety act together, and that there weren't enough resources being expended on some of the new entrants and some of the other carriers? THE WITNESS: That -- that is a fact. I believe the FAA's situation is not different from other agencies. We probably will never have all the resources that we would certainly like to have and -- and could use. Our challenge is to build systems that leverage those resources and allow us to do our job with -- CHAIRMAN HALL: What has actually happened to the number of inspectors over the last 10 years, to your knowledge? THE WITNESS: Following the 90-day safety review in the ValuJet incident, we had an opportunity to substantially increase inspector resources. Due to budget contingencies last year, we were not able to do that, and this year, I believe we will be, at least at present, able to replace inspectors that we lose. But we're not in a growth mode, certainly. BY MR. TEW: Q You just mentioned that you just developed -- not that you developed, that a Comprehensive Surveillance Plan was developed for American, I believe? A Yes. Actually for all 10 of the carriers under ATOS. Q Okay. During testimony yesterday, I believe that it came out that there was not an analyst working at American, I believe. A That's correct. Q And therefore the data was not being analyzed, so it couldn't be used in a development of this. How -- the POI of the American unit said that he developed this comprehensive plan based on his experience, I believe. How can you develop an effective comprehensive plan for ATOS without the aid of an analyst? A The experience, I believe, that was referenced yesterday would be in the context of experience that came from ATOS data. The fact is that these comprehensive surveillance plans are developed using very detailed tools. There's a system safety analysis tool and an air carrier analysis tool that involve extensive processes to analyze the conditions that -- and the environment that face a particular carrier, in this case American, and from that, the plan is developed. So, it's not done on the basis of subjective expert opinion as somewhat the old plan is under the National Program Guidelines. This is completely different. It's very systematic. It's very detailed. It doesn't have to do with the analyst function. I do have one illustration that actually might be helpful in pointing out how the system works and where analysis comes into play. It might help understand while it's important to have analysts in these offices, we can still do what we're doing, which is an improved surveillance process, without the use of the analysts. CHAIRMAN HALL: Well, now you realize if you tell Congress that, they'll cut all your analysts? THE WITNESS: Well, I wouldn't attempt to say that this system is valid as a system without analysts. We do need the analysts. What is important to -- to point out is -- CHAIRMAN HALL: I mean I just -- I'm just asking because I mean either they're part of the system or they aren't part of the system. As a taxpayer, I'm not interested in funding accessories. THE WITNESS: They're a hundred percent part of the system, and what was said yesterday about the ability to analyze the data that comes from this system is limited without them. I would also point out, though, that -- that among the activities that is on-going to -- to enhance the system is further development in the process module that deal with the analysts. So, we're working on the procedures that the analysts will use right now. CHAIRMAN HALL: Have you got a diagram you want to put up? THE WITNESS: Yes, I think it would be helpful. CHAIRMAN HALL: Victor? THE WITNESS: This diagram shows the system that Sandia Labs helped us design, and again we used our own line inspectors basically to help develop this model. It has eight processes, and they're represented by the boxes, thank you, and if -- if you -- CHAIRMAN HALL: Can people in the back see the writing? Make it out? THE WITNESS: Well, -- CHAIRMAN HALL: So, you might read it. THE WITNESS: -- I'll read it. You know, just beginning with this first box, it says, "System Configuration". That, we would call a process module. There are eight of them around the circle, obviously linked in a closed loop system. So, the first one, System Configuration, deals with the initial certification of operators. It's our certification activity, and it also deals with how our office or how the FAA's internal structure is prepared to deal with a new operator. CHAIRMAN HALL: For the benefit of the audience, and I think people have grasped this, but basically -- and put the chart back up, Victor. Basically, what you're talking about is how we spend our tax dollars through our Government to oversee the aviation safety system, is that correct? THE WITNESS: That's correct. When -- CHAIRMAN HALL: And this is a -- you know, the latest of how you all have put together an effective way to do that. THE WITNESS: That's correct. In fact, I would advocate it as such a major step forward, that it truly is the leading edge of a regulatory system. So, I believe it puts us in a posture again to be making a world standard or setting a world standard. But going quickly around the system as it's designed and represented by this model, after initial certification, the next process is certificate management. It's this process that develops the Comprehensive Surveillance Plan. CHAIRMAN HALL: And every major carrier, every Part 121, has a certificate, right? Is that -- THE WITNESS: That's correct. CHAIRMAN HALL: That gives them the right to -- THE WITNESS: That's correct. CHAIRMAN HALL: -- use the skyways that belong to all of us? THE WITNESS: I believe it will be clear once I have described briefly this entire model, that its application to new entrant carriers, and again when -- when you think about how this came about after ValuJet, that's one of the major things we wanted to deal with, improved quality of our certification services, that it will be clear this model really is -- can be very effective, will be very effective for new entrants and the remaining carriers that are not under it today. In the second module, as I said, the Comprehensive Surveillance Plan is -- is developed. The third module deals with the resources required to carry out that surveillance. So, this is where we're ensuring we have enough people, that they're properly trained, that there's enough money to go out and do the actual work. The fourth module is surveillance implementation, self- explanatory. It's where we carry out the Comprehensive Surveillance Plan. The fifth module deals with reporting. This is the module where we have moved away from PTRS, which is basically a work program tracking system. That's the old model. This one is -- is in truth much improved from a PTRS activity. It allows us to enter much richer data fields. I should point out that in surveillance implementation, one of the criticisms of the Board and others with which we agreed was that our old system did not require the extensive use of checklists and job aids to complete an activity. We could call an activity complete no matter how rudimentary our surveillance work had been for that activity. Under the current system, there are extensive job aids which must be completed before either an element performance inspection or a safety attribute inspection can be called complete. The data is reported and entered in the fifth module. In the sixth module, evaluation. We do something with the data we've never done before, and that is we evaluate it to make sure that it is complete and accurate, and that it meets the standards that we're looking for with this type of data. The seventh module, after the data's been validated, it's then analyzed to determine what it's showing us. From the analysis, we are looking for root causes, trends, other things that would be of use to bring about changes or to confirm that in fact the carrier's continuing to operate in accordance with its design systems, and that those systems are well-designed. And from the analysis, then clearly there could and will be actions that -- that come forth, even if that action is -- is simply to document that we have confirmed the carrier's doing what it said it would do and doing it well. It is a closed loop system. So, from what we've learned from our data collection and analysis, if there are changes that are required, we go right back into Module 1 to change operations specifications or whatever else might be required to bring about the necessary changes. CHAIRMAN HALL: And as it pertains to just American Airlines, the loop right now has problems that Number 3, Number 4 and Number 7? THE WITNESS: Number 3 and -- Number 3 is certainly a problem in that that deals with staffing issues. CHAIRMAN HALL: Hm-hmm. THE WITNESS: However, there are clearly-developed policies and procedures to identify those staffing needs, and I would expect, now that there is the first -- actually the second surveillance plan in place, that as resource needs are identified, that they will be elevated. Last year, I can say in my own region, Western Pacific Region, when the American Certificate Management Unit requested new inspector resources, we were able to provide those in Western Pacific. I can't speak for other regions. So, there is a process in place to deal with the needs, and again if that were to happen this year, I have no doubt that I would be able to meet the requirements of any ATOS carrier in terms of surveillance needs. CHAIRMAN HALL: I'm just asking as an observation today, do we still have -- we still have some challenges -- THE WITNESS: Yes, sir. CHAIRMAN HALL: -- in 3 and 4 and 7? THE WITNESS: I would say there are challenges in those modules. I would also point out -- CHAIRMAN HALL: I'm in the position, too. I sometimes don't have enough resources at the Board, and I don't like to ever admit I don't have enough resources in terms of being able to do the job. I want to be able to say yes, we're able to do the job. But the truth of the matter on that situation is you got 3, 4 and 7 for you to really have your loop closed like you want it closed, and you've designed it, need -- need attention. THE WITNESS: That's correct. I would also point out that the predominant work that has been done on this system to date deals with surveillance. We are working currently on the certification activity. I'm sure the Board is aware that we have a new Certification Standardization Evaluation Team, CSET. CSET is part of this model. Its function is primarily in that first system configuration module. They are using procedures in a much more standardized way than we have used them in the past, but they still predominantly are our old procedures. So, we have extensive activity. It will take time to finish on-going in that module now to convert or re-engineer our certification activity to comply with the system safety concepts. CHAIRMAN HALL: Good. BY MR. TEW: Q Just for clarification, the Chairman mentioned that there were deficiencies in three of the modules. There's eight modules here in this closed loop. Could you tell us how many have been fully developed of the eight? A The modules that have had the most work are all of those dealing with surveillance. So, it would be 2, 3, 4 and 5. Now, again, the problem that we're having with Module 3 is not so much that there aren't processes that have been developed; it's the ability to have full resources to carry that work out. Again, it gives us tools, I think, to better use the resources that we do have, but as the Chairman pointed out, and as I said earlier, government agencies will always be more in need of resources than are generally available. Q So, we really haven't achieved this closed loop yet then, I believe? A We have not. I think it was a valid point, for example, that there's not an analyst in every office. I mentioned that that's one of the modules that we've done rudimentary work on. We need to do much more in that module. When that work is completed, there will be -- the analysts would be better used, not to say that they couldn't function in the system as it's designed today, but that's clearly an area where there's still need. Q How do you develop an analysis module in this work? How do you develop this without the help of the analysts? Wouldn't they be an aid? A We -- well, there are two ways to provide that expertise. We do have a function in the Headquarters organization that is staffed with operations research analysts. I believe there are three or four analysts on board in that Headquarters staff. We have the one analyst hired in Southwest that gives us a core group of analysts as subject matter experts to use internally. Also, we are using contractors to help with this activity. We're continuing to use Sandia to help with this activity. All of those outside sources can provide expertise and in some cases an external input which is very important to have. So, that's where the expertise would come from. Q So, you're using staff analysts and then outside help? A That's correct. Q Is -- is -- is this effective in analyzing this data then? Because this is -- A Well, when -- when you asked me the question about developing the analyst module, we're -- when it comes to analyzing the data, which is an operational concern, we're still somewhat deficient in that area in that again we only have the one analyst on site at Southwest. This group in Washington has a limited capability to help with analysis in CMTs. However, I think as -- as Corky Valentine testified yesterday, although there isn't an analyst present at American Airlines, clearly the data which he said was much improved from the PTRS data is being looked at by Corky and others. It is being analyzed, certainly to the extent that we analyze PTRS data in the old system. So, there is a rudimentary activity going on in that process module. There are actions coming out of that analysis. There are changes to comprehensive surveillance plans in terms of targeting that are occurring. So, again it's a rudimentary function. We need to take it much further, but -- but it is occurring, and -- and there is a difference between the development activity and the operational activity. CHAIRMAN HALL: Well, David, as you know, they said -- they had told us in correspondence that this thing would be up and running. Has the budget cuts -- if you hadn't had the budget cuts, and you'd had the staffing, would this thing be ready to take off the shelf now? THE WITNESS: Yes, but I don't want to under-play the fact that -- two facts, actually. That there's extensive work to be done with initial certification. We need to develop better analytic procedures. We need to develop better implementation procedures, and this model is designed for continuous improvement. One of our past mistakes is to fix -- CHAIRMAN HALL: That's life, though, right? THE WITNESS: It is life, but -- but we, unfortunately, didn't learn the lesson as well as we might have in that we have always tried to fix it once, once fixed, keep it in place. That's not the intent here. We realize that we need to continuously revise this system through a continuous improvement process, and that will be on-going throughout the life of the system. CHAIRMAN HALL: Good. THE WITNESS: And I think we're in a much better posture to do that than we were with the old system, which was quite linear. It was not a closed loop system. CHAIRMAN HALL: But the reality of the old loop system was that we used to have days that we -- we tried to -- we had enough inspectors, and the Government -- the way the Government approached inspecting has changed, and a lot of that's changed because of the realities of Government being cut as well as all the modern philosophy about things, in my personal -- now, that's my personal opinion. As we sit here trying to do the job of regulating an industry that has grown in about 10 years with essentially the same resources, you're put in the situation of trying to look at different ways of accomplishing the same goal with the same manpower. THE WITNESS: That's largely true, but I would point out that the value of this system as it matures is that it does allow us to leverage resources through better targeting, and -- CHAIRMAN HALL: I mean the Board's on record. We -- we think it's a good system. We just want to try and get it -- the loop closed. BY MR. TEW: Q You said this system was -- it was put in place. It wasn't finished, and it was designed that -- to be improved as it goes along. But, you know, four of the eight modules, I believe, were not developed to start with, and none of the analysts were in place. Do you feel possibly that it might have been put into place just a little bit premature as a replacement for the PTRS System? A Well, again, all the modules were developed to an extent. Four of them were worked on more extensively than others, and the remaining four had rudimentary improvements. Even the initial certification module, which -- which we're re-engineering now, was substantially improved when we put the CSET team in place over what we had before. So, all the modules were developed to an extent. We did have a complete system. We did make a conscious decision to implement that system, knowing that there was further work needed in some of the modules, simply for the reason that after working on it for two years, and government things tend to become either another study that gets put on a shelf and never implemented or you move forward, we were very convinced that what we had done in the surveillance arena was a substantial improvement over what we had before, and I think again that came out in testimony yesterday, that the data coming from the system is much richer than the data that we were getting from the old system. Q How were the airlines and industry representatives involved in the development of ATOS? A The airlines and industry were -- were not involved to any great extent in the initial development of the system. We realize that industry input is important. We have sought it. Certainly once the system was initially implemented, we've had meetings with ATA to discuss the system, to make sure it's understood, to get some feedback about its shortcomings from that perspective. CHAIRMAN HALL: You might explain, David, who ATA is. THE WITNESS: The Air Transport Association, which represents most of the major carriers in the 121 arena. So, we are getting that input now, and we certainly will get the input as we develop the initial certification activity. BY MR. TEW: Q You just mentioned a minute ago that you -- this is -- this program is the leading edge of the -- of a regulatory system -- new regulatory system. I understand that a significant number of the questions in the inspection reports are not based on the FARs or any regulatory basis. How can you collect data and analyze data that is questionable or -- and non-regulatory and expect to achieve compliance? A I don't believe the data is questionable. I believe the data are legitimate areas for FAA to -- to be concerned. The Board and others have pointed out that simply looking at the regulatory requirements for compliance falls short of what's needed to take the next step in safety in terms of looking at corporate cultures, safety attributes, a number of concepts that are standard in the safety business, whether it's nuclear safety or safety in airlines. These are all legitimate areas for the FAA to collect data in order to target its resources. When it comes to insisting that certain changes be made, clearly we're limited to what we can enforce to what is written in our regulations. However, that doesn't limit what we can look at and what we can bring to the carrier's attention as a concern. One of the things that's very important about this system, and I believe it came out to some extent yesterday, is that one of the action items that can occur is a system analysis team or a SAT that did occur with the American unit, and it would correct some perception that I believe existed yesterday in that regard. A system analysis team is not an audit team. In fact, it is a team that undertakes a problem-solving activity. It's not something that can be done unilaterally by FAA. It necessarily involves -- requires the involvement of -- of other parties to do it properly. We can't problem-solve for American, for example, and in that regard, I believe that the other two activities which were mentioned probably were legitimate audits, as I understand, done by outside auditors hired by APA and by American to -- to do an audit of a certain function for which it wouldn't have been appropriate to include FAA's team members, although I do understand it's not my -- the certificate clearly isn't in my region. I do understand that there were extensive briefings from those audit efforts to the CMU or CMO as it occurred, but I think it is important certainly to distinguish again that this SAT or system analysis team is a problem-solving activity that looks at specific trend data or any set of data that may indicate a problem or a concern, and the attempt then is to identify that -- that risk or hazard, to mitigate it, eliminate it or in some way manage it, which is what occurred here. The recommendations that came out of that SAT team, I believe 17 of them have been implemented already by American, which is -- obviously without some follow-up implementation, it's not an effective process. I believe that does prove that as a tool which is part of ATOS, there are clearly good things happening that -- that are making real changes. Q Thank you. Under the old system, you could put an airman's name into the system and check/see how many inspections he's had, what areas he was inspected on, and you could do the same thing with an N number of an airplane. You could put it in and check on a particular airplane and see what oversight had been done on that. Why does the ATOS System not allow this? A Well, to -- to say that it doesn't allow this, I'm not sure it's quite accurate, and I -- I believe that we -- we still are capable of deriving from the system inspection work that we have done involving a particular aircraft or a particular airman. But I would point out again that we're trying to move away from this technology that won't take us any further in terms of looking at individual pieces of the system for a quality control-type activity, that -- for example, it was mentioned perhaps nobody's looking at every piece of American's system wherever they operate. Well, that -- that well might be true, and we do that with some confidence. If we're sure that American's system was properly designed, and by what we have looked at, that American is complying or continuing to -- to operate with what was designed initially as it was designed. In other words, they're keeping integrity in their systems. So, we may not have to look at every airman or every check airman or every facility in order to have confidence that the system that American designed and is -- is practicing is in fact being carried out the way it was designed. Q Thank you. I understand that inspectors are having difficulty in interpreting the safety attribute inspections and the element performance inspection questions, and in fact, inspectors are responding differently to the same questions. If this is so, how do you achieve quality data, and how do you achieve standardization? A Through lessons learned, and again this is a continuing improvement process. We did implement ATOS, and since implementing it, we've -- we've learned several things about shortcomings in the initial training and the design of the initial questionnaires and checklists that we're using. We have revamped and redesigned both the training program and the checklist for EPIs, and we intend -- and -- and the procedures as well. We intend to conduct seminars in the near future with all of the CMTs, the complete units, all the inspectors involved, in other words, to ensure that there's an understanding of how to use those new tools. So, we're moving forward in all the areas you mentioned. Q Okay. Under the -- the General Accounting Office did an audit, and they mentioned in there that the training was ineffective. Is this what you're saying? A That's correct. In fact, the training that was offered was developed and provided in a fairly special environment in order to quickly train the inspectors that would be involved with these 10 carriers or the initial cadre. Since that time, the training has been moved to our Academy in Oklahoma City, where it's now resident, and there's been several -- there have been several changes made to the curriculum. My understanding from talking to our people who have come through that new program is that it's a very good course. Q Also according to the same GAO report, the guidance for planning and performing inspections was not very effective. Has this guidance been reviewed and improved? A The -- yes is the answer. The -- the job aids that are being used to do element performance inspections have been revamped, field tested, and they are ready now for implementation. That should occur within the next month or so. There was a decision made that the guidance used on the safety attribute inspection is adequate, but there's further explanation needed in terms of how to apply it, and that will be accomplished through the seminars that I mentioned previously. Q Under the ATOS Program, since the inspections are kind of specifically targeted, what method is there for identifying and reporting problems or areas of concern? A The -- there are two methods actually. Because these inspections are very detailed, they take time, and our reporting process is -- is one that allows entry of data as the inspection is on- going. Initially, CMUs, certificate management inspectors, could not access that data easily. That's another correction that we've made. So that as an inspection activity is on-going, the -- the information is now available to our principal inspectors. At the same time, we believe that with this concept, there is the capability for much improved communications between the geographic inspectors and principal inspectors. The good old phone still is -- is a -- is a very usable system. Now that we don't have a 190 geographic inspectors doing generic work on American Airlines but rather have a much smaller team, working almost full time on American, it's easier for that team to communicate to the principals when they find immediate problems that need attention. Q Once again referring to the GAO report, could you explain why an airline's screening, boarding and cabin briefing procedures are given more weight than cockpit procedures in determining the number of inspections that an ATOS team is to conduct? A I'm not sure that I would agree that they're given more weight than those other factors at all. In fact, I believe the CMT has a great deal of latitude on where to place its emphasis. It could be for any given carrier, if the data show that the training and cockpit procedures are being adequately executed as designed, but cabin safety procedures are not, that we would focus our resources in those areas, but there is flexibility intentionally built into the system, and targeting is to occur based on the data that we're deriving from the system. CHAIRMAN HALL: How does the system deal with that manual situation, where the manufacturer makes the change and half of your major carriers implement it and the other half don't? Do you all communicate on that and try to -- is there any policy or -- THE WITNESS: There are -- there are things that the system can do to deal with that issue. I would point out, outside of ATOS, that the agency recently has made available through a web site all changes of that nature, changes to AFMs and bulletins from manufacturers, and we've sent instructions not just to ATOS principals but to all of our principals telling them to check that web site in order to determine whether there are applicable changes that are recent, and, if so, to take those changes to the operator to ensure that the operator's aware of them. We still have some work to do in that area. Again, because this is a closed loop system, when we find those kinds of problems with one carrier -- one of the modules that I didn't point out, it's a ninth module on the schematic that I displayed, is the policy module. That's really the headquarters work function. But the system allows us to -- to insist actually that we provide feedback to that policy group when there are problems that need dissemination to a broader audience or that need the attention that only the policy group can deal with. So, again, I think that the system is in place. It's a matter of refining it and making it work to our advantage. CHAIRMAN HALL: And under ATOS, how are you going to evaluate training and oversee training? THE WITNESS: Carrier training? CHAIRMAN HALL: Carrier training, yeah. THE WITNESS: The -- the system is -- is designed to do inspections in 88 different elements of air carrier systems. The system has defined some generic air carrier systems as part of the model. There's seven air carrier systems that we're looking at, 14 subsystems, 88 system elements. We do element performance inspections in each of those areas. In fact, one of the major air carrier systems is training. So, all the subsystems and elements under that activity have specific protocols that we use to -- to examine how the training's been put together and how well it's being executed. CHAIRMAN HALL: And you have -- we heard yesterday that your inspectors have average of 38 years of -- 36-38 years with the service, with the FAA. So, I guess the majority of that 38 years, they've been using the old system. So, how are you going to train these gentlemen or ladies with all this experience to do this new job without having gaps and -- and problems? Life's full of problems, though. THE WITNESS: Well, -- CHAIRMAN HALL: Disregard the problems, the gaps. THE WITNESS: When -- when you're speaking of gaps, I'm assuming that it's in reference to bringing on new people or these -- CHAIRMAN HALL: Bringing on your new system. I mean it -- obviously this event occurred during the transition of these two systems. Was that an impact or not an impact? Whether this was an audit or a problem-solving thing, ATOS is designed, I believe, to prevent incidents such as this occurring. THE WITNESS: Absolutely. CHAIRMAN HALL: Not evaluate them after they occur, and what we have here is we've had -- whether it was an audit or problem-solving, is it 13 or 17 different items that -- that were noted by the carrier, the union and -- and everyone, that were of an urgent - - we were told of an urgent situation. That's my -- my concern -- observation last night was if I was in the situation of government oversight, that's -- that's a reflection and a concern to me. How did that occur? THE WITNESS: Well, I think it's a very good point, and clearly the FAA's very reliant on the expertise that our inspectors bring to the agency, and we have focused in the past on enhancing that type of traditional expertise. Clearly we have a need with this new model to focus also on training that develops skills, such as auditing skills, system safety skills, and risk management skills. These are not skills that are particularly common to mechanics, pilots and other people that operate in this industry. So, we have a clear requirement ahead of us to identify those needs and to develop appropriate training courses that can provide for those needs. That's -- CHAIRMAN HALL: And attract people to Oklahoma City to take the courses. THE WITNESS: That's true, also. CHAIRMAN HALL: Go ahead, then. BY MR. TEW: Q All the data that's been collected so far, has it been entered into the system? A Yes. I say yes in that one of the things that -- that has always been a problem, and I believe we have the ability to -- to work on quite effectively with this system, is that inspectors take corrective action on site without entering data into the system, and therefore valuable information is lost to us. Again, this system demands that inspectors use protocols that require documentation before an inspection can be called complete. That forces the issue of data entry much more effectively. It's not to say that we're still not losing some valuable data, and I think that as we move forward with some of these safety programs that have recently been approved, particularly the partnership programs, gives us the ability to collect data in a much more extensive and valuable way. Q Well, following up on the Chairman's question, I understand that a number of inspectors are having a good bit of trouble working with the system, entering the data, that it's not really user-friendly. A Well, two things about that. First of all, the system uses a web-based technology, and we are -- our data entry was dependent upon accessing the database through the Internet, through modems, when we initially implemented the system. So, the speed and user- friendliness was not so much system design as -- as how well the Internet service provider provided that service. We now have moved to the database inside the FAA's fire wall. It's on the FAA Intranet. It's accessible through our wide area network. The speed of the system has substantially improved from the people I've talked to. Those types of problems have -- have been virtually eliminated. In terms of the ease of entering data and the ease of what we're doing, I would be the last to say that ATOS was designed or that it will make the inspector's job easier. In fact, it's tougher. It's tough work. It requires a great deal of expertise. It's -- it's not easy to -- to do what we're asking, and yet the only way to improve the data quality is to ask people to do this work and do it well. That's one reason, also, that it's not so well accepted. Change is very difficult to bring about in any organization, such as ours, or anyone else that has an organization of comparable size or complexity. But this -- this is tough work. Q Okay. Thank you. The FAA spent $95 million on the Safety Performance Analysis System or SPAS, as it's called, so that it would analyze key aviation data, identify trends and potential safety concerns, and target inspection resources accordingly. Why isn't the ATOS System linked up with this existing Flight Standards Aviation Risk Analysis System? A SPAS is -- is an excellent system, still being used with a 140 carriers that are not under ATOS and will be the primary analytic engine for the analytic module process once that process is developed and once SPAS has been re-engineered, so to speak, to use ATOS data. That work is currently on-going. The initial interfaces should be done this fiscal year, and within two to three fiscal years, SPAS, as I said, will become the primary analytic engine for ATOS. Q Since it doesn't currently link with SPAS, how do you ensure that the information gathered by the ATOS is sufficient to allow the agency to perform critical trend and safety analysis, also since the analysis module is not developed? A Well, again what -- what is lacking for principal inspectors, such as Corky testified yesterday, is an analytic tool, either through an analyst or through a tool such as SPAS. So, the analysis that's being done has to be done based on what he and his staff are able to do simply by reviewing the data and identifying trends using manual techniques. Excuse me. So, there -- there clearly is work to be done in that area yet. We do have plans to do it. It's on-going now. Q ATOS was originally designed that all new entrant airlines would come under ATOS. Is this being done? A It's not being done. Again, as I mentioned, we're reluctant to move forward in expanding the application of ATOS until we've done two things. One, finished the work that is on-going now to make the remaining four process modules more sophisticated in their activities, and, two, till we have a fair comfort level with the resources that are needed to move forward with the system. Q You touched briefly on this. Has FAA upper management received any complaints from inspectors in the field and the principal operating inspectors? What are the complaints, and what is being done to respond to them? A Well, I believe we have touched on this extensively actually. We have certainly received many complaints. As I said, some of those complaints are based simply on not wanting to do something different that's harder work. The other complaints that -- that truly are valid have all been, I think, dealt with quite effectively, either through the improvements that we have made in terms of job aids, automation, changes to procedures, some of the educational changes that I've talked about, or they have been duly noted as changes that need to be made as we re-engineer our initial certification activities and continue to develop the analysis, evaluation and implementation processes. Q Have you uncovered any major safety defects as a result of the ATOS Program? A By that, do you mean have we discovered safety defects in air carrier operations? Q Yes. Have you uncovered any major safety defects in the operations of the airlines themselves? A Yes. I mean in -- in the regard that the data and the surveillance that we have done have identified problems that need to be fixed, yes. Again, I believe that you can go to any of the 10 initial cadre carriers, the Certificate Management Units, and find examples where there have been problems that led to system analysis teams and eventually actions that were taken to correct problems. Q Thank you, Inspector Gilliom. MR. TEW: Mr. Chairman, I have no more questions. CHAIRMAN HALL: Well, I guess I see Mr. Feith turning his microphone on. Do you have questions, Mr. Feith? MR. FEITH: Yes, sir. CHAIRMAN HALL: Please proceed. BY MR. FEITH: Q Mr. Gilliom, you were talking about the fact that there are no analysts in all of the offices, and that that function has now fallen on to basically the POIs and their staff. Given the level of detail now that you're receiving through this program as far as the information that's being gathered, how much time in your view do you see the POIs and their staff having to take to analyze this data, and doesn't that compromise their ability to be out in the field doing the surveillance work and doing the inspections rather than being in the office trying to analyze that data? A I think there are two -- two important things in this regard. First, the time that's spent is no more than the time that was spent under the old system. So, again, while we don't have everything we need for this new system, we -- we -- we certainly have no less than what we had before. Secondly, the inspection work or the surveillance work that -- that you're referencing really shouldn't be done by the principal inspectors. They're there to manage the certificate activities. When they need inspection work done, they should rely on the geographic inspectors who are there to do that work, and in fact, when we finally have the training in place, as I mentioned, the audits, system safety, risk management training in place that's needed, that area of expertise of doing that surveillance will be highly specialized. It's not something the principals should be focused on doing. Q Okay. Now, two things. First off, you've got more -- more information. So, how can you be taking the same amount of time as in the old system, evaluating new data that has more detail to it and still be able to do it in the same period of time? A Well, -- Q And, second, -- and then, second, you were talking about that the POI relies on the geographic inspectors, and as we heard yesterday from Mr. Valentine, he's got a geographic inspector sitting in Las Vegas where he really needs the inspector in New York because that's a hub, that's a big operation, and he can't rely on anybody there because he doesn't have anybody there. A Right. They're both important points. First of all, the data may be more in quantity. I couldn't verify that. What was testified to yesterday is that it's much improved in quality. That quality improvement could in fact make it easier to analyze now. I'm not here to say that that's ideal in terms of that we do need analysts for this system. So, that's a clear need that is ahead of us. But the -- the enhanced data should be easier to work with in the long run. Secondly, the -- the workforce that is in place today to do the surveillance work may not be in a most efficient place to do that work. That's for a reason. We had to use the people that we were using before. The person, for example, doing operations work in Las Vegas for American Airlines is the same person who is there doing that work predominantly under the NPG and PTRS activity. We couldn't just eliminate that person from the system. We had to keep using that person, even though he's not in the most efficient place to do the work. When he moves on, the position will not exist in Las Vegas any longer. It will go to in fact wherever American Airlines Certificate Management Unit tells us they want it. If that is in New York, we'll do our best to fill the position in New York. In my region again, this is my personal experience, we have moved many of our geographic inspector positions from where they began to where they are now most needed. Second point in this arena is that that person in Las Vegas is not limited to Las Vegas as he used to be under the old system. It used to be a very confined geographic area in which that person operated. That person now has the authority to travel wherever he needs to travel to do the work under the Comprehensive Surveillance Plan, including offshore, because there's a fair amount of international work that needs to be done. Q Okay. Following up on that, we heard yesterday in previous testimony that there are travel restrictions because of budgeting. A Yes. Q So, now you've just told me two different things. A I -- I don't believe -- I didn't intend to do that. There -- there are always travel restrictions in relation to budget. However, we this year have funded the Certificate Management Units, each of them, with twice as much travel funding as they had last year. It's roughly a $100,000 per unit. We have the capability now of costing this activity much more accurately than we've had in the past. We're going to continue to try to use the existing resources to serve the priorities that are specified by the agency, and clearly surveillance of Part 121 operators, whether they're under ATOS or not, is a high priority for us. Q Do you have funds in Western Pacific Region to do all of the things you need to do with your people? A Well, you asked a very sweeping question, and my answer will be more specific. I believe we have the funds to do what's needed under this ATOS model with the United America West certificates which my region's responsible for. I think that will be true across the board for the 10 certificates we're dealing with. Notwithstanding that, again I don't believe government agencies will ever have all the resources that they think they need to do the work that's there. So, we're prioritizing in Western Pacific Region to make sure that we do the first things first. Q Okay. And I'm not going to belabor this, but I just want to get clarification. Let's say towards the latter part of the year, you see a projection that you're going to run short of funds to fulfill the mission. How hard is it to get additional funds to complete the work necessary for that particular year or is that going to be sacrificed? A My experience with the agency is that when a true need is identified, such as you're talking about, it gets met one way or another. Usually it means by creating pain in another part of the agency. Within the things that I have control over in my own region, if I see that situation arising, I'll be moving resources from General Aviation activities to serve this priority, and, similarly, the agency will attempt to cope with that. Again, we're dealing with some tough situations in the long run, and as I pointed out at the beginning of this, there's a limit to -- to what I can provide valid testimony to in regards to resources and probably have reached that point. Q Let's see. In this -- in this transition period from old to new, and the fact that the training had been identified by a GAO report, and you've acknowledged that the training was basically insufficient for the inspectors that are using this system. A It was -- I -- I can understand why you would use that word, and certainly others have. It's not how I would characterize it, but it needed improvement which has occurred. Q Well, it's our understanding, we've -- I mean prior to coming to this hearing, we -- we have talked to a variety of people, and they told us that it was ineffective, that it didn't give them the level of knowledge necessary to start this new system. They had a lot of problems with it. It was very time-consuming in entering this information into the system and basically bogged them down, and it was their opinion that they weren't able to do the job that they were being paid to do because of this time-consuming transition, and I know that there's been changes that have been evaluated throughout this past year that the system's been in place, but have those changes been filtered down? Have the inspectors received additional training or recurrent training on these changes that are continually being -- A Well, again, I believe the answer is yes, and if you look at the testimony from yesterday, I think there was a clear distinction drawn between the first activity to develop the initial Comprehensive Surveillance Plan a year ago versus what occurred last week, which was an enhanced process. I think it went much more smoothly in all of the CMOs, and, frankly, I think we got much better plans out of it. So, yes is the answer. We have made improvements that have been effective that are in place. Q Okay. Thank you, Mr. Gilliom. CHAIRMAN HALL: Okay. We'll move to the tables now. The National Weather Service? MR. KUESSNER: No questions, sir. CHAIRMAN HALL: Little Rock National Airport? MS. SCHWARTZ: No questions. CHAIRMAN HALL: Little Rock Fire Department? MR. CANTRELL: No questions, sir. CHAIRMAN HALL: The Federal -- excuse me. I'll hold the Federal -- Boeing Commercial Airplane Group? MR. HINDERBERGER: No questions. CHAIRMAN HALL: American Airlines? MR. BAKER: We have no questions, Mr. Chairman. CHAIRMAN HALL: Allied Pilots Association? MR. ZWINGLE: No questions, sir. CHAIRMAN HALL: The Association of Professional Flight Attendants? MS. LORD-JONES: No questions, sir. CHAIRMAN HALL: And the Federal Aviation Administration? MR. STREETER: A couple, sir. INTERVIEW BY PARTIES TO THE HEARING BY MR. STREETER: Q The -- I will, first of all, start off with the ATOS data. Is that the only data that is planned to feed into the analysis process? A No, it isn't. We in that process want to look at frankly the entire world of data that's available. So, not only FAA data, such as accident and incident data, enforcement data, other things that come from FAA databases, but in the long run, the system is really designed to use the wealth of data that the carrier's generating, and I believe through our new partnership programs, there will be a variety of data inputs from that activity. Any data is valuable data, and as we develop the analysis module, we will be, first of all, determining exactly what questions we need to answer in order to effectively manage the certificate, and any data that helps us answer those questions, we would hope would be available and used in the process. Q I believe you mentioned that the Comprehensive Surveillance Plans were finished for most of the ATOS carriers within the last week or so. Do you view that as a static document then for the remainder of the year? A No. The system and those documents are intended to be flexible. It's quite different from the old system, and for that reason, it's unlikely that any of these surveillance plans would be in a strict sense 100 percent completed in any given year because the intent is, as we learn from the surveillance we're doing, to shift or retarget our resources from areas where there are not problems to areas where there are more likely to be problems. Q Okay. So, then, if you saw a surveillance plan that had been retargeted one or more times during the year, would you consider that a normal -- A That's excessive. Q Okay. Thank you. I have no further questions, sir. CHAIRMAN HALL: Mr. Sweedler? MR. SWEEDLER: Yes, Mr. Chairman. I just have one question. INTERVIEW BY BOARD OF INQUIRY BY MR. SWEEDLER: Q I think the testimony has been that only one of the ATOS carriers has an analyst. Could you give us some idea of what the planning is to add the other nine analysts, and how long it might take to get them on board? A Planning to date was to hire at least four more analysts this year, and that's on hold due to a hiring freeze that is currently in place. Analysts are new to us. So, these are not internal hires. These have to be outside hires, and the population that's available to fill those positions is still somewhat of an unknown, and until we know exactly who we're dealing with, it will be hard to determine what training may be necessary, and again this is one area where we're still doing extensive work, and we know that's ahead of us. But our plan at least would be to move forward to that extent this year. Q So, not only do you have the problem of trying to fill the positions that you're not usually familiar with, but you have the hiring freeze to deal with? A That's true. Q Thank you, sir. CHAIRMAN HALL: Mr. Berman? MR. BERMAN: No questions, Mr. Chairman. CHAIRMAN HALL: Mr. Haueter? MR. HAUETER: I have a couple. BY MR. HAUETER: Q As I mentioned yesterday, under the NASIP Program, which no longer exists, there was an internal audit function where FAA inspectors in different regions came in and looked at another region. How is that being handled now under ATOS? A Well, first of all, I would correct something that -- that I think was -- was said yesterday about NASIPs. They have not been eliminated. They've always been available on request to -- to be accomplished, and, so, the program is -- is still there. What has been decided and what is on-going is that the NASIP Program, like our old surveillance program, is not adequate to do the work that needs to be done in the future in order to make the advances that we'd like to make. So, it also has been redesigned in terms of -- of along the same system safety lines, so that when we do a NASIP, it's not simply a compliance inspection but it will look at the very things that the Board, I think, has rightly pointed out need to be looked at in terms of safety cultures and management structures and the systems of the carrier. Also, the program will be turned internally to look at FAA's own capabilities and how well we're doing our job in terms of our own culture and structure and compliance with our own procedures. So, it's -- it'll be a two-pronged approach in the future once it's redesigned, but it still is there. We don't intend to eliminate it. I also would point out that Certificate Management Teams have two components, the Certificate Management Unit are the principal inspectors who are there to manage certificate activities, and the geographic arm of the team which is doing surveillance. By design, those geographic inspectors, while they are team members, do not report to the same supervisors or managers. Those people do come from other regions. They are independent. We designed it that way as a control, so that those people shouldn't feel pressure to not enter honest data as a result of their surveillance. So, in fact, we have a full-time look now by those people from inspectors who are outside of the region, and hopefully we will keep some level of discomfort between those two elements of the team in order to maintain that objectivity. Q I guess a follow-up there. Do you have any idea when the NASIP Program will be redesigned and available again? A I -- I -- I know that it's an activity that is on-going. I don't think it's so extensive that it won't be done this fiscal year, but it's not a program I'm working directly with. So, I don't -- I don't know the dates. Q Okay. If a geographic, you know, inspector were to note problems in how a principal operations inspector was running the shop or something else, how would he get that information up higher in the FAA? A Beyond the Certificate Management Team? Q Right. A Well, first of all, because of this separation of powers, so to speak, there's nothing in the system that can override whatever data the geographic inspector wants to enter. So, even with the evaluation function that is there to validate the data, there's -- there's no way that whatever that person has put down into the system, that it can be eliminated. In fact, to some extent, that person, if he makes an error, it's fairly difficult to correct that error. It can be corrected, but there will be a record of the change. So, there's -- there's that type of integrity built in. There's always -- there are processes in place to elevate concerns outside of the CMT. Part of those are covered by our collective bargaining agreement, and again we're looking at our own safety culture to make sure that there's no impediment to raising safety issues to whatever level's necessary to address them. Q Will ATOS be customized for each carrier? A It is customized for each carrier. I mean that really is part and parcel of the whole concept. Before, our geographic inspectors were very generic. Some of them were -- were trying to look -- do inspections on 20 or more carriers, couldn't be experts in the systems of that carrier. Now, they're working on a single carrier, and the design is to make them experts on that carrier's systems. Q Okay. We had, you know, mention yesterday that the inspector had no choice over where the geographic inspectors would be. Who makes those decisions? A Well, that's a misperception that was generated again by the way we put the system in place initially. We needed to use the inspectors that had been doing the work before where they were. We acknowledged from the beginning that that was not the most efficient choice, but it was necessary at the time. We acknowledged from the beginning that it would change over time. So, when these people leave their jobs for whatever reason, they're not replaced where they were. Instead, the Certificate Management Unit and that unit alone has the authority to request where the new position will be, not who the person will be but where it will be, and the design is to get it in the most efficient place to do the work. Q It seems like it could take quite some time if you're waiting for a period to resign or retire for that to happen. A In some cases, that may be. One of the unfortunate problems we're dealing with is that with the geographic inspectors, there's a high rate of turnover, too high a rate of turnover. So, some -- the -- as I said, when we make an investment in the inspector, and it's a new system, it will be quite an investment. So, we need actually to focus on reducing the rate of turnover. What we obviously haven't done a good job on is communicating that CMUs do have control over where those positions are, and perhaps this hearing will help correct that to some extent. Q But I guess, you know, how much control do you have when you're waiting for somebody to retire before you can make the switch? That seems -- A To some extent, it's a problem that can and will be addressed. We have some union concerns there that are legitimate in terms of fairness to employees, and those are problems, I think, that we have to work. But, once again, where a person is doesn't limit where that person can do the work. Q Okay. And I guess also from the testimony yesterday, it indicated that he believed he's about 40 percent under-staffed, and that's a huge number in my opinion. A It is, and I'm sure quite valid from -- from where Corky sits, and maybe -- maybe quite valid period. Every -- every inspector you ask will have, including myself, a different opinion about what the staffing needs are. What the ATOS model does is help us become much more explicit about these needs, much more -- make the numbers much more quantifiable, and I'm looking forward actually to -- as the system is applied, how -- how that piece of it works well for us. Q Okay. Thank you. CHAIRMAN HALL: Mr. Clark? BY MR. CLARK: Q There were three studies, one on SAT and one by Allied Pilots and American, that came up with 89 recommendations. How is that type of information or recommendations or resolutions of those recommendations transferred to other airlines or to other CMTs or CMUs? A It's -- it's an area of challenge, I think. Excuse me. It's an area of challenge not only for FAA but for industry and everyone else, how to share best practices and problems. It's one that we're working on. I frankly believe this model is -- is a good model for sharing the data. As we've talked about it for the most part throughout this hearing, we've talked about ATOS as it applies to American Airlines or to another individual carrier, but if you look at again that schematic, it is a model that -- that -- that integrates all those activities and permits data to flow into again that policy module in a systematic way. So, I think ultimately, we have now a structure and an infrastructure in place that can better utilize data, such as you're talking about, and spread it than we've had in the past. But there are a lot of problems involved from proprietary issues to litigation issues to other things that impinge on the industry and the FAA and all of us involved in dealing with this issue and how well we can do it. So, we need to keep working on it. Q Okay. Is -- is there something -- we hear about ATOS under development, and is there something functionally in place now that can distribute the lessons learned or at least those 17 recommendations that have been adopted by an agreement with American and the FAA? Have those been distributed or is there -- the model is a little nebulous. Is there -- what is the -- what shows up on somebody's desk that says this information has been transferred? How does that get done? A The -- well, again, of those recommendations -- and I'm not familiar with each of them individually, some of them may be proprietary, and ceratinly that would be a concern in terms of sharing them. Otherwise the ATOS activity at this point only would provide an opportunity for those recommendations to be surfaced through the ATOS CMO which is the Headquarters group that provides support to this activity, and -- and through that group disseminated to other ATOS carriers. Part of the -- the analysis function and the implementation function is to identify best practices so that they can be shared. Again, I think we have rudimentary elements in place at this time that need further development. Q Okay. Thank you. CHAIRMAN HALL: Well, David, I have had the honor and privilege of being the chairman of this agency since 1994 and being on this Board since the Fall of '93. So, most of your testimony I have seen this process take place, and, of course, I have my own observations as to why this -- this took place, but clearly after ValuJet, there was an attention on some of the new entrant carriers, feeling that the major airlines basically had their act together, and we needed to do a better job, and this system was designed. Hopefully it's going to do a better job on all carriers, but I think what we've seen in the discussion today is that, you know, you have a very experienced workforce that's been working with the system that by and large has done very well historically, as you pointed out in your opening comments, and that now you're putting a new system in, and I'm still concerned about the gap that's taking place and the resistance that we obviously are seeing, and I'm sure you're aware of, which is just human nature. All of us don't like change. It's difficult, and if you've been doing something, particularly if it's something you think is effective, somebody comes along and wants you to do it differently, you -- you resist, and there, we've got to have some training and the system. The only thing that's been sort of discussed, and I think you're aware as well as I am or much, much better than I am, are the problems and the challenges. You know, I observed two things that I want to be sure of. One, I see all these people in Washington go up to Congress and say, you know, less is more, and many times, less is less, and when it comes to safety, less means -- can mean the loss of lives and serious injuries to individuals that are lying on the Federal Government to provide safety oversight of the aviation system. But the main concern I have is -- gets to what Mr. Clark said. I guess we got a free audit here out of the airline union, which we appreciate, and American Airlines, looking at the system with some recommendations after an accident, and, you know, we live in a great country. We have very -- these fatal accidents, such as this, are rare in the system. We killed over 400,000 people on our highways last year, and a lot of the -- a lot of the good -- last year -- last decade. I apologize. 40,000 a year over the last 10 years. It's the Number 1 killer of kids. One of my interests has been to try to get a lot of the technology and practices in aviation on our highways. But one of the reasons aviation is so safe is because of the detail we go into safety and the culture we have as demonstrated by this hearing and everyone's participation. But you need to see if you can't figure out how these 88 items get transferred within the system. Just to say it's not -- I understand your answer, short answer is there's no way to do it right now, but I would suggest if you -- if you can go back and do that, and I'll try to get -- I'd like to ask Mr. Haueter -- I'd like to send a letter to Ms. Garvey on the basis of this hearing and ask her to -- to see if you all can't make that a priority because there are -- it's important information. There are a number of factors that we're going to be looking at, have looked at and will look at in this hearing that are important to the safety of each and every flight that takes off every flight, and -- and -- and obviously an oversight, a close look at the safety system of American, we went through. There was another major carrier that had five accidents in the earlier part of my service on the Board, and they had to do a very serious internal look at themselves and learned a great deal and improved greatly, and that information needs to be transferred through the -- through the system. Now, you have been an excellent witness, and I compliment on your service to the Government and the military. You don't have quite as much as what, 57 years as one of our previous witnesses has, but it's obviously excellent service, and I think service to our nation should be acknowledged and recognized, and I appreciate that. I'd like to present you an opportunity as I have with all the witnesses, and I think Mr. Tew now is not going to be stepping on the Chairman's question here, and I appreciate that. Do you have any additional suggestions or thoughts for the Board in light of this accident or your experience that might be helpful to aviation safety to prevent an event such as this from recurring? THE WITNESS: Thank you, Mr. Chairman. The only thing that I would like to leave, I guess, as a thought is that in my opinion, this move to a new air transportation oversight system from a regulator's point of view is very important, and it's very easy to derail a process like this. One of the things that Sandia did for us when we were initially designing this system and considering designing it was a study of why we had been so unsuccessful at making changes with the old system, and they looked at our entire environment, the whole environment that includes not only the regulator but the industry and various oversight groups, and determined that -- that from their observation of that phenomena, that that environment's immutable. It really is very change-resistant because the overall interplay of all the players is that the static status is nice, and clearly I think we're dealing with that internally in making these changes. It's very difficult. We admittedly have some rough edges. We're working to smooth those out, but it's going to take a great deal of fortitude, I think, to -- to see this through, and I -- I hope that we have the opportunity and support to do that because in my personal opinion, this is the one thing that as a regulator, if we don't do, we will never progress, and while I would not say that the system we have in place is -- is everything that it should be because it is one that is designed for future generations and improvement, over time, I think it will be again the leading edge worldclass system, and I'd like to continue participating in that development. So, thank you for that opportunity. CHAIRMAN HALL: Okay. Well, thank you very much, and we -- as I said, the Board's on record in favor of ATOS, on record in the favor of expansion of ATOS, and I will individually try, and the Board will, to get you the necessary resources to implement the program. I understand you have to -- but it's clear to me when you have a closed loop, and you don't have the resources for three boxes in the loop, your system is not going to perform as -- as designed. Well, we've had -- I appreciate this first witness. We're going to take a break now before our next witness. I would like to make a very important announcement. The City of Little Rock and its Mayor, Jim Daley, have provided coffee outside the ballroom compliments of the citizens of this great city. The Mayor properly observed yesterday that the Federal Government with all its wonderful rules and regulations is unable to provide funding for coffee for you all, and, so, the City of Little Rock has rushed in to the -- to the gap, and there's coffee outside the door compliments of the City of Little Rock. I hope that will in no way bias anyone in their opinions or later proceedings in this hearing, but the coffee is there. So, we'll stand in recess -- MR. STREETER: Mr. Chairman? CHAIRMAN HALL: -- until -- MR. STREETER: Mr. Chairman? CHAIRMAN HALL: For 15 minutes. MR. STREETER: Mr. Chairman? CHAIRMAN HALL: Yes? MR. STREETER: Over here, sir. CHAIRMAN HALL: Yes, sir? MR. STREETER: I would ask that Mr. Gilliom be released as a witness so he can return to his duties in Western Pacific Region before our weather arrives. CHAIRMAN HALL: Oh, I forgot. The National Weather Service is also providing five to 10 inches of snow later today, we are told. We're told. Now, you are excused to return, Mr. Gilliom. THE WITNESS: Thank you, Mr. Chairman. (Whereupon, the witness was excused.) (Whereupon, a recess was taken.) CHAIRMAN HALL: We will reconvene this public hearing of the National Transportation Safety Board, and I'll ask Mr. Berman if he'll call our next witness. MR. BERMAN: I call Mr. Robert Baker. Whereupon, ROBERT BAKER having been first duly affirmed, was called as a witness herein and was examined and testified as follows: INTERVIEW BY BOARD OF INQUIRY BY MR. BERMAN: Q Good morning. A Good morning. Q Would you please state your full name and business address for the record? A My name is Robert W. Baker. I office at American Airlines Headquarters, Dallas-Fort Worth, Texas. Q And by whom are you employed? A American Airlines. Q What's your present position, and then, please, your position at the time of the accident? A I am currently Vice Chairman of AMR Corporation, the owner of American Airlines and American. At the time of the accident, I was Executive Vice President of Operations. Q Of American Airlines? A Of American Airlines. Q Not AMR. Okay. And how long have you held these two positions? A I've been in the Vice Chairman's role as a part of the Office of the Chairman for approximately three weeks. I was in charge of American's Airline Operations under various titles since 1985. Q Thinking back to the Executive Vice President and similar positions at the airline, could you please summarize your duties and responsibilities in those positions? A I'd be glad to, and in a few moments, I'll show you an organizational chart that will give you more detail. Operations within American Airlines encompasses all of those functions and responsibilities having to do with the actual operation of the aircraft and our airport functions. That would include the Flight Department, the hiring, training and day-to-day operation of the flying part of the company, Maintenance and Engineering for our fleet of aircraft, some planning functions in the crew area, dependability monitoring, the Operations Center, including the dispatch function, crew schedule, and then other support roles, such as security, corporate real estate, safety, and our cargo operations. Q And also please describe your responsibilities as Vice Chairman of AMR. A That is a relatively new concept at American Airlines. It is an attempt to -- to create a little different structure as we go forward. The chairman of the corporation, Mr. Don Carty, and I will co-manage the airline. I will be principally responsible for continued oversight of the operating side of the company. He will be more involved in the finance, marketing and external affairs. Q Thank you. And could you summarize your education and training that qualified you for these positions? A I received a Bachelor's in Economics from Trinity College in Hartford, Connecticut, in 1966, and an MBA from the Wharton School of the University of Pennsylvania in 1968. Q And do you hold FAA airman certificates? A I do not. Q Thank you very much, sir. A Thank you. INTERVIEW BY TECHNICAL PANEL BY DR. BYRNE: Q Good morning, Mr. Baker. A Good morning. Just one follow-up question as far as your background and current duties and responsibilities. Do you have the opportunity to ever observe flight operations from the flight deck during your daily activities? A Yes, I do. I travel two to three days a week to carry out various assignments. In that context, I have a personal policy of spending approximately 50 percent of those travel days on the jumpseat in our aircraft. Q Thank you. Before -- what we'd like to talk about today are your internal oversight programs in terms of flight safety, and also your response to recent accidents and incidents at the company. But before we do so, I know you have a presentation prepared, and would you please start that presentation by going through the management structure and hierarchy as it relates to flight operations and flight safety at American Airlines? A Thank you very much. I very much appreciate the opportunity to participate in this process. Let me -- before showing you some organizational charts, which I think is the most efficient way to describe the way we are organized and some changes that we have recently made and are evolving toward. I think it's appropriate that I begin by trying to describe the culture within the company and the way in which the management team at the senior level attempts to run the company. First of all, I think it is fair to say, and our reputation is widely known, as a company that is highly analytical and highly data rich. We -- we very much like to deal with problems in an analytical and a data-driven sense. We have lots of feeds of data that we believe tell us where our successes and failures lie. Secondly, we cherish very much the notion of innovation, whether it be in the marketing area or the operating area. We are prepared to spend the resources and human capital to create constant innovation in an attempt to find better ways to do business and to serve our customers. The third point that is important is that we have the corporation under constant evaluation. Throughout the staff groups and as a group of senior managers, we sit once a week and evaluate the operation of the airline from almost every conceivable direction. We are not the least bit reluctant to identify our failures and to then use that same analytical approach to designing the solutions and fixes. We are very hard on ourselves on a routine basis. We believe very much in constant and continuous change as a means of keeping up with the business that is operating in an environment that changes rapidly throughout our network. One item that I hope to demonstrate to you this morning is something a bit new for us, and that is we are moving very quickly toward more of a third party evaluation. We think we're pretty tough on ourselves. We do a lot of self-evaluation. We think it appropriate, particularly as the company gets larger, that we bring to bear the resources of third parties to help us evaluate our performance. One thing in the safety area that -- that guides us throughout the operating departments is that we never feel we achieve perfect safety. That is our goal, but we never get there, and if we ever believe that we've gotten there, we probably have a very serious problem, and that's what keeps us moving and bringing change to -- to the corporation. Let me now show you -- and I apologize if -- if these are hard to read in this large room. I'll show you a series of organizational charts that represents American Airlines in the past and at the point of the accident, and then I will highlight some changes that we have and are in the process of implementing at the corporation. Let me say at the beginning that this appears to be a stacked chart, but in reality, all of the people on this chart report directly to the chairman, president and CEO, Mr. Don Carty. If you were to draw this on a piece of paper, it would be a single horizontal string of positions reporting to -- to the chairman's office. These are all of the typical functions of -- of an airline and a big company. Marketing, Government Affairs, General Counsel, and then some of our subsidiary efforts and in the computer business, American Eagle, our regional airline subsidiary, and here is -- is the Operations side of the company, showing myself as -- as being in that role. This basic organization has been in place at American for many years. Dropping now to -- to the Operations organization, specifically it is, as I indicated, all-encompassing to the -- to the airline operations, but more specifically, starting at the left, we have Maintenance and Engineering responsible for the maintenance of -- of our aircraft. That is done both in line maintenance stations and in our two major maintenance bases, one at Tulsa, Oklahoma, and one at Alliance Airport in Fort Worth, Texas. We employ approximately 10,000 people in this Maintenance and Engineering function. The second function are those Planning and Performance Evaluation functions that deal primarily with our crew resources and also manages our dispatch and crew schedule function. The Vice President of Flight and Chief Pilot is responsible for the hiring, training and day-to-day operation of our -- of our flying. The President of Cargo is a self-contained separate company in charge of all aspects of the cargo business, which, to put it in perspective, is roughly a $1 billion business for American Airlines. We have on the far right shown a VP of the Reno Air Integration. In 1999, we purchased the assets of Reno Air and integrated them into American. This function was principally the place in which we -- we focused that integration, coordinated it across all the company's departments. Then Corporate Real Estate on the bottom left, responsible for the real estate and building of our facilities, purchasing, responsible for everything that we buy, including aircraft parts. Managing Director of Security, both internal and external security, and, finally and importantly, in the far right, the Managing Director of Safety. This function includes the safety processes and the environmental oversight at American Airlines and the subsidiary companies. Q Mr. Baker, just a quick question. A Yes. Q Pardon me. With those nine operational units underneath you, would you describe for us your relative amount of time in oversight or your -- where you allocate your responsibilities? Is it equally across those nine operational units or bias towards one versus another? A Well, I -- first of all, it is a plate full. I can assure you every one, I think, in the airline business works extraordinarily long hours. It is to some degree a reactive environment. Clearly, the -- the areas that get the majority of my attention are maintenance and flight. Those are the two functions that probably between them get 60 percent of my attention. Q Thank you. A This chart now reflects the changes that we have most recently made. The important pieces are that I have been elevated to the Office of the Chairman. Mr. Carty and I will share the management of the company, and as you -- as you will see in a minute, the important operations functions report directly to the Office of the Chairman, and I will retain primary operational responsibility. Again, these are -- the stacking of these boxes are not intended to imply that the upper row is more important functionally than the bottom. They would all be horizontally. I have been replaced in the Executive VP of Operations role, and a new function at American at the very senior level of the corporation. We have now taken safety, security and our environmental assets and put them together in one department reporting directly to the Office of the Chairman. We think that is an important step that we have been considering for -- for a number of months, and we have now implemented that organization, and let me show you what that organization looks like. This is the top line, if you will, of this new organization. On the far left, the safety functions, then security, environmental, and those are the three functions that existed but not under a single leadership prior to this. The two new areas are on the right side of the chart, labeled Operational Audits and Compliance. In these areas, we intend to substantially pick up, if you will, our auditing processes by third parties who are not responsible for the operation of particular operational functions throughout the company. All in all, I expect as we -- we populate this organization, we will be adding on the order of 20 to 25 additional people to these functions. In going now one step below that, this shows you that far left box, the safety function. Here, we have taken each of the areas that are in the operation. We have assigned safety responsibility to a member of the safety staff, also focusing on those interfaces to the FAA in terms of airworthiness on the far right, and the flying operation on -- on the far left. So, this is a highly- integrated but extensive attempt to provide more safety emphasis at the company. One additional change that I believe is important for this context is in our Flight Department itself. These are the basic functions of the Vice President and Chief Pilot in Flight. The change we have made is an important one on the left side of the chart, in which we have separated very distinctly from an organizational point of view the function of doing the training through the ground school and -- and simulator to the Flight Standards Group or the Checking Group to ensure that the training has been done according to the syllabus and done in a standardized way. In the past, those functions have been combined within the training organization. We think separating them will give us an improved level of objectivity and enable us to -- to enhance our level of standardization. That, I believe, is an important change for our company, and that about does it for the organizational changes. Q Thank you, Mr. Baker. A couple follow-up questions. What was the fundamental reason for these organizational changes at the high level? A Well, at the very highest level, the creation of the Office of the Chairman was aimed at a number of -- of the objectives. The first objective is to provide some additional focus in some areas, one of those being the combined safety, security and environmental. Clearly, it was time to -- to provide more resources there and to focus them at the top of the corporation. Secondly, we have an obligation to our shareholders to prepare for successorship, and if you could understand all of the details of -- of the people we put into the various jobs, what we are essentially doing is preparing for the successorship to Mr. Carty and I when we retire down the road. So, this was an opportunity to do some training while we're still here and to bring the younger men and women forward in the organization. So, with those two thoughts in mind, that's how the organization evolved. Q Were these changes proposed or considered before the Little Rock accident? A Yes, they were. Particularly in the safety area, they were under consideration and evaluation, but we had not acted on them. Q And why hadn't you acted on them? A Well, we had not reached a consensus that any particular change was -- was appropriate or meaningful. We've spent a good deal more time on that and reached that conclusion and have now made those changes. Q And when you say "we", could you give me a picture of who you're talking about? Who are making these decisions? A Well, the decisions are made essentially by Mr. Carty and I with the concurrence of our Board of Directors, since our board is required to approve all officerships in the corporation. Q Let me turn to the Flight Safety aspect of the management structure, and this touches more -- gets away from box diagrams and into philosophy. Does American Airlines have a formal written statement outlining corporate safety policies? A Well, I'm not sure we have a statement that -- that outlines them for the entire corporation, other than safety being the Number 1 priority of all of our responsibilities, and I believe that is continuously stated in public documents for both our shareholders and our employees. As you get down into individual departments and through our internal audit function, we do have statements of that philosophy. Q And as far as the Department of Flight Safety is concerned, at the time of the accident, what was its scope or reach? What was its mandate? A Let me go back and -- and also indicate that at the time of the accident, the Flight Safety responsibility was in the Flight Department itself. Under this organizational change I've just reviewed, the direct reporting relationship will be to the Vice President of Safety, Security and Environmental with a dotted line to the VP of Flight and Chief Pilot. So, we are making that -- that change as well. The mandate of -- of the Flight Safety Department was (1) to investigate all events and incidents for what we could learn. That is a very important ingredient of our management style, to learn from our shortcomings and hopefully translate them into changes. Secondly, to make suggestions and input to flight management and -- and to myself as to things that they observed needed to be changed or -- or looked at. Q Why does the dotted -- why does the straight line reporting stop at the Vice President level and not continue on to the - - or bypass the Vice President and go directly to the Chairman level? A Well, we -- we felt, and -- and with the help of some outside folks who came in and looked with us at the way we were organized, which I'll review in a few minutes, that a third party relationship of -- of the safety -- the Flight Safety function reporting to the VP of Safety was a better approach than to the Vice President responsible for producing the product, if you will. So, again it is an attempt to separate functions and responsibilities of providing the product from assessing its -- its quality and its compliance, and, so, we thought separating those would be an appropriate step to take. Q What criteria do you use to evaluate the effectiveness of the Flight Safety Department? A Well, we have a number of management tools that we use for all of our senior people in evaluating their performance and in running a particular function. So, we will utilize those tools. We use such things as 360 reviews. Obviously we -- we will look at the output, if you will, of -- of that function. Are they probing deeply enough? Are they coming up with issues that we need to deal with? We want to get to an aggressive approach of problem finding and solving, less reactive types of things. So, those are the -- the -- the -- the points that we will be looking for as this organization matures and develops over the next year or so. Q And who is doing this review or oversight of the Flight Safety Program? A Well, the Vice President of Safety, Environmental and Security and myself will be personally involved in -- in that organization's maturing and its performance. Q Who had direct oversight of that program at the time of the accident? A At the time of the accident, our Safety and Environmental functions were a department in my organization. That department was managed by Mr. Tommy McFall, who reported directly to me. Q And does the Flight Safety Department or did the Flight Safety Department at the time of the accident have a formal means to communicate safety-related information to pilots and other operational personnel at the company? A Oh, yes. I -- we -- we have a premium, if you will, on communications. We -- we emphasize continuously the importance of -- of communication, horizontally and vertically. I think we have a culture that -- that cherishes the flow of information. As I said, we are data-driven. So, I don't think we -- we have a problem of communicating issues, findings and problems. Q Mr. Baker, before we proceed in your presentation, I'd like to, with the Chairman's concurrence, pass the microphone to Mr. Feith for further questions. CHAIRMAN HALL: Okay. MR. FEITH: We'll keep going. BY DR. BYRNE: Q I guess no further questions. So, if we could at this point transition from the management structure and hierarchy to specific oversight programs and responsibilities that the company has, and continue your presentation at that point. A Thank you very much. As I indicated in my introductory remarks, we are a company and a group of -- of managers that believe in getting all of the data we can possibly get. In fact, there is no such thing as too much data. We value input from all sources, and we want to use that data to identify problems and hopefully help us articulate solutions that pinpoint the problems very carefully. We do like to learn from events. There is no point in having things happen unless you learn from those events and use that knowledge to try to -- to prevent their reoccurrence. I will attempt in this to not use jargon. So, I'll define some of these acronyms as we go forward. We have a number of programs that help us provide this data and this input to our management process. The first is a program called AASAP, which stands for the American Airlines Safety Action Program. This is a program conceived at American Airlines in 1992 by one of our captains. We worked with it internally in a small group, and in 1994, the program was actually put into place in the Flight Department at American Airlines, and I will go into more detail on this program in just a moment. A companion program is FOQA, which stands for Flight Operations Quality Assessment, and that is another form of data- gathering, again looking for problems, looking for trends, that point us at difficulties that we can then correct and deal with. Next is a system that we call FlightNet Reporting. This is a more traditional feedback system from our line pilots. In essence, it's often called a debrief in aviation terms. We get approximately 18 to 21,000 debriefs a year from our cockpit crewmen. There are certain mandatory debriefs for some 21 events that happen on the airline. We want to know about these events, so that we can go understand them, see what drove these events to happen. A couple of examples. We want to know about diversions. We want to know about engine shutdowns. We want to know about bird strikes, and there's some 21 of those mandatory things that happen in the airline every day that must be debriefed through this FlightNet reporting system. Again, though, this is an opportunity for our line crews to tell us of problems and things that bother them as they fly the airline. We have, and I spoke briefly about our internal and external audit program. We have had an industry-leading internal audit program for some 10 years. Every six months, we look at the output of that internal audit process. It is a true compliance to the FARs' approach, and we have this internal audit process operating in all of the operating departments of the company, and, of course, in the maintenance area, we have a very extensive inspection and quality assurance organization of several hundred people constantly looking and back- checking our maintenance organization. I will review with you some of our external audit processes in just a minute. Another important data generator, if you will, is the Air Carrier Voluntary Disclosure Program. This is an opportunity under the FARs for an air carrier or its personnel to step forward and disclose to the FAA a discrepancy or an operation that -- that may in fact be a violation, so that we can get all of the facts on the table and review exactly what happened. Next is the American Airlines and the Allied Pilots Association safety review process, and I will talk further about that in a minute, but this is an organized interface between the pilot union and the company again to -- to pull up to a level of action any input that we can get from our line pilots. Finally, the ATOS Program. We spent a lot of time on that this morning, and I won't go into that any further. We are very, very hopeful that ATOS will mature very quickly. We look forward to mining that data. We think it's very important, and we're very anxious to get on with that. Q Mr. Baker, were all six of those programs in place at the time of the accident? A Yes, with one exception, and I'll touch on that. The safety review process. Q Thank you. A Now, specifically to -- to AASAP. As I indicated, this is a program that was developed at American. It started in the Flight Department. It now involves not only our pilots but our mechanics and dispatchers as well. It is a true accident prevention program. It operates on the basis of the following principle, and that is that the best safety information is the information that we don't know, and, so, any way that we can create to bring information that otherwise would not be known to those who can effect change, to bring that information to the surface is very, very valuable, and that's what AASAP's principle concept is all about. Since the inception of the program, we have had over 20,000 reports submitted to AASAP. These are all in a database which is continuously updated. This database provides us trending information on particular subjects. It feeds, if you will, our training syllabus and our other functions of managing our pilots. Every six weeks, we take the data from the reports that have come in, and we circulate those reports and descriptors of incidents to all of our line pilots because we believe that line pilots, too, can learn from the other -- from the incidents and -- and events that others have been involved in. It is a self-reporting program. It is intended, as I said, to identify and correct safety concerns. Once a report is submitted and accepted into the program, and not every report that is offered is accepted -- if a pilot - - I'll use a pilot example. If a pilot has intentionally violated the rules and the Federal Air Regulations, that report is not acceptable for consideration under the AASAP Program. Secondly, if the FAA is aware of the event and the incident from other known and public sources, an AASAP report is -- is -- is not appropriate and is not accepted. We have found in the five or so years of this program that we have been able to -- to get a handle on improvements in all operational areas. We have characterized the -- the database and tried to create the top 10 event categories for our own focus in our training and in our surveillance of the airline. Operational distractions are Number 1. Rushing to comply, Number 1. Automation dependency, Number 3, and situational awareness, Number 4, just to give you a flavor of the types of things that have come out of our database work and our trending. Let me now give you some specific examples of some improvements that have come directly out of the input of the AASAP Program. First, we've been able to make operating manual changes across all of our fleets. One of the problems we dealt with early on in AASAP was to try to understand why our pilots, particularly flying the Super 80 aircraft, were having difficulty maintaining the assigned altitude given to them by Air Traffic Control. We found two -- we made two changes. First, we found a mechanical problem with the altitude knob on the -- on the glare shield that moves the assigned altitude into a window, if you will, for the crew to -- to observe, and, secondly, we adopted a -- a new approach that we call point and shoot, and that is when the crew is given a new altitude assignment, the pilot not flying will put it in the box. The pilot flying will physically point at it and affirm that he understands that there has been a change in the assigned altitude. We have put that across all of our -- all of our fleets. We have likewise adopted that point and shoot approach to changes in our FMC computers on board the airplane, which is crucial that both crew members understand the changes have been made in the FMC input. The second example is a change in our policy on position hold. We had an event in which one of our aircraft was cleared into position hold, and another carrier's aircraft flew very closely over the top of the airplane. We believe after looking at this in some detail, because of the report that we got in AASAP, that we were being held too long in a position hold and not given clearance. So, we now forbid our aircraft to take a lengthy assignment in the position hold at our airports. We think that's a prudent thing to do. The third example is the situation in which we have an engine failure. At slow speed, the rudder will be confined. There is a technique to kick the opposite rudder to free that rudder. This is a manual change that has come directly out of AASAP. We've made policy changes for both pilots, dispatchers and mechanics, including some additional work and guidance on thunderstorm avoidance, decision-making and crosswind approaches. In the area of training procedures, we've done a lot more work because we now understand more clearly the issues of distractions and so forth, human factors and safety. We've used the AASAP incidents as feed to our loft design, so that we actually fly things that our pilots experience, and we use the items that we learn through AASAP as -- as briefing items. In the air traffic area, we feed these -- these topics to -- to ATC by way of the FAA member of the evaluation committee. We were able to deal with a runway incursion problem in El Paso that was observed by several of our crews. That brought about some additional painting and identification and the threshold of two convergent runways. We have identified problems at particular airports on the arrivals in which our enhanced GPWS suggests that we are being brought in too low for the approach, giving an alarm. This was the case in an airport at Middletown. Finally, we identified a very dangerous situation that we worked with Air Traffic Service to modify the air space in an area that was being used by parachute jumpers, which is a very incompatible use of air space. But we would only know about those almost-kinds of situations by having the ability of our crew members to step forward, and, finally, we do on occasion determine and detect that additional proficiency for individual crew members is an appropriate strategy, and we take action on that basis. I indicated that I would talk more about the flight safety review process. This is the interface of American Airlines and the Allied Pilots Association. There are two committees that are very important to -- to providing data and helping us manage our operation; that is, the Training Committee and the Safety Committee. These have been very -- historically very strong relationships with the company, with open doors. I don't think it's any secret that we have had some economic differences with our pilots union. Unfortunately, this drove, if you will, a closing of some of those doors. The relationship deteriorated. The flow of information was reduced. We were not getting the benefit of -- of the APA Safety Committee and Training Committee's input to our process. We believe that that is an essential part going forward. As the senior leader in the Operating side of the airline, I am committed to restoring that relationship to a very positive role. I believe that the union's leadership and board are as well. They have assured me they are. We are rebuilding that consensus, and I expect that as we go forward, this is going to be a tremendously-important and positive process for American Airlines and for our crew members. So, I think we are -- we're going to put the past behind us and go forward and mine that data very -- very well. That is all that I have to offer on our systems, Mr. Byrne. Q I'd like to follow up on two general areas. One is on the reporting systems, the AASAP Program, and the second area is the internal audits. As far as the AASAP Program is concerned, how many personnel from American specifically are assigned to the AASAP Program? A One part of my description probably didn't -- didn't explain very well how it works. When a -- when an AASAP report is -- is put forward, there -- there is a three-party meeting that takes place every week. It involves the FAA, our CMO office, the Allied Pilots Association, and American Airlines, and the three parties look at the submitted report. They decide who are the appropriate people at APA or FAA or the company to go and investigate the details or to take action to -- to change or correct things. So, that meeting takes place, and those are three individuals that probably spend on the order of half a day a week. In addition, within the company, we have about two and a half people that are full time kind of running the program continuously. Q In terms of the volume of reports that come in, Captain Griffith testified in July that approximately 3,500 reports come in specific to the AASAP program on an annual basis, is that correct? A That's correct. Q And for the FlightNet System, which encompasses, I guess, both AASAP and the OF25 Program, he testified between about 18,000 and 21,000 reports come in annually? A That's correct. Q And could you just back up and describe the OF25 Program? A The OF25 Program is what I described as the FlightNet Reporting System. It is a more traditional debrief system for our flight crews to use. As you indicated, 18 to 20,000 per year. It encompasses or solicits general comments about the operation, but it also is a routine reporting mechanism to report to the company some 21 specific events that may or may not occur out in the day-to-day operation. Q Are flight diversions part of those OF25 criteria? A Yes, they are. Q What happens to OF25 reports that are in that category of flight diversions? A We look at those reports for the purpose of understanding why the diversion took place. Was it handled correctly? It is much more an informational thing than an action item, but we want to know what the frequency of our diversions are. For instance, if we were to see a large number of diversions to one small city, we'd worry a lot about that because of the overload on that small city and their ability to do a good job, both for the crew and -- and our passengers. So, it's really kind of a visibility kind of a reporting system, so that we know what's going on in the system, and we trend it. Q And my understanding is the AASAP Program's synonymous with the OF25 Program when a captain submits a report of diversion. Is that -- how is that information held? A That's held in the OF25 system, I believe under the cockpit crew member's name. Q How long is it held in the OF25 system? A I do not know. Q Are captains tracked for diversions? A Absolutely not. Q Moving on -- moving back to the staffing and how the AASAP reports are evaluated, with 3,500 reports coming in, and recently I read in a newspaper article, approximately a 150 reports are evaluated each week by the ERT, Event Review Team, how do you ensure that specific nuggets of information or safety-critical items are not being missed? A I -- I think the number of unduplicated reports suggests a somewhat lower number. It's not unusual, and I'll just use one example of, let's say, an altitude excedance, that we will get an AASAP report from both the captain and the co-pilot. So, within the 3,500, we really have two about the same incident. A lot of the reports fall into known categories of altitude excedances, misreads back leading to -- to difficulties, and we can categorize those fairly quickly. The unique and -- and different ones come out of that reporting flow pretty -- pretty quickly and are dealt with by the staffs that are assigned those reports to come back to the ERT with feedback on what happened and what we're going to do about it. So, while we have just the three kind of clearinghousing all of those reports, there's a large staff that gets the reports all over the company, including our Airport Managers, who have an obligation to come back and -- and tell us of the investigation and the results. Q And you mentioned the top 10 issues that are coming out of the AASAP Program. Would you elaborate on what is meant by "rushing to comply"? A Rushing to comply, as I understand the concept since I'm not a pilot, I'm a little less able to articulate that -- that concept very well perhaps, but it is the -- the concept that -- that a crew member or a crew is asked to do something, and without thinking it all the way through will just simply execute the request. It's termed "rushing to comply". Q And who's asking the crews to do something? A Well, my understanding is it's typically a combination of Air Traffic Control perhaps giving them instructions or (2) just the normal course of operating the flight gets into this rush to comply process. Q And you mentioned one of the products of the AASAP Program was new thunderstorm guidance procedures or new thunderstorm guidance. When did that come into place? A That is in -- in work at the present time. Q So, it was not in place at the time of the accident? A I don't believe so. That -- that isn't to say we didn't have any guidance, but we're redoing that guidance. Q I'd like to turn now to the Internal Audit Program. In general terms, not specifically related to any accidents or incidents, how frequently are these -- is this an on-going evaluation program? Are they time based or event driven? A This is a continuous internal audit process. It is a true audit in the sense that what we are looking at is -- is compliance to the FARs and our operating specification as approved by the FAA. We are looking very narrowly at are we doing what we said we were going to do in our ops spec, and are we within compliance of all the FARs as -- as we know them? That process cycles through every operating department every six months. We then pull the results of that together. We sit down at my level and review each department's performance. We then go to our Certificate Management Office at the FAA, and we share our findings with them as to what we have found and what we're going to do about it. So, it's continuous in six-month increments. Q And that's each department, each operational unit, Flight Operations? Training and Standards is conducting their own audits? A Dispatch, Weather, Flight Training. Maintenance has a huge part of that process. Our Flight Service people that manage our flight attendants. We've tried to get that into as many of the areas of the company that have an operating and safety impact. Q Who is the point person within each department that's conducting these audits or responsible -- A We have a coordinator assigned in each department to make sure that the -- the analysis and the audit is -- is conducted, conducted properly, and then assembled into a -- into a report. Q At what level is that coordinator? A Usually a manager or managing director, one step below the vice president level. Q The scope of these audits, how often does it go beyond regulatory compliance? A These particular audits are not intended, and I don't believe they -- they go beyond regulatory compliance or compliance to our operations spec. We use other processes to get at some of those issues. Q And the other processes are? A External audits, general management reviews and performance reviews. Q Who conducts the external audits, other than the FAA? A Various folks, and in a minute, I -- I will kind of give you some detail on that, if I might. Q Okay. DR. BYRNE: I guess at this point, I'd like to pass the microphone to Mr. Feith. BY MR. FEITH: Q Well, it looks like it's still morning. Good morning, Mr. Baker. A Good morning. Q You had talked about several things, and let me just back up to one point that you had just made regarding the -- I believe it was the guidance about weather that you had just spoke of, and that it was being revamped, and you said that that had come from the OF25 and AASAP input? A Yes. Q Okay. Can you be specific as to what kind of reports came from the AASAP Program or the OF25 Program that necessitated that guidance change? A I don't think I'm qualified at this point to give you a detailed understanding of specifically what drove it, but a collection of -- of those input prompted the ERT to suggest to our Flight Management that -- that some modification was appropriate. Q As far as that same program, the OF25 and the AASAP Program, do you have any understanding about any other anomalies or deficiencies in the organization with regard to crew pairing? We had looked at the crew pairing issue with having the chief pilot with a new first officer. Have you seen any kind of crew pairing issues come through the AASAP Program? A No, I have not. Q How about with any other training issues, such as FMC training, or any particular training with the MD-80 aircraft? A Not with the MD-80, but as I indicated, one of our top 10 items that have come out of the trended data in AASAP is what we call "automation dependence". That relates more to the FMC- driven aircraft, 757 and above. But I have -- I'm not aware personally of any direct input relative to the Super 80. Q And then using the OF25 and the AASAP Program, you're using that system to monitor trends and that kind of thing, to identify deficiencies or anomalies that you can take corrective action with. Given the fact that the Chairman had made a statement earlier regarding American Airlines, and during his tenure, American has had several accidents and some serious incidents, have you seen anything that -- that would lead to a systemic problem from your perspective through that program to identify the specific target areas that needed to be corrected? A Well, I think the -- I think it's fair to say that the top 10 trended issues coming out of AASAP in particular represent the most rewarding areas for us to concentrate on to avoid a systemic problem. That's where our resource ought to be -- be focused and changes made as we understand that there are better ways to do things. I think it's very, very helpful to us to be able to get out of that data those kinds of trended areas, and we work those very hard, and we'll continue to. Q As far as -- can you give me a characterization of your relationship with the FAA? We -- we heard the FAA's perspective yesterday, and the fact that they had some manpower issues and that kind of thing. But could you just briefly describe your relationship with the FAA? A Well, the FAA is a critical part of -- of our Aviation Safety Program. We take the view that the FAA has the resources and, most importantly, the expertise to bring to American Airlines additional input on things we do well and things we don't do so well. So, we have a very open door with the FAA. We -- we -- we listen attentively to what they say. We are absolutely committed to -- to ATOS as a process that can help us provide more input and more data to identify problems. So, the -- the FAA is really a member of the Safety Team at American Airlines in every respect, and they're very important to us. Q During the course of the investigation of not only this accident but previous accidents, typically we're made aware via different sources about concerns that on-line pilots have within the organization. We've talked about fatigue in this hearing and in this accident, and I just wanted to know if you're aware of any AASAP or OF25 Program issues regarding to crew fatigue, and, if so, what measures were taken to accommodate those or -- or fix those? A I don't think I can -- I can comment as to the presence of fatigue within AASAP or the OF25. Fatigue is -- is something that any one who runs big machinery in challenging areas needs to pay attention to in our view. We have recently reaffirmed our fatigue policy that permits any one of our crew members to take him or herself off duty because of fatigue with no consequences adverse to that employee, and we think that is a sound policy, and -- and it has been used, and that crew member will never hear from us. Q As far as taking this one step further in a different vein, we know that the Department of Transportation does track on-time performance of the carriers, and given the fact that this is a very competitive market, do you see, especially now in your new position as Vice Chairman, the fact that if we have a late-arriving airplane, and the consequences and the publication of that on-time performance, do you see that driving any of these flights to be completed? A No, and if you look at the -- the incident particulars at American Airlines since 1995, in almost every case, these operations were woefully late before they ever got in the air, and -- and, so, that the dependent -- the arrival dependability was -- was certainly not a factor. I don't believe that American's professional pilots are ever going to make a trade-off of being on time versus the safety of the operation, and we certainly would not condone that trade-off or that approach either. Q Given the facts of what we know thus far about this particular flight and some of the things that had been said early on in this investigation by, I guess, some American -- well, one particular American Airlines person who is no longer at the airline, can you just characterize for me and for the rest of us what -- what your assessment may have been as to the captain flying into Little Rock that particular night under the conditions that -- that we're talking about here? We had a two-hour late flight, and going into an area where weather is a -- is a very big consideration in the decision- making process, and this captain's continuation into an area of weather that, based on some testimony and witness interviews, was very questionable at the time. A Mr. Feith, I -- I am not a pilot. I have not been trained on the Super 80 or nor American's procedures. It would be grossly inappropriate for me to register any kind of opinion on Captain Bushman's decision-making that evening. I remain very optimistic that through this process that we're involved in here today, that we, too, will get a better understanding of all the factors involved in this tragic event. But I'm -- I'm not qualified to make that kind of a judgment. I'm s