VOLUME 3 General technical administration
CHAPTER 21 THE ADVANCED QUALIFICATION PROGRAM
Section 5 Safety Assurance System: Crew Resource Management (CRM)/Dispatch Resource Management (DRM) Integration and Assessment
3-1591 GENERAL. This section is related to Safety Assurance System (SAS) Element 2.1.6 (OP), Advanced Qualification Program (AQP).
A. Background. The origins of Crew Resource Management (CRM) training can be traced to research conducted by the National Aeronautics and Space Administration
(NASA) during the 1970s. This research sought to determine the causes of accidents in commercial aviation and came up with the startling (at that time) finding that the majority of accidents
had causal factors related to crew interaction, communication, leadership, and decisionmaking. The outcome of this awareness was the development of training that addressed these aspects of
flight. The training was initially called Cockpit Resource Management, but the name changed to Crew Resource Management as the scope changed to include interfaces with the cabin crew and other
components of the system. NASA records much of the early development of CRM in the proceedings of two industry workshops hosted in 1978 and 1986.
B. CRM Training. CRM training develops skills that enhance flight safety through the effective use of all available resources: human, hardware, and information.
CRM training increases awareness of human and system error and provides techniques and skills that will minimize their effects. This is accomplished through awareness of crewmember attitudes
and behavior as well as the use of practical flight management skills. CRM training has achieved a greater degree of integration through the Advanced Qualification Program (AQP).
3-1592 CURRICULUM INTEGRATION. One of the major objectives of AQP is the seamless integration of CRM/dispatch resource management (DRM) and technical training,
such that CRM/DRM becomes an integral part of the training and the job. Where appropriate, CRM/DRM procedures are identified, documented, integrated, and accorded the same weight as the technical
procedures required for the execution of a given phase and its associated tasks.
A. CRM/DRM in Procedures. Integration does not mean that only those aspects of CRM/DRM that can be proceduralized are addressed in operations, technical training,
and evaluation. However, some aspects of CRM are inherent to maneuver performance and have already been proceduralized. For example, communication procedures for coordinating callouts during
takeoffs, approaches, and other maneuvers are specified in terms of what should be said and when it should be said. These callouts take place during most flight phases and are performed at fixed
points in the flight-phase sequence of task activities. Similarly, some aspects of communication during the management of non-normal conditions can be easily positioned within the sequence of
activities performed to manage the non-normal condition.
B. CRM/DRM on an As-Needed Basis. Other equally important CRM/DRM activities, in contrast, are performed on an as-needed basis, in order to manage the work
load, work well as a team, or respond to unique situations. Recognizing the need for and effectively executing these activities is critical to coordinating the various duties the crew must perform
during the flight. For example, certain communications should be performed in order to maintain crew awareness of flight status. Regardless of the phase of flight, it is critical that the crew
recognize this communication requirement and effectively act upon it in a timely manner to maintain crew situational awareness.
C. Use as a Template. These global activities do not fit neatly within a hierarchical list of technical activities organized by phase of flight. Instead, they act
as a “shell” or “template” that overlays and organizes the activities that may need to be performed during any phase of flight. These skills as a whole constitute a management
strategy that represents a critically important part of the inventory of defenses against error that crews bring with them to the flight deck. In particular, they can provide the crew with the tools
needed to resolve problematic situations to which they may never have been previously exposed in training or in flight operations. It is important, therefore, that training activities seek to fully
instill these skills, in order to provide a basis for generalization to a potentially wide variety of situations.
3-1593 ASSESSING THE EFFECTIVENESS OF CRM/DRM PROGRAMS.
A. Use of Data. AQP stresses the use of data to develop and validate its components. Similarly, the success of CRM/DRM and human factors training rests on data
showing its effectiveness and identifying areas of strength and weakness.
B. Determining Effectiveness. Because of the complexity of the aviation system, multiple sources of data are needed to reach valid conclusions about the effectiveness
of programs. Data on individual and crew practices in the form of observable behaviors in the training environment, as well as student feedback, provide organizations with essential information on
the strengths and weaknesses of training programs. Operational data indicating the frequency and nature of human factors problems encountered in line operations allow determination of how CRM/DRM
is being practiced in an organization.
1) Evaluation and Validation. Training of CRM/DRM is mandatory. A means of evaluating the effectiveness of such training is also mandatory, but pass/fail
CRM/DRM standards are not required. Applicant-developed evaluation strategies must at least include provisions for assessing the extent to which poor CRM/DRM skills are a contributory factor in a
failure to meet technical standards of operational performance in validations and evaluations.
2) Assessment. Assessment, on the other hand, carries no jeopardy for individuals or crews and can be employed to gain an overall understanding of complex issues
and to assist in the development of more effective training. The Performance/Proficiency Database (PPDB), built on the assessment of performance during training, also serves as a feedback loop for
organizations on the effectiveness of their AQP. Non-jeopardy assessment can also provide critical information from line operations as described below.
3) Line Checks/Surveillance. The line check/observation provides not only an opportunity to observe and evaluate crew performance but also the chance to provide
feedback and reinforcement of crew behaviors. The limitation of line check/surveillance data is that they document crew mastery of skills but indicate nothing about the likelihood they will be
applied in an unmonitored environment. Nevertheless, these data form an important component of understanding crew performance.
4) Incident Reports. A number of sources of data can be used to monitor CRM/DRM in line operations.
5) Flight Operations Quality Assurance (FOQA). Through the use of quick access digital recorders, FOQA provides important information on instances where aircraft
operational parameters exceed predetermined operational limits (e.g., unstable approaches). Through agreements between airlines, pilots’ organizations, and the regulator, confidentiality of data
is maintained and the data are used proactively for safety initiatives. For example, FOQA can indicate if “slam dunk” approaches are prevalent at certain airports. Awareness of this can
lead to changes in control practices, if further investigation suggests that compliance with air traffic control (ATC) clearances necessitate such action, or to changes in charting, procedures, or
automation guidance if other culprits become suspect. The major limitation of FOQA data is that they indicate when, but not why, events occur.
6) Safety Audits. Performance during training and formal evaluations provides evidence that skills have been learned but no information about whether the lessons have
been internalized and incorporated into daily operational practices. Safety audit data are a valuable data source because they provide the strongest evidence regarding how much of the material taught
in training and published corporate policy makes its way to the operation. In a line audit, trained observers record their observations of crew behaviors associated with threat recognition, error
avoidance, and error management under non-jeopardy conditions. These data can provide an accurate picture of the acceptance and practice of CRM concepts across an organization, fleet, or operation,
as well as the prevalence of threats, errors, and deviations from standard operating procedures (SOP). It must be noted that if there is no trust between the employee group and organizational
management, the safety audit may be perceived as having the same jeopardy as an evaluation and an accurate snapshot of the operation may not be obtained.
7) Aviation Safety Action Program (ASAP) Reports and Surveys. ASAP reports and anonymous surveys of employees can provide additional, direct data on the status of CRM/DRM.
Such reports and surveys are also helpful in identifying areas in need of additional training and in providing feedback on the operational relevance of training. Non-punitive incident reporting systems
such as ASAP provide incentives for employees to share their experiences (and errors) on the job. Such data provide more immediately useful information than national, de-identified programs such as
NASA’s Aviation Safety Reporting System (ASRS), although the two programs are complementary and both are essential to monitoring the aviation system.
RESERVED. Paragraphs 3-1594 through 3-1605.