4/21/20

 

8900.1 CHG 306

VOLUME 3  GENERAL TECHNICAL ADMINISTRATION

CHAPTER 21  THE ADVANCED QUALIFICATION PROGRAM

Section 1  Safety Assurance System: Scope, Concepts, and Definitions

Indicates new/changed information.

Source Basis:

    Section 121.901, Purpose and Eligibility.

    Section 121.903, General Requirements for Advanced Qualification Programs.

    Section 121.907, Definitions.

    Section 121.909, Approval of Advanced Qualification Program.

    Section 121.911, Indoctrination Curriculum.

    Section 121.913, Qualification Curriculum.

    Section 121.915, Continuing Qualification Curriculum.

    Section 121.917, Other Requirements.

    Section 121.919, Certification.

    Section 121.921, Training Devices and Simulators.

    Section 121.923, Approval of Training, Qualification, or Evaluation by a Person Who Provides Training by Arrangement.

    Section 121.925, Recordkeeping Requirements.

Indicates new/changed information. Indicates new/changed information.

3-1496    ADVANCED QUALIFICATION PROGRAM (AQP) OVERVIEW. In response to the recommendations from a Joint Aviation Task Force and from the National Transportation Safety Board (NTSB), the Federal Aviation Administration (FAA) on October 2, 1990 published Special Federal Aviation Regulation (SFAR) 58, Advanced Qualification Program. The FAA on October 2, 2005 incorporated SFAR 58 into Title 14 of the Code of Federal Regulations (14 CFR) part 121 as a permanent subpart (subpart Y). Under part 121 subpart Y, certificate holders, as well as the training centers they employ, are provided with a regulatory alternative for training, evaluating, qualifying, and certifying pilots, Flight Engineers (FE), flight attendants (F/A), aircraft dispatchers, instructors, evaluators, and other operations personnel subject to the provisions of part 121 and 14 CFR part 135. The capabilities and use of full flight simulators (FFS) and other computer-based training (CBT) devices in training and qualification activities have changed dramatically, as has our knowledge of the role of Crew Resource Management (CRM) issues involving fatal accidents. Part 121 subpart Y allows certificate holders to develop innovative training and qualification programs that incorporate the most recent advances in training methods and techniques. It also requires participants to validate the training and qualification program empirically through continuing data collection and analysis. This section is related to Safety Assurance System (SAS) Element 2.1.6 (OP) Advanced Qualification Program (AQP).

Indicates new/changed information. Indicates new/changed information.

A.    Safety Enhancement. Although AQP is a voluntary program, the Flight Standards Service (FS) encourages certificate holders to participate. The AQP methodology directly supports the FAA’s goals for safety enhancement. AQP provides for enhanced curriculum development and a data-driven approach to quality assurance (QA) along with the flexibility to target critical tasks during training. AQP aligns very well with a systems safety approach for training development, implementation, quality oversight, and surveillance. FS envisions that data obtained from an AQP curriculum(s), in conjunction with information sharing from the flight operations quality assurance (FOQA) program and the Aviation Safety Action Program (ASAP), can provide FAA personnel with information that will enhance oversight capability, capacity, and improve operational safety.

Indicates new/changed information.

B.    AQP Methodology. The AQP is designed to provide a systematic methodology for developing the content of training and qualification programs for crewmembers and aircraft dispatchers. It replaces programmed hours with proficiency-based training and evaluation, integrating CRM, derived from a detailed Job Task Analysis (JTA). AQP incorporates data-driven quality control (QC) processes for validating and maintaining the effectiveness of curriculum content. The AQP encourages innovation in the methods and technology that are used during instruction and evaluation, and efficient management of training systems.

Indicates new/changed information.

C.    Technical and CRM Skills. It is a common misconception that the AQP targets only CRM skills. Although AQP participants must integrate CRM throughout the curriculum and CRM is indeed important to a complete training and qualification program, it is not the only important factor in AQP training. While analysis of commercial accidents indicates that deficiencies in CRM skills have been a significant contributor to mishaps, skill-based errors are associated with a large percentage of aviation accidents. Nearly 60 percent of all accidents involving scheduled air carriers have involved at least one skill-based error. AQP combines technical and CRM skills in training and evaluation in order to achieve proficiency in the entire range of skills required for safe operations.

D.    Other Voluntary Programs. Existing and future voluntary systems will be an integral part of FAA safety oversight activities. An increasing number of AQP participants have begun to implement programs that enable proactive corrective action to reduce the likelihood of accidents. Most AQP participants are using other voluntary data-driven programs (e.g., the FOQA program, the ASAP, and the Line Operations Safety Audit (LOSA) program) to feed information into the AQP in order to better identify training needs, improve training practices, or to simply validate assumptions. FOQA, ASAP, and LOSA information can also help in the design of AQP scenarios that focus on real-world operations.

E.    Regulatory Flexibility. Since its release in 1990, AQP has provided the regulatory flexibility for participants to modernize and innovate training content and methods. AQP participants have pioneered instructor, check pilot, and check FE training, the use of flight simulation training devices (FSTD), distance learning methods, computerized testing, training strategies based on the mission – international versus domestic, the use of pilot demographic information, training intervals based on performance, the use of data-derived training decisions, the integration of other safety feedback programs into training, and fostering a strong relationship between the FAA, industry, and labor.

Indicates new/changed information.

NOTE:  Confusion sometimes arises concerning the relationship between policy guidance and the regulations. Inspectors and certificate holders must bear in mind that Public Laws and regulations—including exemptions—comprise the body of requirements that cannot be changed by policy. Policy guidance documents, such as advisory circulars (AC) and orders are always subordinate to those requirements. Such policy documents may provide guidance on one means, but not necessarily the only means, acceptable to the FAA of achieving compliance with a regulatory requirement, or they may provide guidance on implementing programs for which there is no underlying regulatory requirement, such as ASAP.

NOTE:  AQP, by part 121, §§ 121.903(b) and 121.909(b)(4), allows participants to depart from traditional practices with respect to what, how, when, and where training, evaluation, and certification is conducted. An applicant may propose to replace certain requirements of 14 CFR part 61, 63, 65, 121, or 135 with an AQP curriculum. An AQP may also employ alternatives to the practical test requirements. This is subject to FAA approval of the specific content of each proposed program. Policy established by this section defines the limits and obligations of the proposed alternatives.

Indicates new/changed information.
1)    Goal. The primary goal of AQP is to achieve the highest possible standard of individual and crew performance. In order to achieve this goal, AQP seeks to reduce the probability of crew-related errors by aligning training and evaluation requirements more closely with the known causes of human error. Accidents are typically caused by a chain of errors that build up over the course of a flight that, if undetected or unmitigated, results in a final, fatal error. Singular maneuver-based evaluation artificially segments simulation events, preventing the realistic buildup of the error chain. The AQP development process will integrate CRM skills, systems knowledge, and flying skills in ground training, flight training, validations, and evaluation modules. Traditionally, training and qualification programs have focused primarily on individual training and evaluation. Under AQP, the focus is on individual and crew performance during training and evaluation. An AQP evaluation is scenario-based, allowing the error chain to progress to a logical conclusion. It extends this concept from Line-Oriented Flight Training (LOFT) and Special Purpose Operational Training (SPOT) to Line Operational Evaluation (LOE). This approach allows both technical and cognitive skills to be tested together in a crew-oriented setting.
Indicates new/changed information. Indicates new/changed information.
2)    Equivalent Level of Safety (§ 121.909(b)(4) and (e)). AQP was also established to permit a greater degree of regulatory flexibility in the approval of innovative training and qualification programs. Based on a documented analysis of operational requirements, a certificate holder under AQP may propose to depart from traditional practices with respect to what, how, when, and where training, evaluation and certification is conducted. An applicant may propose to replace certain requirements of part 61, 63, 65, 121, or 135 with an AQP curriculum. An AQP may also employ alternatives to the practical test requirements. This is subject to FAA approval of the specific content of each proposed program. Part 121 subpart Y requires that all departures from traditional regulatory requirements be documented and based upon an approved continuing QA and data analysis process sufficient to establish at least an equivalent level of safety. To determine an equivalent level of safety, a certificate holder’s AQP must be examined as a comprehensive whole rather than considering any one component in isolation.
Indicates new/changed information.
3)    Process. Under an AQP, the FAA monitors the process as well as the product. Instead of basing curriculums on prescribed maneuvers, procedures, and knowledge items, AQP curriculums are based on a detailed analysis of the specific job tasks, knowledge, and skill requirements of each duty position of the individual certificate holder. The AQP process provides a systematic basis for establishing an audit trail between training requirements and training methodologies.
Indicates new/changed information. Indicates new/changed information.
4)    Systematic Basis. AQPs are systematically developed, continuously maintained, and empirically validated, proficiency-based training systems. They allow for the systematic analysis, design, development, implementation, progressive evaluation, and maintenance of training and qualification programs that include integrated resource management, improved instructor/evaluator training and standardization, scenario-based evaluation, and a comprehensive data-driven QA system. AQP provides a systematic basis for matching technology to training requirements and for approving a training and qualification program with the content based on relevance to operational performance.
5)    Requirements. To ensure that the AQP does establish an initial justification and a continuing process to show an equivalent level of safety, mandatory requirements have been established. AQP participants will:
a)    Comply with all aspects of the approved AQP.
b)    Continue the use of the approved processes for development, implementation, and maintenance of program operations throughout the life of the program.
Indicates new/changed information.
c)    Document the requirements of the Airman Certification Standards (ACS), practical test standards (PTS), and part 61, 63, 65, 121, or 135, as applicable, which would be replaced by an AQP curriculum.
Indicates new/changed information.
d)    Comply with each applicable requirement of the ACS or PTS and part 61, 63, 65, 121, or 135 that is not specifically documented in the AQP.
e)    Provide satisfactory justification that the proficiency-based qualification of personnel under AQP meets or exceeds existing part 121 and/or part 135 standards.
f)    Establish an initial justification and a continuing process to show how the AQP curriculum provides an equivalent or better level of safety for each requirement to be replaced.
g)    Include all flightcrew member positions, instructors, and evaluators, and may include other positions, such as F/As, aircraft dispatchers, and other operations personnel.
h)    Base curriculums on an Instructional Systems Development (ISD) methodology. This methodology must incorporate a thorough analysis of the certificate holder’s operations, aircraft, line environment, and job functions.
Indicates new/changed information.
i)    May build upon an existing training and qualification program or be completely new.
j)    Any training or evaluation that is satisfactorily completed in the calendar-month before or the calendar-month after the calendar-month in which it is due is considered to have been completed in the calendar‑month it was due.
k)    Accommodate make, model, and series (M/M/S) aircraft (or variant).
Indicates new/changed information.
l)    Provide three basic types of curriculums for every duty position: indoctrination (for new hires, new instructors, and new evaluators), qualification, and continuing qualification. Specialty curriculums (transition, conversion, upgrade, requalification, refresher, etc.) will be derivatives of the basic types.
m)    Integrate the training and evaluation of technical and CRM/dispatch resource management (DRM) knowledge and skills.
n)    Develop a list of, and text describing, the knowledge requirements, subject materials, job skills, and qualification standards of each task to be trained and evaluated.
Indicates new/changed information.
o)    Develop a list of, and text describing, Operating Experience (OE), evaluation/remediation strategies, provisions for special tracking, and how recency-of-experience requirements will be accomplished.
p)    Include planned (not programmed) hours for ground training, flight training, evaluation, and OE.
q)    Include line operational training (e.g., full crew Line-Operational Simulation (LOS)).
r)    Use LOE or an equivalent evaluation under an AQP acceptable to the FAA for proficiency evaluations.
Indicates new/changed information.
s)    Integrate appropriate flight simulation training devices (FSTD). Flight training devices (FTD) and FFSs will be used to support scenario-based training as appropriate.
Indicates new/changed information.
t)    Develop data collection and analysis processes in order to obtain performance information on crewmembers, aircraft dispatchers, instructors, evaluators, and other operations personnel that will enable the certificate holder and the FAA to determine whether the form and content of training and evaluation activities are satisfactorily accomplishing the overall objectives of the curriculum.
u)    Provide a Master AQP Transition Schedule (MATS) that provides a plan to transition from a traditional program to an AQP. In addition, the MATS should include a plan on how the applicant would return to a traditional program if that becomes necessary or desirable at some later date.
v)    Develop and implement their AQP in five sequential phases:

    Phase One—Initial Application.

    Phase Two—Curriculum Development.

    Phase Three—Training System Implementation.

    Phase Four—Initial Operations (Initial Approval).

    Phase Five—Continuing Operation (Final Approval).

3-1497    UNIQUE CHARACTERISTICS OF AN AQP.

Indicates new/changed information.

A.    Extended Review Team (ERT). The review and surveillance of an AQP prior to final approval is equivalent to a recertification of the applicant’s training and qualification program. The AQP process includes the Principal Operations Inspector (POI) in all decisions and provides for a data-driven validation for all program revisions and modifications.

Indicates new/changed information. Indicates new/changed information.

B.    Components. The ERT combines the national program perspective and standardization with the expertise and insight of the responsible Flight Standards office. The appropriate Safety Standards office will provide assistance to the Flight Standards District Office (FSDO), certificate management office (CMO), or certificate management unit (CMU) from initial application through the final fleet approval as a collaborative effort. An accepted certificate holder AQP application will initiate the Safety Standards office/FSDO/CMO/CMU partnership. The appropriate Safety Standards office will provide assistance during the development, implementation, and review as well as followup reviews for the certificate holder’s AQP. The appropriate Safety Standards office and the FSDO/CMO/CMU will manage program approvals and revisions through an ERT process.

Indicates new/changed information.

C.    JTA and Qualification Standards (§ 121.907). Programmed hours are replaced by a proficiency‑based curriculum. The JTA documents each major task and subtask, together with their associated knowledge and skills, to be addressed in an AQP curriculum. Proficiency requirements are embodied in applicant-developed qualification standards, which specify the performance, standards, and conditions to be employed for training and evaluation purposes. Qualification standards are derived from a JTA that is specific to an individual certificate holder’s aircraft, operating environment, and trainee population.

Indicates new/changed information.

D.    ISD (§ 121.907). The use of the systematic Instructional Systems Development Methodology, known as Instructional Systems Development, must be incorporated into the AQP development process. Applicants may employ any of a wide range of current models or customize their own approaches to curriculum development. Applicants are encouraged to be creative in tailoring their own approach to their requirements, as long as the FAA deems them appropriate. Innovation and practical application may result in equally acceptable AQP. A document entitled “Applied ISD in AQP Development” is available from the appropriate Safety Standards office. The minimal ISD requirements are:

    Develop a job task listing;

    Analyze that listing to determine essential skill and knowledge requirements (either directly or by reference);

    Determine which skill and knowledge requirements must be trained/tested;

    Develop qualification standards that define acceptable operational performance levels;

    Develop proficiency objectives that capture all training requirements;

    Develop tests that measure proficiency in skill and knowledge areas;

    Provide instructional programs that teach and test training requirements;

    Establish and maintain an audit trail of explicit links between task requirements, training requirements, training and evaluation activities, and evaluation results;

    Measure student performance against proficiency objectives and qualification standards for all curriculums; and

    Revise the training program based on student performance levels on an ongoing basis. This data (stored in the Performance Proficiency Database (PPDB)) will be collected and reported to the FAA on a regular basis.

E.    Integration of Resource Management (CRM/DRM) (§ 121.917(a)). Training of CRM/DRM skills is mandatory. Using the ISD process, CRM/DRM skills must be integrated into the ground and flight training segments. A means of evaluating the effectiveness of such training is also mandatory, but pass/fail standards are not required. Applicant-developed evaluation strategies must at least include provisions for assessing the extent to which poor CRM/DRM skills are a contributory factor in a failure to meet technical standards of operational performance in proficiency evaluations and line checks.

Indicates new/changed information. Indicates new/changed information.

F.    Data Reporting (§ 121.917(c)). Data collection and analysis are fundamental parts of AQP. Data collection and analysis processes must ensure that the certificate holder obtains performance information on its crewmembers, aircraft dispatchers, instructors, and evaluators that will enable the certificate holder and the FAA to determine whether the form and content of training and evaluation activities are satisfactorily accomplishing the overall objectives of the curriculum. AQP requires certificate holders to report data on performance in training and evaluation to the appropriate Safety Standards office. This data must be submitted in a digital electronic format approved by the appropriate Safety Standards office. All such data is de-identified. AQP data augments, but does not replace, the physical surveillance of certificate holder training and evaluation programs.

Indicates new/changed information.

NOTE:  The appropriate Safety Standards office uses de-identified data for program monitoring, not to monitor individual crewmembers, aircraft dispatchers, or other operations personnel.

Indicates new/changed information.

G.    Duration of Evaluation Periods and the Continuing Qualification Cycle (§ 121.915(a), (b), (d)). The time period during which all proficiency objectives are trained, validated, or evaluated is called a continuing qualification cycle. Initially, the continuing qualification cycle approved for an AQP must not exceed 24 calendar-months in duration, and must include two or more evaluation periods of equal duration. The FAA may approve extensions of the continuing qualification cycle and evaluation period upon demonstration that the extension is warranted. The maximum permissible duration of a continuing qualification cycle is 36 months. Extensions beyond the initial evaluation period and continuing qualification cycle permitted by AQP (12 and 24 months, respectively, plus or minus 1 month) will require strong justification. To obtain approval for an extension, a participant must show that individuals subject to the AQP are able to maintain their knowledge and skills under the already approved schedules and that a rational basis exists for believing that no loss of knowledge, skill, or ability would result from the extension. An extension will be allowed to continue, or an additional extension will be granted, only if a certificate holder’s record and independent FAA evaluation show that the extension is appropriate as a means to maintain or increase the level of proficiency. This could occur, for example, if the applicant proposes to increase the number of training sessions per evaluation period.

3-1498    KEY CONCEPTS, SPECIAL TERMINOLOGY, AND DEFINITIONS.

Indicates new/changed information.

A.    First Look Maneuvers (§ 121.907). “First Look” maneuvers are applicable only to the continuing qualification curriculum. First Look maneuvers are those maneuvers that are identified as likely to be sensitive to loss of proficiency due to infrequent practice and/or other factors. The principal purpose of First Look is to test the retention of proficiency of the flightcrews over the evaluation period and continuing qualification cycle. First Look is an AQP requirement whenever the evaluation period exceeds the checking/training interval of a traditional part 121 training and qualification program. However, First Look is a valuable tool and should be considered regardless of the length of the evaluation period. For example, First Look may be employed as a means of validating that currency items are performed in line operations with sufficient frequency that proficiency is being maintained. First Look maneuver proficiency assessment is graded using the same measurement methodology and rating criteria used in maneuver validation. The First Look grades are then analyzed to determine trends of degraded proficiency. Repeating First Look maneuvers on an evaluation allows a pretest, post-test measure of the learning during training, where the difference between the session’s First Look grades and the last session’s evaluation grades provide a rough measure of skill decay over the training interval.

B.    Terminal and Supporting Proficiency Objectives (SPO). Terminal Proficiency Objectives (TPO) are derived at the task level from the JTA. They reflect the end-level proficiency to be achieved in training and demonstrated in evaluation. SPO are derived at the subtask level from the JTA. They reflect the subtasks on which proficiency must be achieved in training in order to perform at a satisfactory level on a given TPO. However, a consistent one-to-one correlation between all tasks/subtasks and TPOs/SPOs is not expected. Multiple TPOs may be derived from a single task, or multiple tasks may produce a single TPO, depending on how significantly the performance is altered due to conditions and/or contingencies. Both TPOs and SPOs require specification of the performance to be accomplished, the standards that must be met or exceeded to demonstrate proficiency, and the conditions under which training or evaluation of performance will be conducted. A complete list of TPOs and SPOs would encompass all major tasks and subtasks for all phases of flight/operation.

Indicates new/changed information.

C.    Qualification Standard (§ 121.907). A Qualification Standard is a job task proficiency objective (TPO or SPO) linked to an evaluation strategy together with specification of a media range. A certificate holder’s qualification standards define the requirements of mastery for specific duty positions and replace the ACS or PTS, as applicable, for evaluation and certification. Qualification standards created at the terminal level generally reflect objectives for evaluation, although some such objectives may be addressed only in training to proficiency in the applicant’s approved AQP. Qualification standards created at the supporting level generally reflect objectives for training, although some such objectives may also be identified for evaluation in the applicant’s approved AQP. Figure 3-95, Advanced Qualification Program—Qualification Standards Documentation, presents an example of an AQP qualification standard established at the terminal level. See also Figure 3-96, Advanced Qualification Program—Documentation Checklist and Review Job Aid.

NOTE:  Variation in the format of a given certificate holder’s qualification standards is permissible, provided that all of the categories of information in the example are addressed.

D.    Currency Objectives. The purpose of identifying currency events is to enable training and evaluation to focus on areas of greatest need. An applicant may elect to identify certain objectives (TPOs or SPOs, as applicable) as currency events, based on an analysis of the frequency with which a given objective and condition occur during line operations. Candidates for such a designation include only those objectives on which skills are maintained by virtue of their frequent exercise in flight, such as normal takeoff under Ceiling and Visibility OK (CAVOK) conditions. Approved currency events do not need to be specifically trained in continuing qualification curriculums, provided that the applicant implements a means acceptable to the FAA of periodically verifying that proficiency on such items is being maintained (line checks, safety audits, observations, etc.).

Indicates new/changed information.

E.    Critical Objectives. An applicant may elect to identify certain objectives (TPOs or SPOs, as applicable) as critical or non-critical, based on the applicant’s JTA. The principal consideration with respect to criticality is the frequency of proficiency training or evaluation required to ensure a margin of safety with respect to pilot performance. Critical objectives must be trained, validated or evaluated during every evaluation period. Training or evaluation of non-critical objectives for such a curriculum must occur within every continuing qualification cycle. If the applicant elects not to categorize objectives in terms of criticality, all objectives derived from the applicant’s JTA must be considered critical.

F.    Special Tracking (§ 121.913(b)(4)). Special tracking is the assignment of a person to an augmented schedule of training and/or checking. It is applied to individuals that have failed to demonstrate proficiency during an evaluation event. There are other criteria that the certificate holder may use to place an individual on special tracking. It could include continuing difficulty in completing the maneuver validation, an extended absence from duty, for new pilots in command (PIC), or at the request of the individual trainee. However, the AQP participant may implement an AQP strategy such as reduced training intervals that would offset the need for a specific special tracking strategy.

G.    Train to Proficiency. Training to proficiency is training to a performance level that meets or exceeds a qualification standard. The training must include enough repetition and practice to ensure that each individual can perform at the qualification standard level over the entire evaluation period or continuing qualification cycle.

H.    Dual Qualification. For the purposes of AQP, an individual is deemed “dual qualified” if, during the continuing qualification cycle following an AQP proficiency evaluation the individual performs flightcrew member duties in that aircraft type after becoming qualified and operating another aircraft type during that same continuing qualification period.

I.    Inter-Rater Reliability. Rater reliability is a standardization program for instructors/evaluators. The required data collection and analysis is incumbent upon reliable and valid instructor/evaluator grading judgments. This standardization program is necessary to establish uniform grading criteria, address reliability between instructors/evaluators, and develop remediation procedures. The program must provide rater reliability training during the qualification, transition, continuing qualification, and differences curriculums.

Indicates new/changed information.

J.    Crew-Oriented Scenario-Based Training and Evaluation. All AQP curriculums require crew‑oriented scenario-based training and evaluation. AQP extends this concept from LOFT and SPOT to LOE. LOFT/SPOT/LOE scenarios must be designed to provide the opportunity for training or evaluation, as appropriate, on approved AQP qualification standards and CRM skills. LOE scenarios must be approved for their intended use in an AQP by the POI. A flightcrew member AQP curriculum requires scenario-based training and evaluation conducted with a full crew in a FSTD. Aircraft dispatchers, F/As, and other operations personnel training and evaluation scenarios must be conducted in an operational setting.

K.    Full Crew. A complete paired (PIC/second in command (SIC)) crew complement should be scheduled and maintained. Unpaired flightcrew member (PIC/PIC or SIC/SIC) substitution is highly discouraged. If crew substitutions are unavoidable, the substitute crewmember will be either another line-qualified crewmember or a task-familiar crewmember in a training status comparable to the duty position being substituted. Evaluators conducting a validation or evaluation may not serve as a substitute crewmember. The crew substitution table/matrix will be part of the certificate holder’s approved AQP documentation (Implementation and Operations Plan (I&O Plan)). Refer also to the current edition of AC 120-35, Flightcrew Member Line-Operational Simulations: Line Oriented Flight Training, Special Purpose Operational Training, Line Operational Evaluation.

Indicates new/changed information.

L.    Random Line Check (§ 121.915(b)(2)(ii)). A no-notice line check strategy may be used in lieu of the annual PIC line check. The certificate holder who elects to exercise this option must ensure that the no-notice line checks are administered so that the flightcrew members are not notified in advance of the evaluation. In addition, the certificate holder must ensure that each PIC receives at least one no-notice line check every 24 months. As a minimum, the number of no-notice line checks administered each calendar-year must equal at least 50 percent of the applicant’s PIC workforce, as determined in accordance with a strategy approved by the Administrator for that purpose. In addition, the line checks to be conducted under this paragraph must be representatively sampled over all geographic areas flown by the certificate holder in accordance with a sampling methodology approved by the Administrator for that purpose.

M.    Planned Hours (§ 121.907). All curriculums will include planned hours for ground training, flight training, evaluation, and OE. Planned hours represent the estimated amount of time (as specified in a curriculum outline) that it takes an average student to complete a segment of instruction (to include all instruction, demonstration, practice, and evaluation, as appropriate, to reach proficiency). Planned hours replace the programmed hours associated with traditional programs. Planned hours enable the FAA and the applicant to schedule their personnel resources more efficiently and provide a baseline for curriculum adjustments. Planned hours are not used by the FAA as a basis for program approval, review, or compliance assessment. Planned hours should be shown on the course footprint that is part of the curriculum outline.

Indicates new/changed information.

N.    Conditions. Describe the range of circumstances under which student performance will be measured and evaluated. Conditions include the operational environment (Navigational Aid (NAVAID) inoperable, different aircraft weight, passengers not seated, aircraft configuration, etc.), and natural environment (ceiling, visibility, wind, turbulence, etc.). The qualification standard should (1) indicate those specific conditions to be evaluated as part of the qualification curriculum, and (2) provide a more exhaustive listing of conditions over which crewmembers will be trained and tested during the course of successive continuing qualification cycles.

O.    Contingencies, Abnormal Situations, Minimum Equipment List (MEL)/Configuration Deviation List (CDL), and Emergencies. The qualification standard should (1) indicate those specific contingencies to be evaluated as part of the qualification curriculum, and (2) provide a more exhaustive listing of contingencies over which crewmembers will be trained and tested during the course of successive continuing qualification cycles.

3-1499    REQUIRED DOCUMENTATION. In addition to the supporting documents and manuals provided the FAA in traditional training programs, there are six documents and an annual report requirement that are unique to AQP. The documents are instrumental in managing the AQP and must be maintained throughout the life of the program. The documents can be categorized in two groups, management and database, by their function and interrelationship. The documents along with the reports comprise the Program Audit Database (PADB). The annual report provides a QC function and indicates the status of the program. See SAS Element 2.1.6 (OP) Element Design Data Collection Tool (ED DCT) or Element Performance Data Collection Tool (EP DCT), as applicable, and Figure 3-96.

NOTE:  Variation in the format of a given certificate holder’s qualification standards is permissible, provided that all of the categories of information in the applicable ED DCT or EP DCT and Figure 3-96 (Qualification Standards) are addressed.

A.    Management Documents. These documents include the Application, Instructional Systems Development Methodology, and I&O Plan. These documents are standalone in that a change in one will not necessitate a change in another.

B.    Database Documents. These documents include the Task Analysis, Qualification Standards, and Curriculum Outlines. Because a change to one often means a change to the others, they should be maintained in an interactive database.

C.    Annual Report. The purpose of the report is to identify changes to the curriculums resulting from feedback and analysis of the information in the PPDB.

Indicates new/changed information.
1)    The AQP Application (§ 121.909(a)). The purpose of the application is to establish the applicant’s methodology for developing an AQP for all of its fleets, instructors and evaluators, and for non‑fleet‑specific curriculum (i.e., indoctrination). The application is submitted once and is updated as information in the application warrants changing (a change in the schedule, adding new aircraft, initiating F/A or aircraft dispatcher AQP programs, etc.). See Figure 3-96.
2)    JTA (§ 121.909(b)(2)). JTA is the method or procedure used to reduce a unit of work to its base components. The JTA provides a detailed, sequential listing of tasks, subtasks, and elements (if required) with the knowledge and skills that clearly define and completely describe the job. A JTA provides consideration for conditions surrounding the job both in the environment and in the equipment used. It establishes standards (parameters and tolerances) that provide safe and effective job accomplishment. It also identifies characteristics such as consequence of error, relative difficulty, frequency of occurrence in specific operations, and the time needed to accomplish the task. As a complete document, the JTA has several components: a job task list, learning analysis and crew positions. See Figure 3-94, Advanced Qualification Program—Job Task Analysis Documentation, and Figure 3-96.
3)    Qualification Standards Document (§ 121.909(b)(2)). It is the central AQP document. It provides the regulatory basis for all deviations from current regulations, and summarizes the operational and instructional foundations for all major curriculum elements, such as learning objectives, curriculum outlines, evaluation strategies and gradesheets. The Qualification Standards Document has four parts: the prologue, a regulatory comparison, the evaluation/remediation methodology, and the specific TPO or SPO qualification standards. See Figures 3-95 and 3-96.
Indicates new/changed information. Indicates new/changed information.
4)    Instructional Systems Development Methodology Document (§ 121.909(b)(2)). It describes the approach to be used by applicants to develop and maintain all AQP curriculums. The Instructional Systems Development Methodology document should be finalized before constructing curriculums for each duty position. It applies to pilot, instructor, and evaluator programs, and may be expanded to include other personnel, such as F/As and aircraft dispatchers. This document is divided into two sections. The first section, Curriculum Development Procedures, describes the applicant’s approach for using the JTAs and qualification standards as baseline documents to construct their general training curriculums across all AQP courses. The second section, Line Operational Simulation Methodology, describes the approach for developing LOS scenarios. See Figure 3-96.
5)    Curriculum Outline Document (§ 121.909(b)(2)). This is a listing of course material divided into segments, segments into modules, modules into lessons, and lessons into elements or topics. All AQP curriculum outlines must be structured in accordance with the guidance provided in Volume 3, Chapter 19 for segment, module, lesson, and element. Each part of the curriculum outline must clearly indicate the subject matter to be taught and correspond directly to the hierarchical numbering system of the Task Analysis. The document must detail a clear audit trail tracing each training objective to the appropriate curriculum. A curriculum outline provides the basis for the curriculum footprint, which is a high-level graphical overview of the curriculum content depicting the training and evaluation activities and the proposed hours for each day of the curriculum. Curriculum outlines are developed and submitted with the understanding that application of the course material may require some flexibility regarding the actual day on which each activity is accomplished. See Figure 3-96.
6)    I&O Plan (§ 121.909(b)(2)). This is a milestone schedule detailing the transition to an AQP and a blueprint describing provisions for maintenance, administration, data management, and continuing QC of curriculums. The I&O Plan can be sectioned into two parts. The first part describes how the certificate holder proposes to implement the AQP. Included in this proposal is the schedule for the Phase III training evaluation to include instructor/evaluator training and small group tryouts. It should also include provisos for evaluating the effectiveness of performance measurement tools, and provisions for evaluating facilities, courseware, and equipment before starting the plans for the small group tryouts. The second part explains how the certificate holder intends to operate the AQP in Phases IV and V. Included in this section are strategies for maintaining the program, crew pairing policy, First Look administration, and instructor/evaluator requirements. The operations plan will also describe in detail, the Data Management Plan that includes a statement of understanding addressing the collection and analysis of performance/proficiency data and a description of the PPDB, the data management collection process, and the FAA data submission, analysis, and reporting requirements. See Figure 3-96.
Indicates new/changed information.
7)    Annual AQP Report. Certificate holders will monitor the status of all AQP curriculums and the PPDB and will summarize their findings in an annual report that is given to the POI with a copy to the appropriate Safety Standards office. See Figure 3-96.
8)    Job Aids. SAS Element 2.1.6 ED DCT and/or EP DCT and Figure 3-96.

3-1500    TRAINING, VALIDATION, EVALUATION, REMEDIATION, AND CERTIFICATION.

Indicates new/changed information.

A.    Conformance to This Order’s Standards. AQP follows the direction and guidance provided in Volume 3, Chapter 19 for training and qualification programs. All AQP curriculum outlines must be structured in accordance with the guidance provided in Volume 3, Chapter 19 for segment, module, lesson, and element. Although AQP follows its own phased approval process, the activities described in Volume 3, Chapter 19 with respect to inspector review of curriculum content, are virtually identical under AQP.

Indicates new/changed information.

B.    Elements From Existing Programs. A curriculum approved under an AQP may include elements of existing approved training and qualification programs under parts 121 and 135. Each curriculum must specify the M/M/S and variant of aircraft and each crewmember position or other positions to be covered by that curriculum. Positions to be covered by the AQP must include all flightcrew member positions, instructors, and evaluators, and may include other positions, such as F/As, aircraft dispatchers, and other operations personnel.

C.     AQP Requirements over Part 61, 63, 65, 121, or 135. Each certificate holder that obtains approval of an AQP must comply with all of the requirements of the AQP instead of the corresponding provisions of part 61, 63, 65, 121, or 135. However, each applicable requirement of part 61, 63, 65, 121, or 135 that is not specifically addressed in the AQP continues to apply to the certificate holder. No person may be trained under an AQP unless the FAA has approved that AQP and the person complies with all of the requirements of the AQP.

D.    Completion Month. Any training, validation, or evaluation required under an AQP that is satisfactorily completed in the calendar-month before or the calendar-month after the calendar-month in which it is due is considered to have been completed in the calendar-month it was due.

Indicates new/changed information.

E.    FSTDs (§ 121.921). Procedures for the qualification of FSTDs are described in 14 CFR part 60. An applicant may propose an alternative to the minimum required level of FSTD described in Volume 3, Chapter 19, Section 6, subject to compliance with part 60 and acceptable justification and FAA approval of the intended use of the FSTD in a specific AQP curriculum.

Indicates new/changed information.

F.    Training. Section 121.909(b) requires three primary curriculums for each M/M/S of aircraft (or variant) for each duty position. These curriculums are: Indoctrination, Qualification, and Continuing Qualification. In addition to primary curriculums, operational necessities may require secondary curriculums to fulfill particular needs. Secondary curriculums may include: Transition, Conversion, Upgrade, Differences, Related Aircraft Differences, Requalification, Refresher, and Recency. The three primary curriculums do not have to be implemented simultaneously. For example, participants may elect to develop a continuing qualification curriculum followed by a qualification curriculum implemented sometime in the future.

1)    Indoctrination Curriculum (§ 121.911). (See also Volume 3, Chapter 19.) An indoctrination curriculum consists of all training elements, which will be learned and evaluated before an individual may begin a qualification curriculum. Indoctrination curriculum segments typically consist of ground training and evaluation. Four distinct areas of indoctrination ground training are:
Indicates new/changed information.
a)    Certificate Holder-Specific Training. This training acquaints crewmembers, aircraft dispatchers, instructors, evaluators, and other operations personnel with company policies, practices, and general operational procedures. The subject matter of indoctrination includes elements that pertain to the certificate holder’s methods of compliance with regulations and safe operating practices.
Indicates new/changed information.
b)    Duty Position-Specific Training. This training provides the basic aeronautical knowledge needed to enter the subsequent qualification curriculum. Weather, 14 CFR, security, some emergency, and hazardous material (hazmat) training that is partially certificate holder-specific and partially duty‑position‑specific.
c)    Instructor-Specific Training. The fundamental principles of the teaching and the learning process; methods and theories of instruction; and the knowledge necessary to use aircraft, FSTDs, and other training equipment.
d)    Evaluator-Specific Training. General evaluation requirements of the AQP; methods of evaluating crewmembers, aircraft dispatchers, and other operations personnel, and policies and practices used to conduct the kinds of evaluations particular to an AQP (e.g., LOE).
Indicates new/changed information.

G.    Qualification Curriculum (§ 121.913). (See also Volume 3, Chapter 19.) AQP requires a qualification curriculum for each duty position in each M/M/S aircraft (or variant). Each qualification curriculum will include training, validation, evaluation, and possibly airman certification. The training activities include ground and flight training, operating experience, and may include special qualification training. If the training is to result in airman certification, curriculum segments must explicitly identify the alterative training and evaluation strategy to be used in place of the prescribed practical test requirements of part 61, 63, 65, 121, or 135. The applicant must show that the proposed AQP training and evaluation strategy will ensure individual competence that equals or exceeds the practical test requirements and that that each person certificated through an AQP has demonstrated satisfactory proficiency in the integration of technical and CRM skills.

1)    Ground Qualification Training Activities. To be qualified for a particular duty position, a person will receive job-specific ground training. This training typically includes general operational subjects, technical systems, system integration, and emergency equipment training. Ground Qualification culminates in a systems knowledge validation session that may take the form of a traditional oral examination, written test, or may incorporate other means of systems knowledge validation (e.g., computer-based), as approved by the FAA.
2)    Flight Qualification Training Activities. Each AQP includes curriculum segments for systems training integration and knowledge validation, with training and evaluation in FSTDs, where applicable. Training and evaluation in an aircraft is discouraged, but may be approved by the FAA on a case-by-case basis.
3)    Special Qualification Training. Curriculum segments may include special purpose training. These are portions of ground and flight training that have specific application (e.g., to crewmembers that serve in international operations or for initial introduction of new flight operations, such as Category (CAT) II/III, Global Positioning System (GPS) or Required Navigation Performance (RNP) approaches). Special purpose training and validation may initially be a separate curriculum segment that is later integrated into ground and flight training segments.
4)    Instructor Training. Training and evaluation activities to qualify a person to conduct instruction on how to operate, or on how to ensure the safe operation of a particular M/M/S aircraft (or variant).
5)    Evaluator Training. Training and evaluation activities that are aircraft and equipment specific to qualify a person to assess the performance of persons who operate or who ensure the safe operation of, a particular M/M/S aircraft (or variant).
Indicates new/changed information.

H.    OE. OE curriculum segments are integral to qualification curriculums. OE provides hands-on experience in performing the duties of a newly assigned position under the supervision of an appropriately and currently qualified evaluator/check pilot/check FE. OE is conducted during actual flight operations. Part 121 subpart Y allows participants to propose alternatives to the requirements of § 121.434, subject to FAA approval. Deviation from the OE and operating cycle requirements of § 121.434 based on a designation of related aircraft must be authorized by the Air Transportation Division. (See Volume 3, Chapter 19, Section 12 for additional information regarding deviations based on designation of related aircraft.)

I.    Secondary Curriculums (§ 121.909(b)). Developing a secondary curriculum entails selecting, revising, and arranging modules (with related proficiency objectives) from all three primary curriculums. In all cases, the TPOs, SPOs, and enabling objectives (EO) must include CRM principles and when appropriate include the use of LOS for training, validation, and evaluation. The applicant will identify the differences between the traditional training/checking regulatory requirements and those specified in a certificate holder’s AQP.

1)    Transition Curriculum. This curriculum is applicable for an employee who has been previously trained and qualified in a specific duty position by the certificate holder and is being assigned the same duty position on a different aircraft. In a transition training curriculum, the same qualification standards apply as found in the qualification curriculum. Individuals must meet all the same validation and evaluation sessions as the qualification curriculum. However, the training received may be abbreviated, based on an analysis of the training/validation/evaluation requirements of the qualification curriculum compared to an assessment of the currency, knowledge, skills, and qualifications of the individual. In most cases, the training received would be from modules extracted from the qualification curriculum for that particular aircraft. For example, if both aircraft used the same flight management system (FMS), training may be tailored to be aircraft-specific and proficiency‑validated rather than requiring the student to sit through the entire FMS curriculum segment.
Indicates new/changed information.
2)    Conversion Curriculum. This curriculum is for an employee who has been previously trained and qualified as an FE for the certificate holder and is being assigned as an SIC for the same aircraft type in which he or she was previously trained and qualified. Elements or training modules for this curriculum may be found in all three primary curriculums. In conversion training, the same qualification standards apply as found in the qualification curriculum. The individual must meet at least the same validation and evaluation sessions as within the continuing qualification curriculum for the position that he or she is being assigned. However, the training received may be abbreviated, based on an analysis of the training/validation/evaluation requirements of the qualification and continuing qualification curriculums compared to an assessment of the currency, knowledge, skills, and qualifications of the individual. For example, if the individual is current in the aircraft as an FE, proficiency in training modules, such as systems and emergency drills, may be validated through testing.
Indicates new/changed information.
3)    Upgrade Curriculum. This curriculum is for an employee who has been previously trained and qualified as an SIC for the certificate holder and is being assigned as a PIC for the same aircraft type in which he or she was previously trained and qualified. Elements or training modules for this curriculum may be found in all three primary curriculums. In upgrade training, the same qualification standards apply as found in the qualification curriculum. The individual must meet at least the same validation and evaluation sessions as in the continuing qualification curriculum for the position that they are being assigned. However, the training received may be abbreviated, based on an analysis of the training/validation/evaluation requirements of the qualification and continuing qualification curriculums compared to an assessment of the currency, knowledge, skills, and qualifications of the individual. For example, if the individual is current in the aircraft as an SIC, proficiency in training modules such as systems, FMS, and emergency drills may be validated through testing. Other training, such as seat dependent task training, command authority, and CRM, may be trained and evaluated using a combination of classroom and LOS methodology.
4)    Downgrade. Seat-dependent task training is required when a PIC is reassigned as an SIC on the same type aircraft. It may be appropriate to provide modules from several different secondary curriculums (i.e., transition and requalification) if the flightcrew member has never served as an SIC on that type aircraft. The training required when the downgrade is from PIC to SIC from one type aircraft to another type aircraft will depend on whether the crewmember was previously qualified as an SIC in that type. If previously qualified in the duty position and type, a requalification curriculum based on time away is appropriate. If only previously qualified in the duty position, transition is appropriate.
5)    Refresher Training. This curriculum is for an individual who has not met the time requirements of consolidation. The individual must be requalified using this secondary curriculum in order to resume serving in that duty position.

J.    Continuing Qualification Curriculum (§ 121.915). (See also Volume 3, Chapter 19.) This curriculum provides the means for fully qualified individuals to maintain their proficiency in their duty positions and aircraft assignments. Similar to the Qualification Curriculum, each Continuing Qualification Curriculum will include training, validation, and evaluation.

1)    Continuing Qualification Cycle. During a continuing qualification cycle each person qualified under an AQP, including instructors and evaluators, will receive a mix of training and evaluation on the objectives necessary to ensure that each person maintains proficiency in the knowledge, technical skills, and cognitive skills required for initial qualification. Each continuing qualification cycle must include at least the following:
a)    Evaluation Period. Initially, the continuing qualification cycle is comprised of two or more evaluation periods of equal duration. Each person qualified under an AQP must receive ground training and flight training, as appropriate, and an evaluation of proficiency during each evaluation period.
b)    Training Session. A contiguously scheduled period devoted to training activities at a facility acceptable to the FAA for that purpose. The FAA must approve the number and frequency of training sessions.
2)    Training. Continuing qualification must include training in all tasks, procedures, and subjects required in accordance with the approved program documentation, as follows:
Indicates new/changed information.
a)    Ground Training. For crewmembers, instructors, evaluators, aircraft dispatchers and other operational personnel, a general review of knowledge and skills covered in qualification training, hazmat, security, emergency equipment, updated information on newly developed procedures, and safety information and, if applicable, a line observation program.
b)    Flight Training. For crewmembers, instructors, evaluators, and other operational personnel who conduct their duties in flight; proficiency training in an aircraft or FSTD, as appropriate, on normal, abnormal, and emergency flight procedures and maneuvers. For instructors and evaluators who are limited to conducting their duties in FSTDs: training in operational flight procedures and maneuvers (normal, abnormal, and emergency).
c)    First Look. For PICs, SICs, and FEs, in accordance with the certificate holder’s approved program.
Indicates new/changed information.
d)    Instructors/Evaluators. Proficiency training in a FSTD regarding FSTD operation and inter‑rater reliability training.
e)    Recency of Experience. For PICs, SICs, FEs, aircraft dispatchers, instructors, evaluators, and F/As, approved recency-of-experience requirements appropriate to the duty position.

K.    Requalification Curriculum. This curriculum is for an individual who has not met the requirements of a Continuing Qualification Curriculum and becomes unqualified for the duty position. The individual must be requalified to resume serving in that duty position. An AQP applicant should establish nonqualification duration limits based on a currency analysis beyond which an individual would be required to repeat some or the entire indoctrination and qualification curriculum to requalify.

L.    Dual Qualification. When maintaining qualification in more than one aircraft type, the individual will have one aircraft type designated as “primary” and other aircraft type(s) designated as “secondary.” For each aircraft type for which they are maintaining qualification, the individual must accomplish the AQP Continuing Qualification Curriculum for that aircraft. Those training items that are not “fleet-specific” in nature need only be addressed in the “primary” aircraft’s AQP continuing qualification program. In addition, the individual must accomplish at least one line check during the Continuing Qualification Cycle on the “primary” aircraft. The individual should then receive a line check on a different aircraft type each successive year so they are given a line check on all “secondary” aircraft types prior to reaccomplishing a line check on the first “secondary” aircraft type.

Indicates new/changed information.

M.    Differences. (See Volume 3, Chapter 19, Section 9.) Variations between the FSB Report and a certificate holder’s proposed differences training, checking, and currency requirements under an AQP must be justified, documented, and approved by the ERT.

NOTE:  For every proficiency objective, the participant must designate the testing/validation/evaluation strategy. These strategies may include, but are not limited to, Train to Proficiency, Systems/Knowledge Validation, Procedures Validation, Maneuvers Validation, LOE, or Line Check.

Indicates new/changed information.

N.    Related Aircraft Differences. If a certificate holder’s proposal includes related aircraft differences training, the certificate holder must first receive approval of the related aircraft designation from the Air Transportation Division prior to initial approval of the curriculum. Variations between the FSB Report and a certificate holder’s proposed related aircraft differences training, checking, and currency requirements under an AQP must be justified, documented, and approved by the appropriate Safety Standards office. Deviation from the requirements of § 121.434, § 121.439, or § 121.441, based on a designation of related aircraft, must be authorized by the Air Transportation Division. (See Volume 3, Chapter 19, Section 12 for additional information regarding related aircraft designation, related aircraft differences training, and deviations based on designation of related aircraft.)

O.    Training to Proficiency.

Indicates new/changed information.
1)    The train to proficiency strategy is not unique to an AQP curriculum. Traditional part 121/135 training and qualification programs have been using the “train to proficiency” concept since the early 1980s. It is not practical or necessary to evaluate the applicant in every event in which the applicant has received training. A reasonable number of events per evaluation should accomplish the purpose of ensuring that the applicant is proficient throughout the range of objectives in which training was conducted.
Indicates new/changed information. Indicates new/changed information.
2)    Training to proficiency for initial qualification flightcrew members is permitted by § 121.401(e), Training Program: General, allowing that a person who progresses successfully through flight training, is recommended by his instructor, check pilot, or check FE, and successfully completes the appropriate check for a check pilot, check FE, or the Administrator, need not complete the programmed hours of flight training for the particular airplane. Volume 3, Chapter 19 urges POIs to emphasize that training is of primary importance, and checking is the means of verifying that training has been adequate. The POI can consider the objective to have been met when the crewmember is able to perform at the required standard of proficiency immediately before entering the next cycle of required training. Section 121.441(d) allows waiver discretion under certain conditions to the check pilot (also a qualified examiner), or the FAA inspector conducting the check for those crewmembers employed by certificate holders conducting part 121 operations. This waiver authority requires that the applicant be trained to proficiency in all waiver-eligible maneuvers/procedures in the certificate holder’s FAA-approved training and qualification program.
3)    When using the AQP train to proficiency concept, the training provided must ensure the trainee’s performance is to a level that meets or exceeds the specific qualification standard. This concept must include enough repetition and practice to ensure that each individual can perform at the qualification standard over the entire evaluation period or continuing qualification cycle.

P.    Validation.

1)    Validation is a determination that the training produces the required results as identified in the qualification standards and that the individual has met the performance objectives of the training module.
2)    Participants are encouraged to be innovative with their validation strategy. Any validation strategy must ensure that the level of proficiency of each individual is appropriate and adequate to move on to the next level. All validation/evaluation strategies require ERT approval.
3)    The following are flightcrew member validation strategies by curriculum:

Q.    Indoctrination. End of course exam, generally a written test with a required passing score of 80 percent.

R.    Qualification Curriculum.

1)    Systems Knowledge Validation. This is an assessment of an individual’s technical systems knowledge. Systems knowledge validation may be accomplished by a written, electronic, or oral exam.
2)    Procedures Validation. This is an assessment of an individual’s systems integration knowledge and skill. This validation addresses the individual’s ability to assimilate system and procedural knowledge into the appropriate execution of procedures. This validation session should take place in an FSTD. The intent of the Procedural Validation session is to ensure an individual’s systems and procedural knowledge is at an appropriate level before progressing into the flight training phase. There is no established requirement that the systems and procedures validation have to be accomplished sequentially.
3)    Maneuvers Validation (MV). This validation addresses an individual’s proficiency in the execution of maneuvers. For a Qualification Curriculum, crewmembers are expected to have reached a satisfactory level of proficiency in the maneuvers prior to this validation event.
Indicates new/changed information.
4)    OE. OE provides hands-on experience in performing the duties of a newly assigned position under the supervision of an appropriately and currently qualified evaluator (check pilot or check FE). OE is conducted during actual flight operations. Part 121 subpart Y allows participants to propose alternatives to the requirements of § 121.434, subject to FAA approval. Deviation from the OE and operating cycle requirements of § 121.434 based on a designation of related aircraft must be authorized by the Air Transportation Division. (See Volume 3, Chapter 19, Section 12 for additional information regarding deviations based on designation of related aircraft.)

S.    Continuing Qualification Curriculum.

Indicates new/changed information.
1)    First Look. First Look maneuvers are applicable only to the Continuing Qualification curriculum. First Look maneuvers are those maneuvers that are identified as likely to be sensitive to loss of proficiency due to infrequent practice and other factors. The principal purpose of First Look is to test the retention of proficiency of the flightcrews over the evaluation cycle. First Look is an AQP requirement whenever the evaluation period exceeds thechecking/training interval of a traditional part 121 training and qualification program. However, First Look is a valuable tool and should be considered regardless of the length of the evaluation period. First Look proficiency assessment is graded using the same measurement methodology and rating criteria used for the maneuvers validation.
a)    List of Maneuvers. The listing of First Look maneuvers is developed by the applicant and approved by the FAA. First Look items are performed, graded, and analyzed to validate that flightcrews can maintain proficiency in these items between training intervals. These may include certain items given a designation of “currency” in the qualification standard, if any, in order to facilitate initial validation that these items are being performed outside of training with sufficient frequency that proficiency is being maintained.
Indicates new/changed information.
b)    Validation Strategy. The validation strategy the applicant develops for First Look is detailed in the I&O Plan. An ideal approach would be to have a listing of several critical and/or currency items that will be sampled under a controlled sampling technique that would ensure that each of the items are adequately and evenly assessed during the evaluation period. It is important to remember that First Look is not as much an assessment of an individual’s skills, as it is a measure of the collective retention of proficiency by flightcrews. Individual assessment occurs in maneuvers validation and LOE. The data that is collected from First Look is used for trend analysis and as a tool to validate the AQP’s overall effectiveness.
c)    Administration. First Look objectives/maneuvers must not be briefed in advance of the first execution of such maneuvers. Proficiency data must be collected before the repeated execution of any such First Look item during training. There are several options as to when the First Look validation can be conducted. For example, a First Look maneuver could be introduced as the first event of an FFS training session addressing maneuvers. Other options would be to make it part of an event in a LOFT or SPOT. The common element in all such options is that proficiency is assessed the first time the First Look item occurs in training.
d)    Remediation. First Look proficiency assessment is considered a no-jeopardy event, subject to the requirement that any maneuvers unsuccessfully accomplished be trained to proficiency prior to the LOE. If an applicant proposes to request maneuver validation credit for First Look maneuvers, the applicant must assure that the First Look proficiency assessment is accomplished by an AQP qualified evaluator, rather than by an instructor.
2)    MV. The MV session in the Continuing Qualification Curriculum allows assessment and attainment of technical proficiency prior to evaluation in the LOE. A Continuing Qualification Curriculum MV must be successfully completed within the time limits of the standard company-scheduled FFS session (national norm is 2 hours per crewmember) or an additional training period is required. If an individual requires additional training periods to be able to demonstrate proficiency, consideration should be given to placing the individual in Special Tracking.

T.    Evaluation.

1)    Qualification Curriculum. The LOE evaluation addresses the individual’s ability to demonstrate technical and CRM skills appropriate to fulfilling job requirements in a full mission scenario environment. The intent of the LOE is to evaluate and verify that an individual’s job knowledge, technical skills, and CRM skills are commensurate with AQP qualification standards and that the individual is qualified to begin the OE portion of the Qualification Curriculum (refer to AC 120-35). An LOE failure is reported to the FAA. If the LOE is to result in certification, a failure will result in the issuance of FAA Form 8060-5, Notice of Disapproval of Application.

NOTE:  An Aircrew Program Designee (APD) or FAA Operations inspector must administer all Qualification Curriculum LOEs that result in airman certification; otherwise, a Qualification LOE may be administered by a check pilot or check FE. A check pilot or check FE may administer continuing Qualification LOEs.

2)    Continuing Qualification.
a)    LOE. This evaluation addresses the individual’s ability to demonstrate technical and CRM skills appropriate to fulfilling job requirements in a full mission scenario environment. The intent of the LOE is to evaluate and verify that an individual’s job knowledge, technical skills, and CRM skills are commensurate with AQP qualification standards and that the individual is still qualified. The purpose, administration, and remediation strategy for the Continuing Qualification Curriculum LOE is the same as for a Qualification Curriculum.
b)    Line Check. An annual continuing qualification line check is mandatory for each PIC. It is conducted in an aircraft during actual flight operations under part 121 or 135, or during operationally (line)‑oriented flights, such as ferry flights or proving flights. A line check must be completed in the calendar‑month at the midpoint of the evaluation period plus or minus 1 month.
3)    No-Notice Line Check.
a)    Section 121.915(b)(2)(ii) allows a “no-notice” line check strategy to be used in lieu of the line check required by the above paragraph. The certificate holder who elects to exercise this option must ensure that the “no-notice” line checks are administered so that the flightcrew members are not notified in advance of the evaluation. In addition, the certificate holder must ensure that each PIC receives at least one “no-notice” line check every 24 months. As a minimum, the number of “no-notice” line checks administered each calendar-year must equal at least 50 percent of the applicant’s PIC workforce, as determined in accordance with a strategy approved by the Administrator for that purpose. In addition, the line checks to be conducted under this paragraph must be representatively sampled over all geographic areas flown by the certificate holder in accordance with a sampling methodology approved by the Administrator for that purpose.
b)    During the line check, each person performing duties as a PIC, SIC, or FE for that flight must be individually evaluated to determine whether the person remains adequately trained and currently proficient with respect to the particular aircraft, crew position, and type of operation in which he or she serves; and that the person has sufficient knowledge and skills to operate effectively as part of a crew. The evaluator must be check pilot, an APD, or the Administrator and must hold the certificates and ratings required of the PIC.

U.    Repeating Objectives/Maneuvers During Validations and Evaluations.

Indicates new/changed information.
1)    Training and repeating substandard maneuvers during a traditional proficiency check has been permitted by § 121.441 for many years. Section 61.157(f)(1)(i) allows a § 121.441 proficiency check to meet the part 61 certification flight proficiency requirements for an ATP certificate and aircraft type rating. Section 121.441(e) reads, “If the pilot being checked fails any of the required maneuvers, the person giving the proficiency check may give additional training to the pilot during the course of the proficiency check. In addition to repeating the maneuvers failed, the person giving the proficiency check may require the pilot being checked to repeat any other maneuvers he finds are necessary to determine the pilot’s proficiency.” Volume 3, Chapter 19 calls this repeat process “Training to Proficiency.” It also states that when a check pilot determines that an event is unsatisfactory, the check pilot may conduct training and repeat the testing of that event. This provision has been made in the interest of fairness and to avoid undue hardship and expense for pilots and certificate holders.
2)    Under an AQP, repeats of objectives/maneuvers are allowed in all validations and evaluations under certain specific conditions. The conditions depend on the event and the curriculum.

V.    Qualification Curriculum.

1)    Systems Validation. If a written or electronic testing system is used in lieu of an oral exam, a score of 80 percent or greater corrected to 100 percent would be an acceptable validation. An overall score of less than 80 percent would require retraining and retesting. A failure of an individual module or subsection, with an 80 percent or better overall only requires retraining and retesting of the specific module. Consideration should be given to establishing a maximum number of modules or subsections that, if failed, constitute an overall failure of the validation.
2)    Procedures Validation (PV). Training repeats are allowed and are not counted as a validation repeat. The PV must be successfully completed within the time limits of the standard company scheduled validation session or an additional training period is required. After additional training, the individual need only repeat the objectives that were graded unsatisfactory.
3)    MV. MV must not allow more than two repeats of any one maneuver or one repeat of any two maneuvers. A debriefing of why the maneuver(s) was unsatisfactory is allowed, but the repeats must occur with no training, practice, or coaching. If the crewmember fails to demonstrate proficiency within the time constraints of the FFS session, the MV is considered unsatisfactory and additional training is required. After additional training, the individual need only repeat the maneuvers that were unsatisfactory. Failure of the MV does not result in the issuance of FAA Form 8060-5.
Indicates new/changed information. Indicates new/changed information.
4)    LOE. Grading of an LOE is at the event set level. An LOE with more than 25 percent of the event sets graded unsuccessful would constitute a failure of the LOE and will require additional training and another LOE. For an LOE with 25 percent of the event sets or fewer graded unsuccessful, repeats may be permitted at the end of the LOE session, if time permits and if it is possible to recreate the conditions similar to the original event set for the repeat. For example, if an LOE has five, six, or seven event sets, only one set may be repeated. If an LOE has between 8 and 11 event sets, 2 may be repeated. No event set can be repeated more than once. A debriefing of why the event set(s) is unsatisfactory is allowed, but the repeat must occur with no training, practice, or coaching. If any repeated event is unsatisfactory, additional training and another complete LOE evaluation is required. A failure of an LOE requires that the individual enter special tracking. If the LOE is to result in certification, a failure will result in the issuance of FAA Form 8060-5.

NOTE:  Regardless of the number of events sets, unsafe individual or crew performance that would result in significant damage, hull loss, or loss of life (i.e., crash) during an LOE constitutes a failure of the LOE.

Indicates new/changed information.
5)    Line Check. Crewmembers receiving this evaluation are assessed for their proficiency in the duty position. Successful completion of the line check verifies that the individual is adequately trained and is capable of performing the duties and responsibilities of the crew position. If any task is unsatisfactory, the individual must receive additional training on that task, additional OE if necessary, and if appropriate another line check. If a pilot receives an unsatisfactory overall performance rating on a line check, the pilot must be removed from continued line operations until the approved remediation has been successfully completed.

W.    Continuing Qualification Curriculum.

1)    MV. Training repeats are allowed and are not counted as an evaluation repeat. A Continuing Qualification MV must be successfully completed within the time limits of the standard company scheduled FFS session (national norm is 2 hours per crewmember) or an additional MV period is required.
2)    LOE. Same as Qualification Curriculum.
3)    Line Check. Same as Qualification Curriculum.

X.    Remediation.

1)    Remediation Strategy (§ 121.913(b)(4), (d)(3), and (e)(3)). The Qualification Standards Document must describe the methodology that will be used to remediate unsuccessful testing, validation, or evaluation sessions. This remediation strategy must detail when and what may be repeated and whether or not additional training is warranted. Remediation strategies must also specify when no more training will be offered to the individual and the resulting actions. This strategy may be presented in narrative or flow diagram format.
2)    Special Tracking (§ 121.913(b)(4)). This is a program of monitoring the proficiency of an individual at scheduled intervals. It is applied to individuals that have failed to demonstrate proficiency during an evaluation event. There are other criteria that the certificate holder may use to place an individual on special tracking. It could include continuing difficulty in completing the MV, an extended absence from duty, for new PICs, or at the request of the individual trainee. However, the AQP participant may implement an AQP strategy such as reduced training intervals that would offset the need for a specific special tracking strategy. This section will discuss: the situation(s) that requires an individual to be placed in special tracking, the strategy to be used, and when special tracking is no longer required.

Y.    Certification.

1)    General. AQP provides an alternative practical means to certificate pilots, FEs, and aircraft dispatchers.
2)    Certification Requirements (§ 121.919). An applicant for AQP certification is eligible to receive a commercial, airline transport pilot, FE, or aircraft dispatcher certificate or appropriate rating based upon successful completion of training and evaluation events accomplished under the AQP if the following requirements are met:
Indicates new/changed information.
a)    In Accordance with the Applicable Requirements of Part 61, 63, or 65. Certificate holders may develop and implement an alternative training and evaluation strategy, as specified in the Qualification Standards, if it can be demonstrated that the new criteria represent an equivalent or better measure of competence, operational proficiency, and safety. Once approved, AQP training and evaluation strategies are used in place of the prescribed practical test requirements of parts 61, 63, and 65, § 121.441, part 121 appendix E/F, and the applicable ACS or PTS.
b)    Competence. Applicants for certification and/or qualification, including aircraft dispatchers, must show competence in required technical proficiency objectives and CRM/DRM in actual or simulated operational scenarios (i.e., LOE) that assess both types of skills together.
c)    Qualification Curriculum Completion. An applicant must have successfully completed the entire Qualification Curriculum, as appropriate.
d)    Eligibility. An applicant must be eligible under the applicable requirements of part 61, 63, or 65.
e)    Training. The applicant must be trained to proficiency on the certificate holder’s approved AQP Qualification Standards as witnessed by an instructor, check pilot, check FE, or APD and have passed a LOE administered by an APD or the FAA.
Indicates new/changed information.
3)    Application. An applicant for FAA airman certification must make application for certification on FAA Form 8710-1, Airman Certificate and/or Rating Application. There is a signature requirement on the back of the application form. It is the block entitled “Evaluator’s Record (Use for All ATP Certificate(s) and/or Type Rating(s)).” Signature in the AQP Block constitutes verification, by a competent authority, that all of the requirements under the specific provisions of a certificate holder’s AQP have been satisfactorily accomplished. In this instance, the competent authority is an APD, Training Center Evaluator (TCE) authorized by the FAA, or a qualified FAA inspector.
Indicates new/changed information. Indicates new/changed information.
4)    Personnel Authorized to Conduct Certification Activities. POIs may only authorize aviation safety inspectors (ASI), Aircrew Program Managers (APM), Training Center Program Managers (TCPM), or APDs, to conduct pilot examination activities that have completed the requisite AQP evaluator training. Historically, ASIs and APDs certified airmen using a maneuver-based evaluation scenario derived from the practical test requirements of part 61, 63, or 65, as appropriate. AQP replaces the traditional pilot maneuver‑based certification check with a scenario-based LOE. Due to the unique characteristics and detailed scripting of LOEs, all personnel approved to conduct certification activities are required to complete AQP evaluator training. This training should consist of the evaluator indoctrination and qualification curriculums for new personnel or a differences module for ASIs and APDs who are already qualified under the certificate holder’s traditional program.
Indicates new/changed information.

NOTE:  ASIs who have completed a certificate holder’s AQP evaluator training may administer certification activities for ASIs from other FS offices without completing another certificate holder’s training.

Indicates new/changed information. Indicates new/changed information.
5)    POI Authorization. POI authorization is accomplished by appropriate entry on FAA Form 8710-6, Examiner Designation and Qualification Record, Block 10 (with limitations, if any, noted on side 2) and FAA Form 8430-9, Certificate of Authority, under “is authorized to act in the capacity of a,” specifying the aircraft on which AQP certification activity is authorized. FAA Forms 8430-9 and 8710-6 are not required for ASIs.

Figure 3-94.  Advanced Qualification Program—Job Task Analysis Documentation

This is the section of the Job Task Analysis (JTA) that serves as the foundation document for the sample qualification standards that follow. Task 6.1 serves as the basis for the TPO-level qualification standard while subtasks 6.1.1 and 6.1.2 serve as the foundation for the SPO-level qualification standards.

Generic Airlines Inc.

Operations Manual

Volume 9

Chapter 3: Qualification Standards, B757 Flight Crew

 

Rev. # 5

 

Dated 10/01/07

 

 

 

6. Approach Operations and Landing

 

6.1 Perform an Instrument Approach*

 

6.1.1 Perform a 2 Engine Precision Approach CAT I ILS and Landing*

 

6.1.2 Perform One Engine Inoperative CAT I ILS Approach*

 

6.1.2 Perform CAT II ILS

 

6.1.3 Perform CAT IIIb ILS

 

6.1.4 Perform Coupled Autopilot Approach and Autoland Procedures

 

6.1.5 Perform Non-Precision Approach Procedures (VOR, NDB, LOC, LOC/BC, LDA, SDF, ASR, RNAV/FMS, GPS)

 

6.1.6 Perform an IMC One Engine Inoperative Missed Approach

 

6.2 Visual Approach

 

6.3.1 Perform Visual Approach and Landing

 

6.3.2 Perform Visual Approach and Rejected landing

* Used as an example.

Figure 3-95.  Advanced Qualification Program—Qualification Standards Documentation

Generic Airlines Inc

Operations Manual

Volume 9

 

Chapter 3: Qualification Standards, B757 Flight Crew

 

 

Rev. # 8

 

Dated 10/25/07

 

Task: 6.1 Perform an Instrument Approach

 

 

 

Duty Position: All

 

 

 

Criticality: Yes

Currency: Yes

 

 

Curricula: Qualification & Continuing Qualification

 

 

 

Performance Statement:

All instrument approach procedures will be conducted in accordance with the applicable profiles as outlined in Volume 6 of the Operations Manual (B757 Flight Standards). During all instrument approaches, the PF will configure the airplane so as to be stabilized on the approach by 500 feet below the FAF. The PM will continuously monitor the approach and make the required instrument approach altitude calls and deviation calls based on the parameters defined in Volume 6, Chapter 3 of the Operations Manual.

Conditions:

IMC * lowest approach minimums

Turbulence * light

Strong Cross Winds Present * lowest approach minimums

Icing Conditions Present * light rime

Contingencies:

Flight Director Inop

FMS Inop

Autopilot Inoperative * Wind Shear

Loss of ATC Communication

Engine Failure Inside of the FAF

Loss of Flight/Navigation Instrument

 

Standards:

During all instrument approaches, the PF achieves and maintains a stable airspeed, descent rate, vertical flight path, and configuration by 500 feet below the FAF. Between 1000 feet and 500 feet above DH or MDA, only minimum deviations from the standards below occur and are corrected with the proper calls and responses as defined in Volume 6 of the Operations Manual. Below 500 feet above DH or MDA in IMC conditions, with any deviations greater than these standards, the crew calls for and executes a missed approach. The following standards apply: Prior to the FAF, the PF maintains desired altitude within +/- 100 ft., desired heading within +/- 5 degrees, and desired airspeed within +/- 10 knots; Inside the FAF or Final Segment, the PF maintains desired Airspeed within +5 or -0 knots, Localizer or VOR course within 1/2 dot either side, RMI course within +/- 5 degrees, Glideslope within 1/2 dot either side, Sink rate within 1000 fpm, and desired altitude on a non-precision approach within +50/-0 feet. The crew makes all of the required instrument approach calls and responses as detailed in Volume 6 of the Operations Manual. The crew complies with all ATC instructions and clearances or advises ATC if unable. The crew complies with the specific instrument approach profile as depicted in Volume 6 of the Operations Manual. The instrument approach procedure is flown correctly and adjusted as necessary based on equipment availability or other factors. The crew immediately initiates a missed approach upon arriving at the DH or MAP if the required visual references are not distinctly visible. The landing checklist is completed in a timely manner without errors or omissions.

At all times during the approach, the crew uses standard phraseology and procedures to improve situational awareness, and communicates changes in systems or flight profiles in a clear, timely manner. The crew demonstrates strong knowledge of, and makes sound judgments concerning instrument approach procedures and policies.

Media: Level C FFS

Evaluation Strategy: Qualification – Maneuvers Validation/Continuing Qualification – Maneuvers Validation

Reference:

Volume 6 of the Operations Manual

Airman’s Information Manual Chapter: 1

Airman’s Information Manual Chapter: 5

Jeppesen Airway Manual

* identifies specific condition used during qualification validation/evaluation

Generic Airlines Inc

Operations Manual

Volume 9

 

Chapter 3: Qualification Standards, B757 Flight Crew

 

 

Rev. # 8

 

Dated 10/25/07

 

Task: 6.1.1 Perform a 2 Engine Precision Approach CAT I ILS

 

 

 

Duty Position: All

 

 

 

Criticality: No

Currency: Yes

 

 

Curricula: Qualification & Continuing Qualification

 

 

 

Performance Statement:

The crew will complete the appropriate approach briefing prior to the initial approach fix. Once cleared for the approach, the PF will configure the airplane prior to the FAF per the Generic Airline B767 Precision Approach Profile, so as to be stabilized on the approach by 500 feet below the FAF as outlined in Volume 6 of the Operations Manual. The PF will call for the configuration changes and the PM will perform each action. The landing checklist will be completed during the approach prior to landing. The PM will give the required altitude calls in reference to DH as outlined in Volume 6 of the Operations Manual. When deviations from the standards occur, the PM will give the proper correction callouts as defined in Volume 6 of the Operations Manual. With the adequate visual references in sight, the crew will execute the required calls and responses and transition to a normal landing. If, upon arriving at DH, neither the approach lights nor runway is in sight, then the PM will call for the missed approach. If upon arriving at DH and only the approach light system is visible, the PF may continue the approach down to 100 feet above touchdown zone elevation. If at this point the requirements of FAR 91.175 are not visible, the PF will call for the missed approach and the crew will perform the missed approach procedures. If the runway environment becomes visible prior to 1000 feet above DH, the PM will call the runway in sight and the PF will state “Visual Calls.” The PM will then proceed with stating the appropriate visual approach calls and the visual approach procedures may be applied.

 

Conditions:

IMC * lowest approach minimums

Strong Cross Winds Present * 15K

Icing Conditions Present * Light Rime

Contingencies:

Flight Director Inop

FMS Inop

*Autopilot Inoperative

 

Standards:

For all precision instrument approaches, the PF achieves and maintains a stable airspeed, descent rate, vertical flight path, and configuration by 500 feet below DH. Between 1000 feet and 500 feet above DH, only minimum deviations from the standards below occur and are corrected with the proper calls and responses as defined in Chapter 6, Volume 6 of the Operations Manual. Below 1000 feet above DH in IMC conditions, with any deviations greater than these standards, the crew calls for and executes a missed approach. following standards apply: Prior to the FAF, the PF maintains desired altitude within +/- 100 ft., desired heading within +/- 5 degrees, and desired airspeed within +/- 10 knots; Inside the FAF or Final Segment, the PF maintains desired Airspeed within +5 or -0 knots, Localizer course within 1/2 dot either side, and Glideslope within 1/2 dot either side. The crew makes all of the required instrument approach calls and responses as detailed in Volume 6 of the Operations Manual. The crew complies with all ATC instructions and clearances or advises ATC if unable. The crew complies with the Generic Airlines Precision Instrument Approach Profile as depicted in Volume 6 of the Operations Manual. The instrument approach procedure is flown correctly and adjusted as necessary based on equipment availability or other factors. The crew immediately initiates a missed approach upon arriving at the DH if the required visual references are not distinctly visible. The landing checklist is completed in a timely manner without errors or omissions. The crew demonstrates strong knowledge of, and makes sound judgments.

Media: Level C FFS

Qualifying Evaluation Event: Maneuvers Validation/Continuing Qualification – Line Check (Sample)

Reference:

Volume 6 of the Operations Manual

Airman’s Information Manual Chapter: 1

Airman’s Information Manual Chapter: 5

Jeppesen Airway Manual

 

Generic Airlines Inc

Operations Manual

Volume 9

 

Chapter 3: Qualification Standards, B757 Flight Crew

 

 

Rev. # 8

 

Dated 10/25/07

 

6.1.2 Perform One Engine Inoperative CAT I ILS Approach

 

 

 

Duty Position: All

 

 

 

Criticality: Yes

Currency: No

 

 

Curricula: Qualification & Continuing Qualification

 

 

 

Performance Statement:

The crew will complete the appropriate approach briefing prior to the initial approach fix. Once cleared for the approach, the PF will configure the airplane prior to the FAF per the Generic Airline B767 Precision Approach Profile, so as to be stabilized on the approach by 500 feet below the FAF as outlined in Volume 6 of the Operations Manual. The PF will call for the configuration changes and the PM will perform each action. The landing checklist will be completed during the approach prior to landing. The PM will give the required altitude calls in reference to DH as outlined in Volume 6 of the Operations Manual. When deviations from the standards occur, the PM will give the proper correction call outs as defined in Volume 6 of the Operations Manual. With the adequate visual references in sight, the crew will execute the required calls and responses and transition to a normal landing. If, upon arriving at DH, neither the approach lights nor runway is in sight, then the PM will call for the missed approach. If upon arriving at DH and only the approach light system is visible, the PF may continue the approach down to 100 feet above touchdown zone elevation. If at this point the requirements of FAR 91.175 are not visible, the PF will call for the missed approach and the crew will perform the missed approach procedures. If the runway environment becomes visible prior to 1000 feet above DH, the PM will call the runway in sight and the PF will state “Visual Calls.” The PM will then proceed with stating the appropriate visual approach calls and the visual approach procedures may be applied.

 

Conditions:

IMC * DH +100’

Strong Cross Winds Present * 15K

Contingencies:

*Autopilot Inoperative

 

Standards:

The PF achieves and maintains a stable airspeed, descent rate, vertical flight path, and configuration by 500 feet below DH. Between 1000 feet and 500 feet above DH, only minimum deviations from the standards below occur, and are corrected with the proper calls and responses as defined in Chapter 6, Volume 6 of the Operations Manual). Below 1000 feet above DH in IMC conditions, with any deviations greater than these standards, the crew calls for and executes a missed approach. The following standards apply: Prior to the FAF, the PF maintains desired altitude within +/- 100 ft., desired heading within +/‑ 5 degrees, and desired airspeed within +/-; 10 knots; Inside the FAF or Final Segment, the PF maintains desired Airspeed within +5 or -0 knots, Localizer course within 1/2 dot either side, and Glideslope within 1/2 dot either side. The crew makes all of the required instrument approach calls and responses as detailed in Volume 6 of the Operations Manual. The crew complies with all ATC instructions and clearances or advises ATC if unable. The crew complies with the Generic Airlines Precision Instrument Approach Profile as depicted in Volume 6 of the Operations Manual). The instrument approach procedure is flown correctly and adjusted as necessary based on equipment availability or other factors. The crew immediately initiates a missed approach upon arriving at the DH if the required visual references are not distinctly visible. The landing checklist is completed in a timely manner without errors or omissions. The crew demonstrates strong knowledge of, and makes sound judgments.

Media: Level C FFS

Qualifying Evaluation Event: Maneuvers Validation/Continuing Qualification: Maneuvers Validation

Reference:

Volume 6 of the Operations Manual

Airman’s Information Manual Chapter: 1

Airman’s Information Manual Chapter: 5

Jeppesen Airway Manual

Indicates new/changed information.

Figure 3-96.  Advanced Qualification Program—Documentation Checklist and Review Job Aid

This AQP tool contains seven job aids for both the FAA and the certificate holder to use as simplified checklists for the development and review of the documentation requirements of AQP. Other document configurations may be appropriate to a specific certificate holder. If a certificate holder adopts a different document configuration, other than the one suggested in the current edition of Advisory Circular (AC) 120-54, Advanced Qualification Program, then the applicant should provide clear and specific guidance as to the location of the information for each of these document topics. Certificate holders should include specific references where information can be found that addresses each item. The comment section may be used to record any remarks relative to the review and approval of the document.

There are six document types and one annual report required for each AQP certificate holder. Each document will have its own job aid:

1. Application – One per AQP applicant.

2. Job Task Analysis – One for each trainee type and one for each trainee type’s Instructor/Evaluators.

3. Qualification Standards – One for each trainee type (e.g., pilot, flight attendant (F/A), and aircraft dispatcher) and one for each trainee type’s instructor/evaluators.

4. Instructional Systems Development Methodology – One per AQP applicant.

5. Curriculum Outline – One per curriculum/make, model, series, variant and instructor/evaluator.

6. Implementation and Operations Plan (I&O Plan) – One per AQP applicant.

NOTE: Each of the above documents must remain current throughout the life of the AQP. Each of the documents must utilize a revision control process.

Annual AQP Report – Certificate holders will monitor the status of all AQP curriculums and the performance/proficiency Data Base and will summarize their findings annually in a report to the FAA. Although there is no established format for the report, the associated job aid highlights areas that should be addressed.

Indicates new/changed information.

 

Application, Phase I. The purpose of the application is to establish the applicant’s methodology for developing an AQP for all of its fleets, instructors and evaluators, and for non-fleet-specific curriculum (i.e., indoctrination). The application is submitted once and is updated as information in the application changes (a change in the transition schedule, adding new aircraft, initiating F/A or aircraft dispatcher AQPs, etc.). In order to establish the applicant’s intent and approach for developing an AQP, the application should thoroughly discuss the following topics numbered 1-9 in this job aid.

Indicates new/changed information. Indicates new/changed information. Indicates new/changed information. Indicates new/changed information. Indicates new/changed information. Indicates new/changed information. Indicates new/changed information. Indicates new/changed information. Indicates new/changed information. Indicates new/changed information. Indicates new/changed information.

 

1

Statement of Intent

Y

N

Comments

 

a. Does the Statement of Intent specify the applicant’s intent to develop, implement, and operate an AQP?

 

 

 

 

b. Does the Statement of Intent address all fleets?

 

 

 

 

c. Does the Statement of Intent address how and to what extent the AQP will be operated and maintained?

 

 

 

 

d. Does the Statement of Intent address how CRM will be integrated and measured?

 

 

 

 

 

 

 

 

2

The Applicant’s Staff Organization

Y

N

Comments

 

a. AQP Coordination: Is a person identified that will be the focal point for the applicant’s AQP development and contact with the FAA?

 

 

 

 

b. Subject Matter Expertise (SME): Are current and qualified individuals identified by name or position, who have varying levels of expertise that fairly represent the population of professionals the AQP will address?

 

 

 

 

c. Document and Curriculum Development: Are individual(s) identified by name or position, who interface with the AQP coordinator and the SMEs to develop the requisite AQP process, curriculum, and instructor/evaluator documents?

 

 

 

 

d. Document Management: Is an individual identified, who ensures AQP document control and congruence with FAA approvals?

 

 

 

 

e. Computer Specialist/Database Management: Is an individual identified who will develop and manage the performance/proficiency data acquisition and analysis system?

 

 

 

 

f. In addition, will the computer specialist/database manager be used for other computer-related issues related to the facilitation of an AQP, such as electronic document review?

 

 

 

 

 

 

 

 

3

Data Collection, Submission, and Analysis Reporting

Y

N

Comments

 

a. Does the applicant acknowledge their understanding and acceptance of the AQP performance data requirements by stating the intended purpose for the collection, management, analysis, and reporting of AQP training/evaluation data for each curriculum?

 

 

 

 

b. Does the applicant defer to the I&O Plan for describing the process and methodology for AQP data collection and analysis?

 

 

 

 

c. Does the applicant acknowledge that an electronic data management system will be developed prior to entering Phase III of any AQP curriculum?

 

 

 

 

d. Does the applicant acknowledge that requirement for submitting de-identified data to the FAA no later than 2 months after collecting the data?

 

 

 

 

e. Does the applicant acknowledge the requirement of a more stringent collection and analysis of the data than that submitted to the FAA?

 

 

 

 

f. Does the applicant describe the purpose of the data analysis and how it will be used?

 

 

 

 

g. Does the applicant acknowledge the requirement for submitting an annual AQP report summarizing their data analysis and any resulting changes that ensued in their AQP?

 

 

 

 

 

 

 

 

4

Supporting Documents and Manuals

Y

N

Comments

 

a. Do the supporting documents and manuals list each make, model, and series aircraft or variant?

 

 

 

 

b. Has the applicant provided the FAA with the following documents or manuals:

 

 

 

 

1. A current listing of company and manufacturers manuals that governs company operations?

 

 

 

 

2. General descriptive summary of each aircraft type, including aircraft configuration and the performance baseline?

 

 

 

 

3. Flight Operations Manual (FOM)?

 

 

 

 

4. General Operations Manual (GOM)?

 

 

 

 

5. Manufacturer’s Airplane Flight Manual (AFM)?

 

 

 

 

6. Minimum Equipment List/Configuration Deviation List (MEL/CDL)?

 

 

 

 

c. Does the applicant have available the training and qualification recommendations in the FSB Reports?

 

 

 

 

 

 

 

 

5

Operating Environment Description

Y

N

Comments

 

a. Does the applicant describe the operating environment, including the general meteorological and geographic factors expected to be encountered during operations?

 

 

 

 

b. Does this description include the weather norms and extremes expected to be encountered in operations?

 

 

 

 

c. Does this description include normal, abnormal, and emergency equipment operation in geographic areas that require special procedures (e.g., engine failures in mountainous terrain)?

 

 

 

 

d. Does this description include terminal and en route operating areas such as controlled and uncontrolled airfields?

 

 

 

 

 

 

 

 

6

Trainee Demographics

Y

N

Comments

 

a. Does the applicant provide a general summary of trainee experience and entry level by aircraft make, model, series or variant?

 

 

 

 

b. Does the applicant identify entry requirements for ground and flight instructors and evaluators?

 

 

 

 

c. Does the applicant group students in terms of previous experience (e.g., with high, low, and mean experience included)?

 

 

 

 

d. Does the applicant identify the current and anticipated need for replacement crewmembers by duty position? This information is necessary to determine priority on curriculum development.

 

 

 

 

 

 

 

 

7

FSTDs and Training Equipment—Description and Location

Y

N

Comments

 

a. Does the applicant identify the FSTDs and training equipment to be used, its location, and identify the organization (vendor or applicant) responsible for its security and maintenance?

 

 

 

 

b. Does the applicant identify FSTDs by make, model, serial number, and/or FAA identification number?

 

 

 

 

 

 

 

 

8

Facilities Description

Y

N

Comments

 

a. Does the applicant describe the location, general type of facility, classrooms, training aids, course software, and other resources to be used to support AQP training?

 

 

 

 

 

 

 

 

9

Master AQP Transition Schedule (MATS)

Y

N

Comments

 

a. Does the MATS include all aircraft, crewmembers, instructors, evaluators, and other personnel that the applicant intends to transition to AQP?

 

 

 

 

b. Is the MATS complete? A partial MATS is not acceptable.

 

 

 

 

c. Does the MATS address how currently qualified personnel may transition between traditional recurrent training and continuing qualification curriculums?

 

 

A simple spreadsheet, provided by the applicant, may be helpful to highlight the crew pairing requirements during the first year of AQP.

 

d. Does the MATS address how personnel who have completed initial, transition, conversion, or upgrade curricula may enter a continuing qualification curriculum?

 

 

 

 

e. Does the MATS address personnel who have completed a traditional basic indoctrination curriculum, but have not completed an initial, transition, conversion, or upgrade curriculum?

 

 

 

 

f. Does the MATS address personnel who are current instructors or evaluators and how they may transition to AQP via a differences course?

 

 

 

 

g. Does the MATS address the incremental implementation of the curriculums as opposed to all at once?

 

 

 

 

h. Does the MATS provide the timeframe necessary to withdraw from AQP if it becomes necessary to revert to the traditional 14 CFR part 121 or 135 training and qualification program?

 

 

 

 

 

 

 

 

 

Job Task Analysis (JTA), Phase II. A JTA is the method or procedure used to reduce a unit of work to its base components. The JTA provides a detailed, sequential listing of tasks, subtasks, and elements with the knowledge, skill, and attitude characteristics (KSAs) that clearly define and completely describe the job. An applicant will provide a JTA for each make, model, and series aircraft (or variant). These may be submitted as individual listings, or a single higher level listing with appendices for each aircraft, showing its unique lower level features. As a complete document, the JTA has four components: a job task list, a learning analysis (KSA), identified crew positions, and references.

Indicates new/changed information. Indicates new/changed information. Indicates new/changed information. Indicates new/changed information.

 

1

JTA Structure

Y

N

Comments

 

a. Does the JTA provide a general introduction explaining the development of the task listing and the subsequent task analysis and how it is to be used to form the basis for the qualification standards, and the AQP curriculums upon which they are built?

 

 

 

 

b. Is the JTA organized using a hierarchical system with the flight phases at the top level, tasks at the next level, component subtasks at the next level, elements at the next level?

 

 

 

 

c. Is the JTA complete with tasks, subtasks, elements, and crew positions?

 

 

 

 

d. Has the applicant completed full development of the JTA to the element level?

 

 

 

 

e. In the JTA applicable knowledge, skills, CRM markers, and (where desired) attitudes are applied at the element level?

 

 

 

 

 

 

 

 

2

Flightcrew Member Training Requirements

Y

N

Comments

 

Does the JTA incorporate all knowledge and skill requirements currently specified in the regulations?

 

 

 

 

a. Aircraft Systems’ Subjects

 

 

 

 

Aircraft General

 

 

 

 

Equipment and Furnishings

 

 

 

 

Emergency Equipment

 

 

 

 

Powerplants

 

 

 

 

Electrical

 

 

 

 

Pneumatic

 

 

 

 

Air-Conditioning and Pressurization

 

 

 

 

APU

 

 

 

 

Hydraulics

 

 

 

 

Landing Gear and Brakes

 

 

 

 

Flight Controls

 

 

 

 

Fuel

 

 

 

 

Communications Equipment

 

 

 

 

Flight Instruments

 

 

 

 

Navigation Equipment

 

 

 

 

Autoflight

 

 

 

 

Warning and Detection Systems (to include TCAS, GPWS/TAWS, and WX Radar)

 

 

 

 

Fire and Overheat Protection

 

 

 

 

Oxygen

 

 

 

 

Aircraft Performance & Limitations

 

 

 

 

MEL/CDL

 

 

 

 

 

 

 

 

 

b. System-Operations Integration Training

Y

N

Comments

 

Preflight Visual Inspection

 

 

 

 

Prestart Checklist and Procedures

 

 

 

 

Powerplant Start

 

 

 

 

Taxi to include lowest visibility allowed by OpSpecs

 

 

 

 

Pre-Takeoff Checks and Procedures

 

 

 

 

Normal Takeoff

 

 

 

 

Crosswind Takeoff With Gusts

 

 

 

 

Instrument Takeoff (Low Visibility)

 

 

 

 

Powerplant Failure on Takeoff (at or near V1)

 

 

 

 

Powerplant Failure After V2

 

 

 

 

Rejected Takeoff

 

 

 

 

Area Departure

 

 

 

 

Cruise Procedures

 

 

 

 

Holding

 

 

 

 

Area Arrival

 

 

 

 

RNAV, RNP, SAAAR

 

 

 

 

Normal ILS

 

 

 

 

Engine-out ILS

 

 

 

 

Autopilot-Coupled ILS—CATII/III

 

 

 

 

Non-precision Approach

 

 

 

 

ILS missed Approach

 

 

 

 

Second Missed Approach

 

 

 

 

Precision Radar Monitored Approaches/Missed

 

 

 

 

Circling Approach

 

 

 

 

No-flap Landing

 

 

 

 

Crosswind Landing with gusts

 

 

 

 

Landing With Engine Failure

 

 

 

 

Landing From Circling Approach

 

 

 

 

Rejected Landing

 

 

 

 

Landing With 50% Power Loss

 

 

 

 

LAHSO

 

 

 

 

Stall Prevention (Approach to Stall)

 

 

 

 

Steep Turns

 

 

 

 

Powerplant Failure

 

 

 

 

Windshear Training

 

 

 

 

Mis-Trim Situations

 

 

 

 

Selected Events ‑ Unusual Attitudes

 

 

 

 

TCAS and GPWS ‑ Escape

 

 

 

 

Normal and Abnormal Procedures

 

 

 

 

Emergency Procedures

 

 

 

 

 

 

 

 

 

c. Company Operations Manual Content

Y

N

Comments

 

Company Policy or Procedures (dispatch and flight release requirements)

 

 

 

 

Regulations, Operations Specifications, and Standards Operating Procedures (SOP)

 

 

 

 

Weather Requirements (seasonal changes, flight into various geographic locations and temperature-related requirements)

 

 

 

 

Hazardous Material

 

 

 

 

Security

 

 

 

 

Special Operations (special airports, special approaches, and departures)

 

 

 

 

Emergency crew assigned duties and procedures

 

 

 

 

Operation of emergency equipment/systems

 

 

 

 

Operation of ditching/evacuation equipment/systems

 

 

 

 

Crew Resource Management (CRM)

 

 

 

 

Leadership and Command

 

 

 

 

Mentoring

 

 

 

 

Emergency Situation Training – Rapid Decompression, Fire (Flight/Surface), and Smoke Control Procedures

 

 

 

 

Assistance of Persons to Exits during Emergency

 

 

 

 

Illness, Injury, or Other Abnormal Situations Involving Passengers or Crew (use of medical kit)

 

 

 

 

Flight Physiology (i.e., Hypoxia or Respiration)

 

 

 

 

Use of Checklist (SOP)

 

 

 

 

Flightdeck Familiarization

 

 

 

 

Preflight Planning and FMS

 

 

 

 

In-flight Planning LNAV, VNAV, RNAV, and GPS

 

 

 

 

Required Navigation Procedures/SAAAR

 

 

 

 

Navigation Systems Integration

 

 

 

 

Autoflight and Flight Director Integration

 

 

 

 

Use of Radar/CRTs

 

 

 

 

TCAS

 

 

 

 

GPWS/TAWS

 

 

 

 

Communication Systems Integration (ACARS/FMS/CPDLC)

 

 

 

 

Surface Movement Guidance Systems (SMGS)

 

 

 

 

Runway Incursion Prevention Strategy

 

 

 

 

Stabilized Approach Strategy

 

 

 

 

Precision Radar Monitoring (PRM) Procedures

 

 

 

 

Land and Hold Short (LAHSO) Procedures

 

 

 

 

CAT II/III

 

 

 

Indicates new/changed information.

 

Qualification Standards (QS), Phase II. The qualification standards document has four parts, the prologue, a regulatory comparison, the evaluation/remediation methodology, and the specific Terminal Proficiency Objectives (TPO)/Supporting Proficiency Objectives (SPO) qualification standard.

1. Prologue: This is an introductory section that explains the methodology, format, and terminology of the document.

2. Regulatory Requirements Comparison: The qualification standards document must also include regulatory comparison information. The comparison must meet the requirement of AQP regulatory guidance, which states that the AQP “...must indicate specifically the requirements of part 61, 63, 65, 121, or 135, as applicable, that would be replaced by an AQP curriculum.” The comparison should be comprehensive and understandable so that a reader can discern the scope and appropriateness of the training.

3. Testing/Validation/Evaluation & Remediation Methodology: This section is a detailed plan describing the point in the curriculum when a test, validation, or evaluation will be applied. It must identify what constitutes a failure and/or unsatisfactory performance. This section will also describe the remediation strategy to be used to correct unsatisfactory performance and special tracking provisions.

4. The Qualification Standard: The qualification standard is constructed by applying a performance statement, conditions, and standards to a task or subtask, thereby creating a TPO or an SPO.

Indicates new/changed information. Indicates new/changed information. Indicates new/changed information.

 

1

Qualification Standards Prologue

Y

N

Comments

 

a. Does the prologue discuss the methodology used to develop the qualification standards document?

 

 

 

 

b. Does the methodology explain how any aspect, from curriculum outline to lesson elements or gradesheet items, is be traceable to an item in the qualification standard?

 

 

 

 

c. Does the prologue discuss the format (structure) that used for the qualifications standards?

 

 

 

 

d. Does the prologue define terms used for the qualification standards?

 

 

 

 

 

 

 

 

2

Regulatory Comparison

Y

N

Comments

 

a. Does the regulatory comparison specify the requirements of the applicable Airman Certification Standards or practical test standards and part 61, 63, 65, 121, or 135 that would be replaced by an AQP curriculum and how they will be addressed?

 

 

 

 

b. Are departures from those requirements identified and justified?

 

 

 

 

c. Are any standards specifications used that differ from applicable Airman Certification Standards or practical test standards?

 

 

 

 

 

 

 

 

3

Testing/Validation/Evaluation & Remediation Methodology

Y

N

Comments

 

a. Does the applicant describe where to ascertain how, when, where, and who will assess a student’s proficiency on each terminal and supporting objective?

 

 

 

 

b. Does this section identify the points in the curriculum where the testing, validation, or evaluation will be applied?

 

 

 

 

Systems Validation

 

 

 

 

Procedures Validation

 

 

 

 

Maneuvers Validation

 

 

 

 

LOE

 

 

 

 

Line Checks

 

 

 

 

c. Does the applicant clearly define the different strategies that will be used to test, validate, or evaluate performance?

 

 

 

 

First Look

 

 

 

 

Train to Proficiency

 

 

 

 

Systems/Knowledge Validation

 

 

 

 

Procedures Validation

 

 

 

 

Maneuvers Validation

 

 

 

 

Line Operational Evaluation

 

 

 

 

Initial Line Check

 

 

 

 

Line Check

 

 

 

 

d. Does this section describe how the criticality and currency ratings translate into testing strategies for TPOs and SPOs in the continuing qualification cycle?

 

 

 

 

e. Does this section describe how a TPO with several SPOs may be alternatively sampled over multiple evaluation periods or continuing qualification cycles? E.g., TPO (Non-Precision Approaches) SPOs (NDB, VOR, BC, etc.)

 

 

 

 

f. Does the applicant specify and clearly describe the rating scale that will be used by instructors/evaluators to score performance?

 

 

 

 

g. Do the rating scale definitions clearly discriminate performance levels? Are they clear?

 

 

 

 

h. Does the applicant identify what constitutes a failure and/or unsatisfactory performance for each validation/evaluation point?

 

 

 

 

i. Does the applicant specify the strategy for remedying unsatisfactory performance?

 

 

 

 

j. Does this remediation strategy detail when and what may be repeated and whether or not additional training is warranted?

 

 

 

 

k. Does the applicant describe the methodology that will be used to remediate unsuccessful testing, validation, or evaluation sessions?

 

 

 

 

l. Does the applicant describe the level of FSTDs or aircraft to be used to evaluate the proficiency objective at each point in the curriculum?

 

 

 

 

m. Does the remediation strategy specify when no more training will be offered to the individual and the resulting actions such as “Referred to Director of Training,” “returned to previous position,” etc.?

 

 

 

 

n. Does the remediation strategy describe the criteria for placing an individual on special tracking?

 

 

 

 

o. Does the remediation strategy describe the strategy that will be used for an individual on special tracking?

 

 

 

 

p. Does the remediation strategy describe what must take place for an individual to be removed from special tracking?

 

 

 

 

 

 

 

 

4

Qualification Standards

Y

N

Comments

 

a. Does each individual qualification standard contain the following:

 

 

 

 

A header identifying the certificate holder and the document?

 

 

 

 

Page revision control dates and revision numbers?

 

 

 

 

Consecutive page numbers?

 

 

 

 

Phase of Operations: Number and title from task listing?

 

 

 

 

Qualification Standard Title: Either TPOs or SPOs?

 

 

 

 

Task or Subtask: Number and title from task listing?

 

 

 

 

Crew duty positions?

 

 

 

 

Criticality/Currency Rating: From the task factors analysis?

 

 

 

 

Curriculum: This field identifies the curriculum(s) in which the task will be trained and evaluated.

 

 

 

 

Evaluation Strategy: The evaluation point for this particular qualification standard: e.g., train to proficiency, procedures validation, maneuvers validation, LOE, or line check?

 

 

 

 

Media: The specific media in which training and/or evaluation will be conducted. For qualification, the media is the lowest media used for final evaluation. For continuing qualification, the media is the range of media used for training.

 

 

 

 

Performance Statement: An expanded statement of expected behavior, which, when executed, will complete the work required for a specific portion of a job?

 

 

 

 

Conditions operational and environmental? Are the specific conditions to be used for the qualification curriculum specified?

 

 

 

 

Contingencies. Are the specific contingencies to be used for the qualification curriculum specified?

 

 

 

 

Maneuver Standards: Are they specific and do they correspond to the standards listed in the applicable Airman Certification Standards or practical test standards?

 

 

 

 

Procedural Standards: They may be specific or general. If they are specific, do they correspond to the standards listed in the manuals listed in the reference block? If general, do they reference information in a document or manual to chapter or section (page numbers are not required)?

 

 

 

 

References: Identify the primary references from which performance statements and associated standards were derived. Cite documents by title and where applicable, chapter or section. Page numbers are not required.

 

 

 

 

b. Are there any operations specifications requirements other than those listed above?

 

 

 

 

 

 

 

 

The Instructional Systems Development Methodology. This document describes the approach to be used by applicants to develop and maintain all Advanced Qualification Program (AQP) curriculums. This document is divided into two sections. The first section, Development Procedures, describes the applicant’s approach for using the Job Task Analyses and Qualification Standards as baseline documents to construct their general training curriculums across all AQP courses. The second section, Line Operational Simulation (LOS) Methodology, describes the approach for developing LOS scenarios.

 

1

Development Procedures

Y

N

Comments

 

a. Is the procedure for allocating TPOs and SPOs to lessons, selecting media and methods, and developing the curriculums described?

 

 

 

 

b. Does the applicant describe how enabling objectives (EO) are developed to support their higher level objectives?

 

 

 

 

c. Does the applicant describe how learning and evaluation activities are developed to support these objectives?

 

 

 

 

d. Does the applicant describe how instructional media and methods to are assigned to objectives?

 

 

 

 

e. Does the applicant describe how objectives are clustered and sequenced into lessons, modules, segments, and curriculums?

 

 

 

 

f. Does the applicant describe how an audit trail will be maintained to link proficiency objectives, lesson activities/content, and test items?

 

 

 

 

 

 

 

 

2

Line Operational Simulation (LOS) Methodology

Y

N

Comments

 

a. Does the applicant describe how the typical scenario is constructed?

 

 

 

 

b. Does the applicant describe how each event set relates to a phase of operation?

 

 

 

 

c. Does the applicant describe how each event set consists of a series of proficiency objectives that include both technical and CRM activities?

 

 

 

 

d. Does the applicant describe the use of event conditions, triggers, and distracters, as well as supporting events?

 

 

 

 

e. Does the applicant identify possible sources of incidents that will elicit the behavior required by the proficiency objectives selected for the scenario?

 

 

 

 

f. Does the applicant define the basic success criteria for the LOS, and each event set within it?

 

 

 

 

g. Does the applicant describe the scenario development process?

 

 

 

 

Drafting - who will do the work?

 

 

 

 

Use of gradesheets?

 

 

 

 

Testing - who will be involved?

 

 

 

 

Training instructors/evaluators to administer LOS scenario?

 

 

 

 

The Curriculum Outline. This is a listing of course content. It should be arranged from curriculum into segments, segments into modules, modules into lessons, and lessons into elements. Each part of the curriculum outline must clearly indicate the subject matter to be taught and correspond directly to the hierarchical system of the task analysis. A curriculum outline provides the basis for the curriculum footprint, which is a high-level graphical overview of the curriculum content depicting the training and evaluation activities and the proposed hours for each day of the curriculum.

Indicates new/changed information. Indicates new/changed information. Indicates new/changed information.

 

1

Curriculum Outline

Y

N

Comments

 

a. Does the certificate holder have indoctrination, qualification, and continuing qualification curriculums for each duty position in each make, model, and series/variant aircraft?

 

 

 

 

b. Does the certificate holder have separate indoctrination, qualification, and continuing qualification curriculums for the instructors and evaluators?

 

 

 

 

c. Does the certificate holder have any special curriculums (transition, conversion, upgrade, requalification or refresher)?

 

 

 

 

d. Is each curriculum constructed in the following order: curriculum, segment, module, lesson, and lesson element?

 

 

 

 

e. Does the curriculum outline provide a level of detail that will allow the AQP applicant to make changes to the syllabus without submitting a new document for each syllabus change?

 

 

 

 

f. Does the curriculum outline include:

 

 

 

 

Operator’s Name?

 

 

 

 

Type of Aircraft?

 

 

 

 

Duty Position(s)?

 

 

 

 

Title of curriculum and/or curriculum segment?

 

 

 

 

A listing of numbered (coded) objectives organized into lessons, modules, and segments?

 

 

Numbers (codes) must allow the FAA to track objectives back to qualification standards and Job Task Analysis.

 

An outline of each training module within each curriculum segment?

 

 

Each module should contain sufficient detail to ensure that the main features of the principal elements or events will be addressed during instruction.

 

The checking and qualification modules of the qualification curriculum segment used to determine successful course completion, including any regulation qualification requirements for crewmembers to serve in part 121 or part 135 operations (such as operations familiarization (pilots), operating experience (OE), line checks, operating familiarization (aircraft dispatchers))?

 

 

 

 

g. Does the curriculum outline indicate that it is part of the revision control system by page format?

 

 

 

 

h. Do the curriculum outlines provide a hierarchical link (proficiency objectives) between the qualification standards and a curriculum?

 

 

 

 

i. Does each part of the curriculum outline clearly indicate the subject matter to be taught and correspond directly to the hierarchical numbering system of the task analysis?

 

 

 

 

 

 

 

 

2

Curriculum Footprint

Y

N

Comments

 

a. Does the curriculum footprint describe the training and evaluation activities conducted each day of the curriculum?

 

 

 

 

b. Does the curriculum footprint include the planned hours?

 

 

 

Indicates new/changed information. Indicates new/changed information.

 

Implementation and Operations Plan (I&O Plan). This document is a milestone schedule detailing the transition to an AQP for crewmembers, aircraft dispatchers, instructors, evaluators, and other operations personnel and a blueprint describing provisions for maintenance, administration, data management, and continuing quality control of curriculums. The I&O Plan can be sectioned into two parts. The first part spells out how the certificate holder proposes to implement the AQP. Included in this proposal is the schedule for the Phase III training evaluation to include instructor/evaluator training and small group tryouts. It should also include provisions for evaluating the effectiveness of performance measurement tools, and provisions for evaluating facilities, courseware, and equipment before starting the plans for the small group tryouts. The second part explains how the certificate holder intends to operate the AQP in Phases IV and V. Included in this section are strategies for maintaining the program, crew pairing policy, First Look administration, and instructor/evaluator requirements. The operations plan should also describe in detail the data management plan. This plan includes a statement of understanding addressing the collection and analysis of performance/proficiency data and a description of the Performance Proficiency Database (PPDB), the data management collection process, and the FAA data submission, analysis, and reporting requirements.

Indicates new/changed information. Indicates new/changed information. Indicates new/changed information. Indicates new/changed information. Indicates new/changed information.

 

1

Implementation – Phase III

Y

N

Comments

 

a. Does this section include schedules for the implementation of the each of the AQP curriculums?

 

 

 

 

b. Do the schedules correlate to the MATS?

 

 

 

 

c. Do the schedules include dates for training the instructor/evaluators?

 

 

 

 

d. Does the instructor/evaluator training include difference training for those previously qualified (if applicable)?

 

 

 

 

e. Does this section include provisions for evaluating facilities, courseware, FSTDs, and training equipment before starting the Phase III training?

 

 

 

 

f. Does this section describe a plan for evaluating and observing the instructors and evaluators during the Phase III small group tryout?

 

 

 

 

g. Does the instructor/evaluator training include provisions for evaluating the effectiveness performance measurements such as the application of the rating scale, use of gradesheets, and student feedback?

 

 

 

 

h. Does this section indicate that the applicant will request no-jeopardy credit for the small group tryout course graduates in a separate letter addressed to the appropriate Safety Standards office and the POI?

 

 

 

 

 

 

 

 

2

Operations Phases IV & V—AQP Maintenance

Y

N

Comments

 

a. Does this section describe the methodology for maintaining control of the AQP approval documents?

 

 

 

 

b. Does this methodology include a procedure for providing document copies to the POI and the appropriate Safety Standards office after the receiving the approval stamp from the POI?

 

 

 

 

c. Does this section describe the strategy to be employed for curriculum maintenance and update?

 

 

 

 

d. Does the strategy for curriculum maintenance and update include plans for acquiring and measuring data for tracking curricula?

 

 

 

 

e. Does the strategy for curriculum maintenance and update identify the person(s) responsible making changes in the AQP?

 

 

 

 

f. Does this section describe the strategy for monitoring and responding to demographic changes?

 

 

 

 

g. Does this section describe using training and evaluation feedback to maintain and improve the AQP?

 

 

 

 

h. Does this section describe how student and instructor feedback will be obtained?

 

 

 

 

i. Does this section describe any plans to upgrade training equipment?

 

 

 

 

 

 

 

 

3

Operations Phases IV & V—First Look Maneuvers Administration

Y

N

Comments

 

a. Does the applicant define First Look maneuvers, their purpose, and the strategy that will be used to administer them?

 

 

 

 

b. Does this strategy indicate who will administer the First Look maneuvers and at what point in the curriculum?

 

 

 

 

c. Does this strategy state that First Look will not be briefed prior to the first execution of these items?

 

 

 

 

d. Does this section describe how First Look maneuvers are selected?

 

 

 

 

e. Does this section describe how the First Look maneuvers would be updated?

 

 

 

 

f. Does the applicant describe how First Look maneuvers will be analyzed to determine trends of degraded proficiency?

 

 

 

 

 

 

 

 

4

Operations Phases IV & V—LOFT/LOE Crew Scheduling and Pairing Policy

Y

N

Comments

 

a. Does this section describe the circumstances that would require a seat substitute?

 

 

 

 

b. Does this section describe the decision rules that would apply in obtaining a seat substitute?

 

 

 

 

c. Do the decision rules that would apply in obtaining a seat substitute ensure that in all cases, the seat substitute must be task familiar with the duty position?

 

 

 

 

d. Does this section describe at what point in the curriculum that a seat substitute would be used?

 

 

 

 

e. Does the applicant acknowledge that all occurrences of seat position substitution including the qualification of the seat substitute must be reported?

 

 

 

 

 

 

 

 

5

Operations Phases IV & V—Instructor/Evaluator Requirements

Y

N

Comments

 

a. Does the applicant identify by title each instructor or evaluator position?

 

 

 

 

b. Does the applicant describe the job function(s) that each instructor or evaluator is authorized to perform?

 

 

 

 

 

 

 

 

6

Data Plan

Y

N

Comments

 

a. Does the data plan have a prologue that establishes the intended purpose and methods for the collection, management, analysis, and reporting of AQP training/evaluation data for each curriculum?

 

 

 

 

b. Does the prologue specify how the data plan will be maintained and updated?

 

 

 

 

c. Does the prologue acknowledge the certificate holder’s responsibility to collect and analyze more data than is required to be submitted to the FAA in order to adequately identify performance trends and make changes to factors that impact the crewmember or aircraft dispatcher performance?

 

 

 

 

d. Does the data plan address the methods (e.g., gradesheets or computer-input screens) used to collect performance/proficiency data for all curriculums?

 

 

 

 

e. Does the data plan address data input quality control, security, and usability?

 

 

 

 

f. Does the data plan address data management as the means and strategy the certificate holder intends to employ to store, access, and assimilate the AQP performance/proficiency data that is collected?

 

 

 

 

g. Does the data plan address the type of software the data management system employs (e.g., relational database or spreadsheet), the organization of the information in the electronic medium (e.g., database definition, database table relationships, or spreadsheet description) and a description of the user interface to this data management system?

 

 

 

 

h. Does the data plan address the type of analysis it will employ to facilitate the AQP performance information needs of the certificate holder and the FAA? This discussion of the data analysis must address how each type of AQP data will be analyzed.

 

 

 

 

i. Does the data plan address the FAA data submission requirements including format and frequency?

 

 

 

 

j. Does the data plan address the type of data format it will employ for the reports, (e.g., tabular reports, graphs)?

 

 

 

 

k. Does the data plan address the frequency of the reports, both internal and FAA?

 

 

 

 

l. Does the data plan identify the internal certificate holder personnel that will receive the reports?

 

 

 

 

m. Does the data plan include copies of all forms used for data acquisition and grading?

 

 

 

 

n. Does the data plan include a database description of data field types and a graphical depiction of the Database table relationships?

 

 

 

 

o. Does the data plan address a quality assurance strategy for ensuring data integrity? Does this strategy include instructor and evaluator grading calibration?

 

 

 

Indicates new/changed information. Indicates new/changed information. Indicates new/changed information.

 

Annual AQP Report – Phases IV & V. AQP requires that each certificate holder prepare an annual AQP report for the FAA. This report is based on the certificate holder’s analysis of the data that is collected during training and at strategic points (validation/evaluation gates) in each curriculum and maintained in the Proficiency Performance Database (PPDB). AQP requires data collection and analysis in order to establish and maintain quality control of curricula for crewmembers, aircraft dispatchers, instructors, and evaluators. The annual AQP report should summarize the lessons learned and adjustments made to the curriculum(s) during the reporting period. The report should also include projected or proposed changes to the curriculum(s) based on the certificate holder’s current analysis. The actual adjustments made to the AQP are reflected in revisions to the approved AQP documents. The report should be submitted to the FAA no later than 60 days past the end of the report period. The reporting period is usually based on the approval date for a particular curriculum in either Phase IV or V. During AQP development, particularly for multiple fleet operators, with different approval dates for multiple curriculums, the reporting period may be modified as agreed upon by the FAA and the certificate holder. Copies of the report should be distributed the Principal Operations Inspector (POI) and the appropriate Safety Standards office at least 2 weeks prior to the annual AQP review meeting.

 

Annual AQP Report

Y

N

Comments

 

a. Is there a prologue or introduction to the report that addresses how each type of AQP data was analyzed? This prologue should corroborate the information in the I&O Plan.

 

 

 

 

b. Does the report discuss data reliability and consistency?

 

 

 

 

c. Does the report summarize the internal quality assurance reports as addressed in the I&O Plan?

 

 

 

 

d. Does the report validate the effectiveness of the AQP with supporting evidence of successful training and evaluation?

 

 

 

 

e. Is the report supported by the FAA analysis of the submitted data?

 

 

 

 

f. Does the report identify any trends, problem areas, and potential deficiencies that could result in decreased proficiency?

 

 

 

 

g. Does the report include a description of corrective measures taken and any resulting changes to curriculums?

 

 

 

 

h. Does the report include any projected corrective measures to be taken and provide rationale for these changes?

 

 

 

 

i. Does the report indicate a need for changes to the AQP maintenance strategy as described in the I&O Plan?

 

 

 

 

j. Does the report indicate a need for changes to the certificate holder’s data plan as described in the I&O Plan?

 

 

 

 

k. Does the report provide an analytical comparison of data between equivalent periods in preceding years?

 

 

 

 

l. Does the report identify any future operational changes that will affect the AQP (operational changes or trainee demographics)?

 

 

 

 

m. Does the report analyze training and evaluation feedback as part of the collected data to determine the effectiveness of the program?

 

 

 

 

n. Does the report analyze on-time completion rates for training curriculums and OE?

 

 

 

 

o. Does the report analyze special tracking rates?

 

 

 

 

p. Does the report analyze instructor/evaluator (I/E) rater reliability training results?

 

 

 

 

q. Does the report analyze instructor comments as part of the collected data to determine the effectiveness of the training program?

 

 

 

 

r. Does the report analyze First Look data?

 

 

 

 

s. Does the report analyze maneuvers data?

 

 

 

 

t. Does the report analyze LOE data by technical topics and CRM elements?

 

 

 

 

u. Does the report analyze line check data (exclusive of OE)?

 

 

 

 

v. Does the report address progress towards Phases III, IV, & V in other fleets (as applicable)?

 

 

 

 

w. Does the report address seat substitution rates?

 

 

 

 

x. Does the report address recordkeeping?

 

 

 

 

y. Does the report address adherence to the I&O Plan?

 

 

 

 

z. Does the report address the validity and usefulness of the qualification standards?

 

 

 

 

aa. Does the report address internal audit or FAA surveillance findings?

 

 

 

 

bb. Does the report address related safety programs (i.e., FOQA/ASAP)?

 

 

 

RESERVED. Paragraphs 3-1501 through 3-1525.