Volume 3 General technical administration
chapter 21 THE ADVANCED QUALIFICATION PROGRAM
Section 2 Safety Assurance System: Surveillance
3-1526 GENERAL. Surveillance activities on approved Advanced Qualification Program
(AQP) certificate holders and training centers must be conducted in accordance
with the guidance provided in this section; Volume 6, Chapter 2; and/or Volume 10, as applicable. See
Volume 3, Chapter 21, Section 4, for
guidance on the approval process. This section is related to Safety Assurance System (SAS) Element 2.1.6 (OP), Advanced Qualification Program.
3-1527 REPORTING ON AQPs. AQP review meeting – Phases IV and V: Extended
Review Team (ERT) activities in this phase will consist of surveillance of AQP operations and analysis of data collection results. Joint ERT and applicant
reviews will be conducted periodically. These meetings will provide both parties the chance to analyze results and discuss program concerns. At the midpoint
of Phase IV, the first annual report will be submitted to the ERT. A final joint review and annual report will be accomplished prior to Phase V approval.
A. Focus Areas for Reviews. Focus areas for these reviews are:
1) Data management:
· Collection problems and fixes.
· Analysis—data reliability/validity/sensitivity.
· Data usefulness.
· Problem areas investigated.
1) Data collection methods/tools.
2) Data analysis methods/tools:
· Review of annual report.
· Continuing qualification.
· Line check.
· Identified trends (positive and negative).
· Corrective measures.
3) Program critique summaries.
5) Do records consistently demonstrate the qualification of crews and instructors/evaluators?
6) Implementation and operation plan adherence.
7) Modifications to the program:
· Due to Performance Proficiency Date Base (PPDB) input.
· Due to other input.
8) Validity and usefulness of qualification standards.
9) AQP maintenance strategy—is the described process working?
· Any changes to the maintenance strategy.
· Currency of Program Audit Database (PADB).
10) Instructor/evaluator programs:
· Inter/referent-rater reliability data.
· Quality assurance (QA) observations summarized.
· Methods to maintain instructor/evaluator standardization.
11) Federal Aviation Administration (FAA) surveillance findings.
12) AQP challenges and difficulties:
· Progress towards Phases III, IV, and V in other fleets.
· Special tracking.
· Seat substitution.
· Line-Oriented Flight Training (LOFT)/Line-Operational Simulation
(LOS) scenarios for crewmembers.
13) Use of information from related programs, if any (e.g., flight operations quality assurance (FOQA) or Aviation Safety Action Program (ASAP).
B. Reports—Annual AQP Report. The AQP requires that each participating certificate holder prepare an annual report for the FAA. This report is
based on the certificate holder’s analysis of the data that is collected during training and at strategic points (such as validation/evaluation gates) in each curriculum
and maintained in the PPDB. The AQP requires data collection and analysis in order to establish and maintain quality control (QC) of curricula for crewmembers,
instructors, and evaluators. The annual AQP report should summarize the lessons learned and adjustments made to the curriculum(s) during the reporting period.
The report should also include projected or proposed changes to the curriculum(s) based on the certificate holder’s current analysis. The actual adjustments made
to the AQP are reflected in revisions to the approved AQP documents. The reporting period is usually based on the approval date for a particular curriculum in
either Phase IV or V. During AQP development, particularly for multiple‑fleet operators, with different approval dates for multiple curricula, the reporting
period may be modified as agreed upon by the FAA and the certificate holder. Once the certificate holder has all its fleets and curricula into Phase V, the
reporting period can be fixed into a particular cycle. Unless requested earlier, copies of the report should be distributed to the principal operations inspector
(POI) and the Air Carrier Training Systems and Voluntary Safety Programs Branch (AFS-280) at least 2 weeks prior to the annual AQP review meeting. (See
Volume 3, Chapter 21, Section 1,
Figure 3‑96, Advanced Qualification Program—Documentation Checklist and Review Job Aid.)
3-1528 AQP-SPECIFIC TYPES OF REPORTING AND INSPECTIONS. Guidance
for inspectors that will be using the unique Program Tracking and Reporting Subsystem (PTRS) activity codes associated with AQP programs.
NOTE: Evaluation of the overall AQP program will be accomplished in SAS Element 2.1.6.
A. Airman Certification. It is understood that, except for the Line Operational Evaluation (LOE), the approved test and evaluation strategy
will vary between different AQP training programs. The activity codes shown cover all potential evaluation strategies, and a code should be chosen that
most closely aligns with the certificate holder’s validation point. Although the LOE is the formal certification test, each time the inspector performs a
qualification/continuing qualification validation point, an appropriate record must be entered.
1) Activity Code 1545 for
System (knowledge) validation performed by the inspector in accordance with the approved AQP syllabus and qualification standards.
2) Activity Code 1546 for
PROCEDURES. Procedures validation performed by the inspector in a flight simulation
training device (FSTD). The evaluation can include ground, normal/abnormal/emergency, flight, autoflight procedures, and profiles. If this validation point is combined
with the maneuver validation, use the maneuver validation activity code 1547. The evaluation is conducted in accordance with the approved AQP syllabus and
3) Activity Code 1547 for
MANEUVERS. Maneuver validation performed by the inspector in an FSTD or aircraft. The
evaluation emphasis is on technical flying skills and maneuver proficiency conducted in accordance with the approved AQP syllabus and qualification standards.
4) Activity Code 1548 for
LOEs performed or supervised in an FSTD by an inspector to check individual and crew
competence. Conducted in accordance with the approved AQP syllabus and qualification standards using the approved LOE scenario.
5) Activity Code 1549 for
CHECK. Line check performed by a qualified inspector.
B. Surveillance Activity Codes (1600). These unique AQP surveillance activities/codes provide the FAA with the data and information to confirm
the carrier’s AQP data analysis. The instructor/evaluator-administered validation points may differ between carriers. The activity codes shown cover all potential
evaluation strategies. Use the code that most closely aligns with the approved test and evaluation strategy for both qualification and continuing qualification AQPs.
1) Activity Code 1674 for
EVLTR/KNOWLDG. Surveillance of AQP instructor/evaluator knowledge validation.
2) Activity Code 1675 for
EVLTR/PROCS. Surveillance of AQP instructor/evaluator procedures validation.
3) Activity Code 1676 for
EVLTR/MAN VAL. Surveillance of AQP instructor/evaluator maneuver validation.
4) Activity Code 1677 for
EVLTR/LOE. Surveillance of AQP instructor/evaluator LOE.
5) Activity Code 1678 for
EVLTR/LINE CHK. Surveillance of AQP instructor/evaluator line check.
6) Activity Code 1679 for
EVLTR/LOFT SPOT. Surveillance
of AQP instructor/evaluator LOFT or Special Purposed Operational Training (SPOT).
RESERVED. Paragraphs 3-1529 through 3-1545.