8900.1 CHG 0

Volume 3  General technical administration


Title: Change Bar - Description: Indicates new/changed information.

Section 2  Safety Assurance System: Surveillance

Title: Change Bar - Description: Indicates new/changed information.

3-1526    GENERAL. Surveillance activities on approved Advanced Qualification Program (AQP) certificate holders and training centers must be conducted in accordance with the guidance provided in this section; Volume 6, Chapter 2; and/or Volume 10, as applicable. See Volume 3, Chapter 21, Section 4, for guidance on the approval process. This section is related to Safety Assurance System (SAS) Element 2.1.6 (OP), Advanced Qualification Program.

3-1527    REPORTING ON AQPs. AQP review meeting – Phases IV and V: Extended Review Team (ERT) activities in this phase will consist of surveillance of AQP operations and analysis of data collection results. Joint ERT and applicant reviews will be conducted periodically. These meetings will provide both parties the chance to analyze results and discuss program concerns. At the midpoint of Phase IV, the first annual report will be submitted to the ERT. A final joint review and annual report will be accomplished prior to Phase V approval.

A.    Focus Areas for Reviews. Focus areas for these reviews are:

1)    Data management:

·    Collection problems and fixes.

·    Analysis—data reliability/validity/sensitivity.

·    Data usefulness.

·    Problem areas investigated.

1)    Data collection methods/tools.
2)    Data analysis methods/tools:

·    Review of annual report.

·    Qualification.

·    Continuing qualification.

·    Line check.

·    Identified trends (positive and negative).

·    Corrective measures.

3)    Program critique summaries.
4)    Recordkeeping.
5)    Do records consistently demonstrate the qualification of crews and instructors/evaluators?
6)    Implementation and operation plan adherence.
7)    Modifications to the program:

·    Due to Performance Proficiency Date Base (PPDB) input.

·    Due to other input.

·    Demographics.

·    Operational.

8)    Validity and usefulness of qualification standards.
9)    AQP maintenance strategy—is the described process working?

·    Any changes to the maintenance strategy.

·    Currency of Program Audit Database (PADB).

10)    Instructor/evaluator programs:

·    Inter/referent-rater reliability data.

·    Quality assurance (QA) observations summarized.

·    Methods to maintain instructor/evaluator standardization.

11)    Federal Aviation Administration (FAA) surveillance findings.
12)    AQP challenges and difficulties:

·    Progress towards Phases III, IV, and V in other fleets.

·    Special tracking.

·    Seat substitution.

·    Line-Oriented Flight Training (LOFT)/Line-Operational Simulation (LOS) scenarios for crewmembers.

13)    Use of information from related programs, if any (e.g., flight operations quality assurance (FOQA) or Aviation Safety Action Program (ASAP).

B.    Reports—Annual AQP Report. The AQP requires that each participating certificate holder prepare an annual report for the FAA. This report is based on the certificate holder’s analysis of the data that is collected during training and at strategic points (such as validation/evaluation gates) in each curriculum and maintained in the PPDB. The AQP requires data collection and analysis in order to establish and maintain quality control (QC) of curricula for crewmembers, instructors, and evaluators. The annual AQP report should summarize the lessons learned and adjustments made to the curriculum(s) during the reporting period. The report should also include projected or proposed changes to the curriculum(s) based on the certificate holder’s current analysis. The actual adjustments made to the AQP are reflected in revisions to the approved AQP documents. The reporting period is usually based on the approval date for a particular curriculum in either Phase IV or V. During AQP development, particularly for multiple‑fleet operators, with different approval dates for multiple curricula, the reporting period may be modified as agreed upon by the FAA and the certificate holder. Once the certificate holder has all its fleets and curricula into Phase V, the reporting period can be fixed into a particular cycle. Unless requested earlier, copies of the report should be distributed to the principal operations inspector (POI) and the Air Carrier Training Systems and Voluntary Safety Programs Branch (AFS-280) at least 2 weeks prior to the annual AQP review meeting. (See Volume 3, Chapter 21, Section 1, Figure 3‑96, Advanced Qualification Program—Documentation Checklist and Review Job Aid.)

3-1528    AQP-SPECIFIC TYPES OF REPORTING AND INSPECTIONS. Guidance for inspectors that will be using the unique Program Tracking and Reporting Subsystem (PTRS) activity codes associated with AQP programs.

Title: Change Bar - Description: Indicates new/changed information.

NOTE:  Evaluation of the overall AQP program will be accomplished in SAS Element 2.1.6.

A.    Airman Certification. It is understood that, except for the Line Operational Evaluation (LOE), the approved test and evaluation strategy will vary between different AQP training programs. The activity codes shown cover all potential evaluation strategies, and a code should be chosen that most closely aligns with the certificate holder’s validation point. Although the LOE is the formal certification test, each time the inspector performs a qualification/continuing qualification validation point, an appropriate record must be entered.

1)    Activity Code 1545 for 121/135—CERT/AIRM/AQP/EVAL SYS KNOWLDG. System (knowledge) validation performed by the inspector in accordance with the approved AQP syllabus and qualification standards.
2)    Activity Code 1546 for 121/135—CERT/AIRM/AQP/EVAL PROCEDURES. Procedures validation performed by the inspector in a flight simulation training device (FSTD). The evaluation can include ground, normal/abnormal/emergency, flight, autoflight procedures, and profiles. If this validation point is combined with the maneuver validation, use the maneuver validation activity code 1547. The evaluation is conducted in accordance with the approved AQP syllabus and qualification standards.
3)    Activity Code 1547 for 121/135—CERT/AIRM/AQP/VAL MANEUVERS. Maneuver validation performed by the inspector in an FSTD or aircraft. The evaluation emphasis is on technical flying skills and maneuver proficiency conducted in accordance with the approved AQP syllabus and qualification standards.
4)    Activity Code 1548 for 121/135—CERT/AIRM/AQP/LOE. LOEs performed or supervised in an FSTD by an inspector to check individual and crew competence. Conducted in accordance with the approved AQP syllabus and qualification standards using the approved LOE scenario.
5)    Activity Code 1549 for 121/135—CERT/AIRM/AQP/LINE CHECK. Line check performed by a qualified inspector.

B.    Surveillance Activity Codes (1600). These unique AQP surveillance activities/codes provide the FAA with the data and information to confirm the carrier’s AQP data analysis. The instructor/evaluator-administered validation points may differ between carriers. The activity codes shown cover all potential evaluation strategies. Use the code that most closely aligns with the approved test and evaluation strategy for both qualification and continuing qualification AQPs.

1)    Activity Code 1674 for 121/135—SURVL/AQP/INSTR EVLTR/KNOWLDG. Surveillance of AQP instructor/evaluator knowledge validation.
2)    Activity Code 1675 for 121/135—SURVL/AQP/INSTR EVLTR/PROCS. Surveillance of AQP instructor/evaluator procedures validation.
3)    Activity Code 1676 for 121/135—SURVL/AQP/INSTR EVLTR/MAN VAL. Surveillance of AQP instructor/evaluator maneuver validation.
4)    Activity Code 1677 for 121/135—SURVL/AQP/INSTR EVLTR/LOE. Surveillance of AQP instructor/evaluator LOE.
5)    Activity Code 1678 for 121/135—SURVL/AQP/INSTR EVLTR/LINE CHK. Surveillance of AQP instructor/evaluator line check.
Title: Change Bar - Description: Indicates new/changed information.
6)    Activity Code 1679 for 121/135—SURVL/AQP/INSTR EVLTR/LOFT SPOT. Surveillance of AQP instructor/evaluator LOFT or Special Purposed Operational Training (SPOT).

RESERVED. Paragraphs 3-1529 through 3-1545.