Volume 10  Air transportation oversight system

chapter 6  Initial CERTIFICATION PROCESS of Part 121 Air Carriers

Section 2  Certification Process Document

10-435    GENERAL. This section contains the text of the Certification Process Document (CPD) in its entirety. This section also provides direct links to reference material including briefing guides, meeting agendas, training requirements, and other guidance material used during the process.

NOTE:  Title 14 of the Code of Federal Regulations (14 CFR) part 119, § 119.39(b) and (b)(1) states, “An application for a certificate may be denied if the Administrator finds that the applicant is not properly or adequately equipped or is not able to conduct safe operations under this subchapter.” In addition, the applicant must maintain an active status of the project. The certificate‑holding district office (CHDO) must evaluate an inactive period exceeding 90 days. Inactivity may result in the CHDO terminating the certification process or when it is clear that continuation would not result in approval or acceptance (i.e., multiple failures of the applicant submissions).

NOTE:  Observe all guidance in this order regarding Program Tracking and Reporting Subsystem (PTRS) requirements.

Figure 10‑61.  Certification Process Document

PHASE 1        APPLICATION.

1.1       Request for Formal Application Meeting.

1.1.1    Applicant—The applicant contacts the certificate‑holding district office (CHDO) to schedule a formal application meeting date. This request must be at least 45 calendar‑days before the proposed formal application meeting to allow the Federal Aviation Administration (FAA) to prepare resources. The request must contain the items listed on the Preapplication Checklist (PAC) (see Figure 10‑62, Preapplication Checklist) when the application meeting date is requested.

1.1.2    CHDO Manager—Advise the Flight Standards National Field Office (AFS‑900) that an applicant has requested a formal application meeting, and email a copy of the submitted documents to the AFS‑900 Management Team at AFS‑900‑Management‑Team@faa.gov. Ensure the CHDO has assigned a certification project manager (CPM).

1.1.3    CPM—Ensure the regional Flight Standards division (RFSD) obtains a precertification/designator number by contacting the Regulatory Support Division, Aviation Data Systems Branch (AFS‑620) via email at 9‑AMC‑AFS620‑CertInfo@faa.gov (see Volume 2, Chapter 2, Section 2, paragraph 2‑133). Upon issuance of a precertification/designator number by AFS‑620, the new designator will automatically be entered into the enhanced Vital Information Database (eVID).

1.1.4    CPM—Begin the Web‑based Operations Safety System (WebOPSS) process by making an email request for access to AFS‑620 at AFS‑WebOPSS@faa.gov.

1.2       Establish Certification Project Team (CPT).

1.2.1    AFS‑900 Certification and Evaluation Program Office Manager—Assign the AFS‑900 Assistant Manager and team, then forward submitted documents from the CHDO Manager.

1.2.2    Certification Team Leader (CTL)—Provides documents and reviews the applicant’s submissions to become familiar with the applicant’s operation with team members.

1.2.3    CHDO Manager, CPM, and CTL—Identify CPT members. Ensure that the CPT members have completed the baseline training requirements (see Volume 10, Chapter 2, Section 3, paragraph 10‑144). If a data evaluation program manager (DEPM) is not assigned to the project, the CPM reviews the data collected by CPT members. CPM must take FAA Course FAA27100118, ATOS 1.2 for Data Evaluation Program Managers and Data Reviewers, before reviewing any data. Participants who completed the original course, FAA27022, Data Evaluation Program Managers (DEPM), do not have to take the new course.

1.2.4    CTL—Create a Project Management Tool (PMT) record.

1.3       Formal Application Preparation.

1.3.1    CPM—Verify a tentative date for the formal application meeting with the applicant.

1.3.2    CTL—Conduct CHDO briefing (refer to the AFS‑900 Certification and Evaluation Program Office SharePoint site for the appropriate presentation materials).

1.3.3    CPM and CTL—Identify team resources to complete the initial review.

1.3.4    CHDO Manager—Coordinate with AFS‑900 Safety Management System (SMS) Program Office to schedule an SMS CHDO/Applicant briefing.

1.4       CPT Automation Preparation.

1.4.1    CHDO Team Members—Complete the Air Transportation Oversight System (ATOS)/PMT Automation Access Worksheet (see Figure 10‑64, Automation Access Information Worksheet).

1.4.2    CTL—Create a team member roster in the PMT using the completed ATOS/PMT Automation Access Information Worksheet. Include all Front Line Managers (FLM).

1.4.3    Principal Inspectors (PI)—Begin entering data in the eVID.

1.4.4    CPM—Ensure the PIs open the appropriate PTRS codes.

1.4.5    CPM—Ensure the PIs coordinate with the applicant in drafting all proposed operations specifications (OpSpecs) and associated paragraphs in WebOPSS. (See Volume 3, Chapter 18.)

NOTE:  Any element(s) that is selected for initial review which requires an OpSpec and associated paragraph(s) to be completed for appropriate review accomplishment, will have that accomplished and saved as Draft, prior to the review.

1.5       Submission of Certification Documents Prior to Formal Application Meeting.

1.5.1    Applicant—The CPM must ensure the applicant submits all additional PAC documents to the CHDO at a minimum of 15 business days prior to the requested formal application meeting (see Figure 10‑62). The CPT has a minimum of 15 working‑days to review documents after acceptance of the PAC submission.

1.5.2    CPM/CTL—Notify applicant in writing of the acceptance/rejection of the certification documents.

1.5.3    CPM—Contact the Department of Transportation (DOT) Office of the Secretary of Transportation (OST), Air Carrier Fitness Division at 202‑366‑9721, to confirm receipt of application (Office of Aviation Analysis). (See Volume 2, Chapter 2, Section 2 for detailed information on DOT requirements.)

1.6       Initial Review.

1.6.1    CTL—Upload the applicant’s documents to the PMT.

1.6.2    CPT—Complete the initial review process (see paragraph 10‑436, Initial Review Instructions).

1.6.3    CPM and CTL—Provide input to FLMs to help them identify team resources needed to complete each Safety Attribute Inspection (SAI) selected for the initial review.

1.6.4    CTL—Complete the “Assign Initial Work Review” document in PMT.

1.6.5    CHDO and AFS‑900 FLMs—Determine resource availability and assign team coordinators (TC) and team members to support the initial review. CHDO and AFS‑900 FLMs will assign the SAIs selected for this review to the CHDO and AFS‑900 team members.

1.6.6    CPT—Accomplish the assigned SAIs and verify the applicant has completed the self‑audited SAI data (see paragraph 10‑436, Initial Review Instructions).

1.6.7    CPM/CTL—If the current submission is sufficient, proceed to step 1.7.

1.6.8    CPM/CTL—If additional data is necessary or if the current submission is not of sufficient quality to proceed (see paragraph 10‑437), notify the applicant with a written explanation of concerns.

1.6.9    CTL—Post a copy of the correspondence to the PMT.

1.6.10  CPT—Upon receipt of the revised submission, complete Design Assessments (DA) of the originally selected SAIs and three additional SAIs (see paragraph 10‑436).

NOTE:  Repeat step 1.6 until submission is sufficient. For each submission, an additional three SAIs will be added.

1.7       Plan Formal Application Meeting.

1.7.1    CPM/CTL—If the initial review finds that the formal application package is of sufficient quality, continue planning the formal application meeting (see paragraph 10‑438).

1.7.2    CPM/CTL—Confirm that Title 14 of the Code of Federal Regulations (14 CFR) part 119, § 119.65(a) required management personnel will attend. Reschedule the meeting if any required management personnel cannot attend.

1.7.3    CPM and CTL—Determine which CPT members will attend the formal application meeting.

1.8       Notify/Coordinate with Applicable Offices.

1.8.1    CPM—Advise the RFSD of the formal application meeting date.

1.9       Conduct Formal Application Meeting.

1.9.1    CPM/CTL—Circulate the attendance roster (see Figure 10‑65, Attendance Roster).

1.9.2    CPM/CTL—Ensure the accomplishment of meeting agenda items (see paragraph 10‑438).

1.9.3    Applicant—CPM must verify that all required management and personnel with process change authority will be available throughout the certification process.

1.9.4    CPM/CTL—Conclude the formal application meeting.

1.9.5    CPM/CTL—Inform the applicant that they will receive notification within 5 working‑days of acceptance or rejection.

1.10     Accept or Reject the Formal Application Meeting or the Entire Formal Application Package.

1.10.1   CPT—Determine that the applicant’s required management personnel are knowledgeable of their air carrier systems and applicable rules and guidance.

1.10.2   CPT—Discuss areas of concern found during the formal application meeting.

1.10.3   CPM/CTL—Determine whether to accept the application or not.

NOTE:  An unsuccessful evaluation of any or all individuals does not always require a rejection of the application. The CPM will conduct a follow‑up evaluation of the appropriate individual(s). The CPM must complete this follow‑up evaluation with satisfactory results before finishing phase 1. During the certification process, if there is a change to any part 119 required management personnel, the CPT member(s) will conduct an evaluation of that person to ensure the person is knowledgeable of his or her systems and applicable rules and guidance. If there is a change to aircraft make and model, the project can be restarted or terminated.

1.11     Notification to Applicant.

1.11.1   CPM/CTL—Provide either a letter of acceptance or a letter of rejection to the applicant within 5 working‑days of completing the formal application meeting. A letter of rejection will state the reason for rejection of the formal application meeting or the application package. The CHDO manager will determine at which step the applicant will reenter the Certification Process Document (CPD), or whether to terminate the project.

1.12     Phase 1 Review.

1.12.1   CPM/CTL—Verify that the applicant has met all Gate I requirements (see Figure 10‑66, Phases and Gates for Initial Certification).

1.12.2   CTL—Solicit applicant and CPT feedback. Refer to Flight Standards Service (AFS) Stakeholder Feedback. The CTL can also accomplish this by interviewing the CHDO and applicant.

1.12.3   CTL—Make appropriate entries into the PMT Project Summary.

PHASE 2        Design Assessment (DA).

2.1       DA—ATOS Business Process (Modules 1 and 2).

2.1.1    CTL—Ensure that a new air carrier account is generated in the ATOS database.

2.1.2    CPM/CTL—Completes the Air Carrier Oversight Profile (ACOP) (Module 1) by deselecting the non‑applicable elements based on the requested scope of operation. Do not deselect any question at this time.

2.1.3    CPM/CTL—Assemble and review a planning data package from the Safety Performance Analysis System (SPAS) and other sources (Module 2).

2.1.4    CPM/CTL—Ensure the Air Carrier Assessment Tool (ACAT) is completed.

2.1.5    CPM/CTL—Ensure the Comprehensive Assessment Plan (CAP) has been developed to complete all elements. PI instructions for each DA will include at least the following:

·    “Name of Inspector” conducting the inspection and indicate “team coordinator” or “team member”;

·    Elements identified as “R” in Figure 10‑6, ATOS System/Subsystem Element Chart—Airworthiness Elements and Figure 10‑7, ATOS System/Subsystem Element Chart—Operations and Cabin Safety Elements, in Volume 10, Chapter 1, Section 2; and Figure 10‑66, Phases and Gates for Initial Certification, in this section. The PI instructions should direct the team member to complete sections 1 through 5 of that element;

·    For all remaining SAIs that are not identified with an “R,” the PI instructions should state, complete sections 1 and any other Systems Requirements Review (SRR)‑related questions in sections 2 through 5. The CPT can add any other sections as deemed appropriate;

·    Proposed completion date;

·    The “National/Local/Field” block should read “Initial Cert”; and

·    The TC, who will be a CHDO aviation safety inspector (ASI)/PI, on all elements completed by AFS‑900.

2.1.6    CPM—Adjust the DA completion due dates to the quarter that Modules 7 and 8 are scheduled for completion.

NOTE:  The CPT may request additional attribute questions (sections 2 through 5) be completed at any time during the DA phase.

2.1.7    CPM/PIs—Save CAP to final.

2.2       Resource Management (Module 3).

2.2.1    CPM and CTL—Provide input to FLMs to help them identify team resources needed to complete each SAI.

2.2.2    CHDO and AFS‑900 FLMs—Determine resource availability and assign TCs and team members to support the DA plan.

2.2.3    CHDO Manager—Concur with the CAP.

2.3       Initial Cadre Training.

2.3.1    CPT—Review the Initial Cadre Check Airman (ICCA) plan to verify compliance with guidance (see Volume 3, Chapter 20). Accomplish the appropriate ATOS element for this ICCA training, and review all courseware provided by the applicant to ensure it is acceptable.

2.3.2    CPM/CTL—Notify the applicant in writing of the approval of the plan. Include in the letter that additional training may be required if deficiencies are found during the certification process.

2.3.3    CPM/CTL—Retarget the CAP utilizing appropriate Performance Assessments (PA) to assess the performance of ICCA training.

2.3.4    CTL—Upload a copy of the approval letter to the PMT.

2.4       Other Initial Training.

2.4.1    CPT—Review any other initial training requested by the applicant to begin prior to the completion of phase 2. Accomplish the appropriate ATOS element for this training, and review all courseware provided by the applicant to ensure it is acceptable.

2.4.2    CPM/CTL—Coordinate with the responsible CHDO inspector to provide initial approval for selected training programs.

2.4.3    CPM/CTL—Notify the applicant in writing of approval. Include in the letter that additional training may be required if any deficiencies are found during the certification process.

2.4.4    CPM/CTL—Retarget the CAP utilizing appropriate PAs to assess the performance of training.

2.4.5    CTL—Upload a copy of the approval letter to the PMT.

2.5       Data Collection—ATOS Business Process (Module 4).

2.5.1    CPT—Accomplish data collection by using Microsoft Word‑formatted SAIs in lieu of the ATOS database entries.

NOTE:  Use the SAI review document that is located on the Certification SharePoint site.

2.5.2    CPT—Complete the assigned SAIs using work instructions provided by the PIs.

2.5.3    CPT—Verify accuracy of the applicant’s self‑audited SAIs.

2.5.4    CPT—Collect and analyze the data by conducting a data analysis meeting (see paragraph 10‑437, Data Analysis Meeting). If data is of sufficient quality, continue to step 2.6.

2.5.5    CPM/CTL—If the current submission is not of sufficient quality and/or when additional data is necessary, notify the applicant with a written explanation of concerns.

2.5.6    CPM/CTL—If a subsequent submission contains minimal concerns that the PI could approve or accept with the program as is, schedule and conduct a meeting with the CPT and applicant to discuss and correct the remaining discrepancies.

NOTE:  If an ASI observes any unresolved finding(s) not related to his or her assigned element, or he or she cannot capture it in his or her assigned element, document the finding(s) in a Dynamic Observation Report (DOR), and notify the ASI(s) working that element. Element DOR is the preferred choice, when applicable. The “Other” DOR is selected if the finding cannot be addressed in an element DOR.

2.6       Data Reporting—ATOS Business Process (Module 5).

2.6.1    CPT—Once the submission is of sufficient quality, the CPT will start entering data into the database.

2.6.2    CPT—Enter the applicant’s self‑audited SAI data by selecting “Begin New Activity.” Enter “Initial Cert” in the “Local/Regional/National” field on the “Common Data” screen. Begin data reporting by entering the applicant’s self‑audited SAI data for the questions and sections not assigned to the CPT for assessment. Identify this data as “Air Carrier Provided Data.” Save activity 1 to “Final” after entering all of the applicant’s self‑audited SAI data.

2.6.3    CPT—Each team member will then enter their responses into ATOS automation by selecting “Begin New Activity.” Enter “Initial Cert” in the “Local/Regional/National” field on the “Common Data” screen. Save activity 2 to “Final” after completing data entry.

2.6.4    TC—Ensure SAI and DOR data meet the Data Quality Guidelines (DQG) (see Volume 10, Chapter 2, Section 6, Table 10‑1, Data Quality Guidelines—Specific Data Requirements Table, and Table 10‑2, Data Quality Guidelines—Data Dimensions Table).

2.6.5    TC—Save SAI to master record.

2.7       Data Reviewing—ATOS Business Process (Module 6).

2.7.1    CPM/DEPM—Ensure the data meets DQGs (see Volume 10, Chapter 2, Section 6, Tables 10‑1 and 10‑2).

2.7.2    CPM/DEPM—Ensure that all data is saved to the ATOS database.

2.8       Design Analysis and Assessment—ATOS Business Process (Module 7).

2.8.1    CPM/CTL—Conduct a DA meeting with identified members of the CPT to analyze collected data by element.

2.8.2    CPM/CTL—Make a bottom‑line assessment for each element and document on the Assessment Determination and Implementation (ADI) tool.

2.9       Design Action Determination and Implementation—ATOS Business Process (Module 8).

2.9.1    CPM/CTL—Follows the Module 8 business process (see Volume 10, Chapter 2, Section 9) to determine the appropriate course of action for each element.

2.9.2    CPM/CTL—If the entire air carrier system design meets requirements for acceptance or approval without mitigation, accept or approve manuals and programs as applicable.

2.9.3    CPM/CTL—If the FAA accepts the air carrier’s system design with mitigation, accept or approve manuals and programs as applicable, and document the need for additional data collection, monitoring, or mitigation.

2.9.4    CPM/CTL—Notify the applicant that the FAA has approved/accepted all manuals and programs.

2.9.5    CPM/CTL—If the system does not meet design requirements, notify the applicant in writing and repeat steps as necessary.

2.10     Check Status of the Following:

2.10.1   CPM/CTL—Environmental Assessment (EA).

2.10.2   CPM/CTL—Drug and alcohol abatement program.

2.10.3   CPM/CTL—Hazardous materials (hazmat) program.

2.10.4   CPM/CTL—Security program.

2.10.5   CPM/CTL—All required training.

2.10.6   CPM/CTL—Progress of economic authority.

2.10.7   CHDO Manager—Certificate Management Team (CMT) baseline staffing and baseline training requirements (see Volume 10, Chapter 2, Section 3, paragraph 10‑144).

2.11     OpSpecs.

2.11.1   CPM/CTL—Spot-check to verify that the draft OpSpecs match the proposed operation.

2.12     Phase 2 Review.

2.12.1   CPM/CTL—Verify that the applicant submits an updated Schedule of Events (SOE).

2.12.2   CPM and CTL—Verify that the applicant has met all Gate II requirements (see Figure 10‑66).

2.12.3   CTL—Make appropriate entries into the PMT Project Summary.

2.12.4   CTL—Solicit applicant and CPT feedback (refer to AFS Stakeholder Feedback). The CTL can also complete this by interviewing the CHDO and applicant.

PHASE 3        Performance Assessment (PA).

3.1       Confirm Operational Readiness.

3.1.1    CPM/CTL—Ensure the applicant conducts an internal safety assessment of operating systems using company audit procedures and documentation. Submit results to the CPM.

3.1.2    CPM/CTL—Ensure that the applicant has properly addressed all concerns found during the internal safety assessment.

3.2       PA Planning Meeting—ATOS Business Process Module 2.

3.2.1    CPM/CTL—Ensure that the ACAT is up to date using information from the DA.

3.2.2    CPM/CTL—Ensure that the CAP is adjusted and plan PAs that can be observed during the demonstration phase. Include elements that were identified as high priority/risk in the CAP.

NOTE:  To prevent a large number of “not observable” answers, the CPT may use a Constructed Dynamic Observation Report (ConDOR) Data Collection Tool(s) (DCT) in lieu of an Element Performance Inspection(s) (EPI) to target specific element questions.

3.2.3    CPM/CTL—Adjust the PA completion due dates to the quarter that Modules 7 and 8 are scheduled to be completed.

3.2.4    CPM/CTL—Ensure that the work instructions for assigned EPI(s) and/or ConDOR(s) include at least the name of the team member recommended and any other work instructions needed.

3.2.5    CPM/CTL—Ensure data collection requirements for each PA are determined.

3.3       Resource Management—ATOS Business Process Module 3.

3.3.1    CPM and CTL—Provide input to FLMs to help them identify team resources to complete the PA plan.

3.3.2    CHDO and AFS‑900 FLMs—Review the SOE to ensure the availability of resources (e.g., travel funds, passports, and country clearance requests).

3.3.3    CHDO and AFS‑900 FLMs—Determine resource availability and assign team members to support the PA plan.

3.3.4    CHDO Manager—Concur with the CAP.

NOTE:  Keep all EPI records in draft until the end of phase 3.

3.4       Aircraft Conformity Evaluation.

3.4.1    Applicant—CPM ensures the applicant notifies the CHDO of aircraft availability at least 10 business days before the proposed aircraft evaluation and forwards the completed Aircraft Configuration Control Document (ACCD) (or applicant’s equivalent) and aircraft information form to the CPM for the aircraft conformity evaluation (see Volume 10, Chapter 6, Section 3).

3.4.2    CPM/CTL—Provide the applicant a copy of the Aircraft Document Request List (see Volume 10, Chapter 6, Section 3, Figure 10‑70, and for more information see Volume 10, Chapter 6, Section 3).

3.4.3    CPM/CTL—Complete an initial review of the applicant’s conformity submissions to confirm all required records are present and of sufficient quality to continue to the next step.

3.4.4    CPT—Evaluate the applicant’s aircraft conformity process using the ACCD, documentation provided by the applicant, and applicable EPI(s).

3.4.5    CPM/CTL—Notify the applicant in writing of discrepancies identified during the aircraft conformity evaluation.

3.4.6    Applicant—CPM ensures the applicant submits a letter containing the required corrections to the CPM. Update the SOE, if required.

3.4.7    CPM/CTL—Confirm that the applicant corrected the discrepancies prior to beginning proving tests.

3.4.8    CTL—Post the correspondence in the PMT.

3.5       Demonstration Plans.

3.5.1    Applicant—CPM ensures the applicant submits the following plans, as appropriate, for the proposed operation:

·    Emergency evacuation plan,

·    Ditching plan, and

·    Proving test plan.

3.5.2    CPT—Review and accept submitted plans.

·    Emergency evacuation plan and ditching plan (see Volume 3, Chapter 30), and

·    Proving test plan (see paragraph 10‑440 and Volume 3, Chapter 29).

3.5.3    CPM/CTL—If applicable, respond to the applicant request for a reduction of proving test hours (see Volume 3, Chapter 29, Section 7).

3.5.4    Applicant—CPM ensures the applicant submits a request for a letter of authorization (LOA) in accordance with § 119.33(c), for demonstrating appropriate operations under 14 CFR part 121. Make this request at least 10 business days before any aircraft flight demonstration (see paragraph 10‑440 and Volume 3, Chapter 29, Section 7).

3.5.5    CPM—Ensure the draft OpSpecs are completed and match the proposed operation.

3.5.6    CPM/CTL—Issue an LOA in accordance with Volume 3, Chapter 29, Section 4; Volume 3, Chapter 29, Section 7; and § 119.33(c).

3.6       Tabletop Exercise.

3.6.1    CPT—Develop scenarios to assess performance during tabletop exercises (see paragraph 10‑439, Tabletop Exercise).

3.6.2    CPT/Applicant—Conduct tabletop exercises and discuss results of each scenario.

3.6.3    CPM/CTL—If the CPM finds any scenario to be unsatisfactory, he or she should confirm that the applicant has made corrections to its system design or has corrected personnel performance issues before beginning proving tests.

3.7       Evacuation/Ditching Demonstrations (as required).

3.7.1    Aviation Safety Inspector—Cabin Safety (ASI‑CS)—Discuss the demonstration plan with the CPT.

3.7.2    CPM/ASI‑CS—Brief applicant on plan and expectations.

3.7.3    CPT/Applicant—Conduct evacuation/ditching demonstrations.

3.7.4    CPT—Document results of each demonstration on FAA Form 8430‑1, Emergency Evacuation Demonstration Report, applicable EPI(s), and/or ConDORs (see Volume 3, Chapter 30, Section 8).

3.7.5    CPM—Forward each completed FAA Form 8430‑1 to the RFSD.

3.7.6    CPM—Save a copy of each completed FAA Form 8430‑1 to be part of the certification report (step 4.4.1).

3.8       Assess Applicant’s Facilities.

3.8.1    CPT—Assess the following facilities, as applicable, using EPIs:

·    Station facilities (EPI 5.1.5),

·    Maintenance facility/main maintenance base (EPI 1.3.3),

·    Line station facilities (EPI 5.1.1), and

·    Maintenance providers (EPI 1.3.7).

NOTE:  EPI records will remain in draft until proving tests are complete.

NOTE:  The CPT may accomplish these assessments prior to or during proving tests by observing the applicant conduct their own internal safety assessment (step 3.1).

3.9       Prepare for Proving Tests.

3.9.1    CPM—Confirm the “show cause” order has been issued.

3.9.2    CPM/CTL—Verify that the applicant has met all other Gate III requirements (see Figure 10‑66).

3.9.3    CPM/CTL—Conduct pre‑demonstration test meeting with CPT (see Volume 3, Chapter 29).

3.9.4    CPM—Schedule the CPT for proving tests.

3.9.5    CPT—Develop proving test scenarios that will facilitate the collection of data the CPT will use to complete the PA (see paragraph 10‑440 and Volume 3, Chapter 29).

3.10     CPT and Applicant Conduct Proving Tests.

3.10.1   CPT and Applicant—Meet to discuss the proving test protocols and the proposed flight schedule (see paragraph 10‑440 and Volume 3, Chapter 29).

3.10.2   CPT—Prior to each day’s proving tests, conduct an FAA meeting to review the day’s plan.

3.10.3   CPT and Applicant—Prior to each day’s proving tests, conduct a briefing with the applicant to review the day’s plan.

3.10.4   CPT and Applicant—Conduct proving tests.

3.10.5   CPT—Collect data for assigned EPIs or ConDORs.

3.10.6   CPT and Applicant—Debrief results after each flight.

3.10.7   CPT—Document results of each scenario on the appropriate worksheet (see Figure 10‑69, Proving‑Flight Test Scenario Worksheet).

3.10.8   CPM/CTL—When the applicant has met all test objectives and repeatedly demonstrated their ability to conduct line operations in compliance with regulations and safe operating practices, recommend that the proving tests conclude (see Volume 3, Chapter 29).

3.10.9   CPM/CTL—If the applicant has not completed all test objectives or not demonstrated their ability to conduct line operations in compliance with regulations and/or safe operating practices, evaluate the need for an extension or termination of the proving tests (see Volume 3, Chapter 29).

3.10.10   CHDO Manager—Notify the RFSD of the normal completion of proving tests or of proving tests terminated due to unsatisfactory performance. Early completion of proving tests requires RFSD approval (see Volume 3, Chapter 29).

3.10.11   CTL—Upload completed scenarios worksheets to the PMT.

3.11     Data Reporting—ATOS Business Process (Module 5).

3.11.1   CPT—Enter EPI, ConDOR, and DOR data in accordance with DQGs and save activities to final (see Volume 10, Chapter 2, Section 6, Tables 10‑1 and 10‑2).

3.11.2   CPT—Save the EPI to the master record when it is complete.

3.12     Data Review—ATOS Business Process (Module 6).

3.12.1   CPM/DEPM/Data Reviewer—Ensure data meets DQGs and save records to the ATOS database. (see Volume 10, Chapter 2, Section 6, Tables 10‑1 and 10‑2.)

3.13     Performance Analysis and Assessment—ATOS Business Process (Module 7).

3.13.1   CPM/CTL—Along with identified members of the CPT, conduct a PA meeting to analyze EPI collected data by element.

3.13.2   CPM/CTL—Make a bottom‑line assessment and document it on the ADI Tool.

3.14     Performance Action Determination and Implementation—ATOS Business Process (Module 8).

3.14.1   CPM/CTL—Follow the ATOS Module 8 business process (see Volume 10, Chapter 2, Section 11) to determine the appropriate course of action for each element.

NOTE:  The CPM will affirm the air carrier’s system if it meets all performance standards.

3.14.2   CPM/CTL—If the air carrier’s system meets performance standards, but with minor problems, determine whether additional data collection, monitoring, or action is necessary.

3.15     Phase 3 Review.

3.15.1   CTL—Send an email to the Field Support Program Office (FSPO) manager notifying him or her that the certification project is close to completion.

3.15.2   CTL—Make entries into the PMT Project Summary.

3.15.3   CTL—Solicit applicant and CPT feedback (refer to AFS Stakeholder Feedback). The CTL can also complete this by interviewing the CHDO and applicant.

PHASE 4        ADMINISTRATIVE FUNCTIONS.

4.1       Accomplish Administrative Requirements.

4.1.1    CHDO Manager—Confirm the CMT is in place.

4.1.2    CPM—Obtain the final certificate number and to make the air carrier “Active” send an email to AFS-620 at 9‑AMC‑AFS620‑Certinfo@faa.gov (see Volume 2, Chapter 2, Section 2, paragraph 2‑133).

4.1.3    CHDO Manager—Schedule the first annual planning meeting with the CMT, the AFS‑900 FSPO, and appropriate members of the CPT.

4.1.4    CPM—Verify that PIs close all PTRS records in step 1.4.4 with proper entries in the “Miscellaneous” and “Comments” sections.

4.1.5    CTL—Send an email to the AFS-900 FSPO manager notifying him or her of the proposed date for the annual planning meeting.

4.2       Approve OpSpecs.

4.2.1    CPM—Verify the accomplishment of final changes to the eVID and draft OpSpecs. Coordinate with PIs on approval/disapproval of the applicant’s OpSpecs.

4.2.2    PIs—Review any OST/DOT limitations before signing the OpSpecs.

4.2.3    PIs—Sign OpSpecs.

4.3       Issue OpSpecs and Air Carrier Certificate.

4.3.1    CHDO Manager—Review and forward the Air Carrier Certificate and copy of OpSpecs to the RFSD manager.

4.3.2    RFSD Manager—Sign the Air Carrier Certificate and return to the CHDO.

4.3.3    CHDO Manager—Issue the Air Carrier Certificate.

4.4       Certification Report.

4.4.1    CPM—Assemble the certification report, to include:

·    Preapplication Statement of Intent (PASI),

·    Formal letter of application,

·    Proving test report,

·    Emergency Evacuation Demonstration Report (FAA Form 8430‑1),

·    Ditching report,

·    Copy of the Air Carrier Certificate,

·    Final compliance statement, and

·    A copy of the OpSpecs issued.

4.4.2    CPM—Sign and retain the certification report.

4.4.3    CHDO Manager—Forward a copy of the certification report to the RFSD.

4.4.4    CMT/CPT/AFS‑900 FSPO—Conduct annual planning meeting.

4.5       Phase 4 Review.

4.5.1    CPM/CTL—Verify the completion of all steps in phase 4 (see Figure 10‑66).

4.5.2    CPM—Solicit applicant and CPT feedback (refer to AFS Stakeholder Feedback). The CTL can also complete this by interviewing the CHDO and applicant.

4.5.3    CTL—Make entries into the PMT Project Summary.

4.5.4    CTL—Comment in the PMT Project Summary verifying that required PMT records are in the supporting documents. Include the CTL’s name.

4.5.5    CTL—Send a message to AFS‑900 Certification and Evaluation Program Office team members to remove company manuals from their computers and request each team member send a message back to the CTL when the team members complete this.

Figure 10‑61A.  Schedule of Events

Figure 10‑61A.  Schedule of Events (Continued)

Figure 10‑62.  Preapplication Checklist

Figure 10‑62.  Preapplication Checklist (Continued)

Complete the Preapplication Checklist (PAC)

At least 45 days prior to the date that the applicant wishes to schedule his or her formal application meeting, he or she must submit the Formal Application Letter to the certificate‑holding district office (CHDO) and include at a minimum:

·    An updated Federal Aviation Administration (FAA) Form 8400‑6, Preapplication Statement of Intent (only if you have made changes to the original PASI).

·    The completed proposed Schedule of Events (SOE) (Figure 10‑61A). Indicate whether it is the original or an updated version.

·    The completed Required Management Personnel Qualification Summary and Quality Audit Forms.

·    The completed list of proposed operations specifications (OpSpecs).

NOTE:  For further information on the PAC items refer to the following Web page: http://www.faa.gov/about/initiatives/atos/air_carrier/pre_app_checklist/.

At least 15 working‑days prior to the date that the applicant wishes to schedule your formal application meeting, he or she must submit all additional PAC items to the CHDO as listed below:

·    An updated PASI FAA Form 8400‑6 only if you have made changes to the original PASI.

·    An updated proposed SOE.

·    Evidence of economic authority request.

·    Corporate documents.

·    Deviation or exemption requests.

·    SAI self‑audit; the completed SAI version must be current within 90 days of the first submission. The requirement for using a current DCT revision will be optional for any additional SAI self-audit submissions.

·    Compliance statement.

·    Company manuals and programs necessary to conduct daily operations. This would include computer software program user manuals and engineering and program substantiation documents that support or justify the manual system or a program design (e.g., Weight and Balance (W&B) programs; aircraft modifications; aircraft manufacturer’s maintenance, flight, operating, and performance manuals; training programs and courseware; and user manuals for dispatch programs that the applicant purchased and contracted).

·    ICCA training plan.

·    Lease or proof of ownership and facility lease agreements.

·    Outsourcing contractual agreement.

·    Aircraft lease or proof of ownership.

·    Current aircraft equipment list.

·    Layout of passenger accommodations.

·    Aircraft information form.

·    Status of drug and alcohol program.

·    Status of security program.

·    Status of Environmental Assessment (EA).

·    Status of hazmat program and hazmat training program.

Figure 10‑63.  Formal Application, Management Questionnaire Worksheet

Figure 10‑64.  Automation Access Information Worksheet

10-436    INITIAL REVIEW INSTRUCTIONS.

A.    Purpose. The purpose of the initial review is to determine the quality of applicant’s application package.

B.    Procedure.

1)    The CPT will:

a)    Verify that the applicant has submitted all PAC documents and they are complete/acceptable.

b)    Verify that all manuals required to conduct the DA are available to the CPT.

c)    Verify that the compliance statement includes applicable parts 119 and 121 rules.

d)    The CPM/CTL will select a minimum of nine SAIs, distributed as evenly as practicable across air carrier systems and ASI disciplines. These SAIs will consist of six SAIs that pertain to programs that require approval or are related to OpSpecs and three other SAIs (see ATOS elements in Figure 10‑6, ATOS System/Subsystem/Element Chart—Airworthiness Elements, and Figure 10‑7, ATOS System/Subsystem/Element Chart—Operations and Cabin Safety Elements, in Volume 10, Chapter 1, Section 2). The CPM/CTL will distribute the SAIs that he or she selects for this review between the CHDO and AFS‑900 Certification and Evaluation Program Office team members.

2)    Review process.

a)    Download the selected SAIs using MS Word format. The CPT conducting the review is required to use the most current SAI DCT revision regarding all submissions throughout phases 1 and 2, through the Flight Standards Information Management Systems (FSIMS) Publications Active Publication list.

b)    Compare the revision date of the downloaded SAI to that of the submitted self‑audited SAI, and make sure they are the same.

c)    Complete all five sections of the selected SAIs during the initial review, per ATOS guidance.

3)    Evaluation criteria.

a)    The system design must not include unacceptable safety issues.

b)    The applicant must submit 100 percent of acceptable documents that the PAC requires.

c)    The compliance statement is acceptable to the CPT.

d)    Ninety percent of the references given by the applicant for each selected self‑audited SAI must be accurate, and the answers to the questions must be answered adequately. The programs must be approvable/acceptable to the CPT. If the reference is to the manual system, it must be to the volume, chapter, section, and paragraph level (or equivalent). This review will determine the accuracy and completeness of the SAI and assess the applicant’s ability to conduct self‑audits. References and answers that do not meet either criteria must be counted against the 90 percent. When multiple references are given, all references must be accurate.

e)    The design of an air carrier’s system described in the selected SAI must, at a minimum, meet all the regulatory and design requirements necessary to be acceptable with minor mitigation.

f)    The CTL should enter the results in the PMT per the PMT instructions.

NOTE:  In the case of a failed review, the CPM will require an additional three SAIs (not previously reviewed) to be reviewed by the CPT. Three rejections of the initial review can be cause for termination of the certification project.

10-437    DATA ANALYSIS MEETING.

A.    CPM/CTL. Collect and analyze all SAI data and schedule a meeting.

B.    CPT. Conduct a meeting to review and analyze the collected data.

C.    CPT. Draft a letter to the applicant and detail the negative responses and cause for the rejection of the formal application package. Do not cut and paste “No” answers. Keep the letter at the element level.

D.    CPM. The letter must advise the applicant that they must return the next revision as an entire submission accompanied by revised self‑audited SAIs. The applicant will label all documents in the revised submission in sequential order (Revision B, Revision C, etc.). Also, the applicant will use their manual revision process when revising their manuals (i.e., change bars or other ways they annotate changes).

10-438    FORMAL APPLICATION MEETING PLAN.

A.    General.

1)    The objective of this exercise is to determine that the applicant’s § 119.65 required management personnel are knowledgeable of their air carrier systems and are knowledgeable of the applicable rules and guidance.

2)    The CPT accomplishes this task by developing questions related to processes under the individual’s areas of responsibility or control.

B.    Process.

1)    CPT. Design questions that will assess the applicant’s required management personnel’s knowledge of their processes and associated system documentation. The questions should directly relate to processes under the individual manager’s area of responsibility. The answers should include references to the manual system or other company documentation.

a)    Questions and answers should be based on the current status of the air carrier system documentation.

b)    Questions should target each manager’s knowledge of the process for which he or she has responsibility; they should not be scenario based.

2)    CPM. Document the questions and expected answers, including the manual/documentation reference(s) on the Formal Application, Management Questionnaire Worksheet (see Figure 10‑63).

3)    CPM. Assign questions to CPT members.

4)    CPT. Discard questions and answers upon successful completion of the meeting.

C.    Develop an Agenda. Suggested agenda items:

·    Introductions and circulate attendance roster (see Figure 10‑65, Attendance Roster),

·    Applicant describes proposed air carrier systems,

·    Review certification process,

·    Explain conduct and criteria of this meeting,

·    Conduct management questions,

·    Resolve discrepancies and open items,

·    Discuss SOE,

·    Discuss proposed OpSpecs, and

·    Review requests for exemption or deviation.

D.    Acceptable Completion Standards. Each of the applicant’s required management personnel should demonstrate an acceptable level of understanding of their air carrier systems by correctly answering all questions. Unsuccessful results may require retesting the manager(s) at a later date.

NOTE:  This event is not an opportunity to reveal inadequacies in the applicant’s air carrier system design. The DA phase addresses deficiencies in the air carrier’s system design.

NOTE:  Do not allow managers to locate manual references by using “word search” functions.

Figure 10‑65.  Attendance Roster

Figure 10‑66.  Phases and Gates for Initial Certification

10-439    TABLETOP EXERCISE.

A.    Objective. The objective of this exercise is to determine if the air carrier systems and personnel perform at a level that justifies conducting proving tests.

1)    The CPM and CTL are responsible for the quality of the administration of this exercise.

2)    The CPM is responsible for scheduling the exercise.

3)    The applicant is responsible for the attendance of all personnel who will participate in the tabletop scenarios.

B.    Procedure.

1)    CPT. Create a minimum of 20 scenarios related to the elements identified in the CAP as high priority. Document the results of each scenario on the Tabletop Scenario Worksheet (see Figure 10‑68).

NOTE:  The CPT will not report tabletop exercise results in ATOS automation.

2)    CPM. Assign scenarios to individual CPT members to present to appropriate applicant personnel.

3)    CPM. Select a minimum of 10 scenarios to use in the exercise. The remaining scenarios are available in the event that a retest is necessary.

4)    CPM/CTL. Develop an agenda with the suggested agenda items:

·    Introductions and circulate attendance roster (see Figure 10‑65).

·    Review certification process.

·    Explain conduct and criteria of this meeting.

·    Conduct the exercise.

·    Resolve discrepancies and open items.

·    Discuss SOEs.

5)    CPT. Present scenarios to applicant personnel. Allow the natural progression of each process to determine the effectiveness of the exercise.

6)    Applicant. After the completion of each scenario, the applicant will brief the CPT on its perceptions of its performance. Document each scenario on the Tabletop Scenario Worksheet (see Figure 10‑68).

7)    CPT. Meet after the completion of each scenario and determine if the applicant’s performance was satisfactory or unsatisfactory.

8)    CPM. Brief the applicant on the CPT’s assessment of the scenario.

9)    CPM. If applicant performance is unsatisfactory, present another scenario that addresses a similar area (ATOS element). If the applicant performance is again unsatisfactory, confer with the CPT, and if there is a consensus, reschedule the exercise.

10)    CPM. If three scenarios (not from the same element) are unsatisfactory, confer with the CPT, and if there is a consensus, reschedule the exercise.

11)    CPM. Notify the applicant of the results.

12)    CPM. Discard the Tabletop Scenario Worksheet.

C.    Sample Scenario. During preflight for the first flight of the day from an outstation where maintenance is available, a crewmember finds minor damage to an outflow valve. Thunderstorms are forecast along the planned flight route, and it is currently raining heavily at the departure airport.

Figure 10‑68.  Tabletop Scenario Worksheet

10-440    PROVING TEST PROTOCOLS.

A.    Introduction. The policies and procedures in this paragraph are intended as general guidance to assist the CPM, CTL, and CPT in managing the administrative elements of proving tests. For guidance concerning operational aspects of proving tests, see Volume 3, Chapter 29.

NOTE:  Obtain concurrence from the RFSD before terminating proving tests with fewer hours than listed in the approved Proving Test Plan.

B.    Scenario Development.

1)    CPM. Identify EPI or ConDOR questions to assess the applicant’s performance.

2)    CPT. Develop scenarios that generate data for identified questions.

3)    CPT. Document each scenario on the Proving‑Flight Test Scenario Worksheet (see Figure 10‑69).

4)    CPM. Match scenarios to specific flight legs.

5)    CPM. Determine ASI assignments for each scenario and document it in the “Assignment” field of the worksheet.

6)    CPM. Determine the method for presenting the scenario and document it in the “Initiation” field of the worksheet.

NOTE:  It is not always practical to initiate a scenario by handing a note or giving verbal instructions to the applicant. The assigned ASI playing the role of the passenger can best initiate scenarios such as a passenger standing during taxi, appearing intoxicated, or using a cell phone before takeoff.

7)    CPM. Assign identified EPIs to the CPT.

C.    Protocols.

1)    Treat FAA personnel as passengers, unless they present their FAA ASI’s credentials, FAA Form 110A, Aviation Safety Inspector’s Credential, and provide instructions.

2)    The ASI assigned to the flight deck is in ASI status while in the cockpit.

3)    At all times during proving tests, ASIs should observe the applicant’s normal and routine operations such as preflight duties, carry‑on baggage, passenger briefings, and crew communication and coordination, to ensure they follow company procedures and achieve all regulatory requirements.

4)    Flights conducted during the proving tests may contain more than one scenario.

NOTE:  The FAA may conduct scenarios simultaneously if each scenario is isolated to specific duty positions (e.g., F/As, pilots, mechanics, dispatchers) and disciplines. The FAA cannot present multiple scenarios to a specific discipline. The CPM must ensure that the applicant is not encumbered with so many simulated scenarios that a proper evaluation of its proposed routine operation is inhibited.

5)    ASIs should discreetly communicate with one another when a scenario is about to begin.

6)    ASIs should act the part of a passenger in “real‑life” scenarios.

7)    ASIs are not to touch or operate any emergency equipment on board.

8)    Personnel are not to actually position engine controls, switches, etc., in response to a scenario.

9)    Tape a note to the equipment, give a note to a crewmember, or have an inspector verbally inform the crewmember in the case of an emergency equipment malfunction scenario.

10)    Assign one ASI to end each scenario.

11)    Use the Proving‑Flight Test Scenario Worksheet (see Figure 10‑69) to document the results of each scenario.

12)    Terminate any scenario if an actual emergency should occur.

13)    Do not discuss scenario results with the applicant. The CPT will need to discuss the results of each scenario as a team at the end of the day, and then brief the applicant.

14)    It is permissible to use the actual aircraft logbook to record scenarios. Record the items entered in the logbook as an FAA scenario to differentiate between scenarios and real logbook entries. The applicant may choose to use a separate log for scenarios, but must ensure that flight hours and genuine malfunctions are recorded in the actual aircraft logbook.

15)    The CPM or CTL will carry the flight schedule, crewmember names, and scenario worksheets, and will coordinate with the ASIs on board. If the CPM and CTL are not on board a particular day, the CPM must designate another ASI to serve as the FAA team lead.

16)    The CPT will give the completed Proving‑Flight Test Scenario Worksheet to the CPM for the Certification Report (see Figure 10‑61, paragraph 4.4).

D.    First Applicant Proving Test Meeting. The CPM should discuss the following items with the applicant:

1)    Proving test flight schedule.

NOTE:  The CPM/CTL may use discretion when determining schedules and should plan events within an 8‑hour workday, including briefings.

2)    How the CPM/CPT will conduct the preflight and postflight briefing. (Debriefing at the conclusion of each day unless major problems require it sooner. The applicant must resolve all major discrepancies before the proving test resumes the following day.)

3)    The purpose of the proving tests.

4)    Treat FAA personnel as passengers, unless they present their FAA credential, FAA Form 110A, and provide instructions. An ASI occupying the flight deck observer’s seat is always in ASI status.

5)    Initiation of the scenario and expected company action.

6)    Personnel are not to actually position engine controls, switches, etc., in response to a scenario, even though a checklist procedure may direct such action.

7)    The termination of satisfactorily completed scenarios and scenarios that the applicant did not satisfactorily complete.

8)    How to react to an actual emergency during the proving test. In case of an actual emergency, the FAA suspends any scenarios that may be in progress.

9)    Use the actual aircraft logbook to record scenarios. FAA scenario logbook entries will be identified to differentiate from real logbook entries. The applicant may choose to use a separate log for scenarios, but must ensure that the company’s actual logbook has a record of flight hours and malfunctions.

10)    Company personnel should not ask the FAA team members about the company’s performance. The daily debriefing will include the necessary information relative to the company’s performance.

11)    The delay, suspension, or termination of the proving test by the FAA for safety concerns.

12)    The location of FAA ASIs at company facilities (e.g., dispatch, maintenance control, stations).

13)    The performance of scenarios by company personnel by following appropriate company policy and procedure. Unless advised otherwise by the FAA, company personnel involved in a scenario should effect normal communication and coordination between the cabin, cockpit, dispatch, maintenance control, etc.

NOTE:  Confine communications to company frequencies unless directed by the FAA ASI occupying the observer’s seat. The ASI occupying the observer’s seat or observing at ground facilities will coordinate as needed with air traffic control (ATC) and company personnel concerning the status. The simulated emergencies will not involve deactivating items such as constant speed drives (CSD), integrated drive generators (IDG), etc., nor will it involve engine in‑flight shut downs (IFSD). Follow the policies, practices, and procedures contained in your manual.

14)    Do not inform other employees that this is a test problem or scenario. This includes telephone, signals, notes, or other means of communication.

15)    Company personnel may not enlist the services of any person not assigned to be on official duty during a scenario.

16)    The company will provide the FAA a list of passengers before each day’s briefing.

17)    An FAA ASI will not manipulate aircraft controls except in the interest of safety.

18)    The CPM will determine the flight hours credited toward the proving test plan.

19)    The Federal Flight Deck Officer (FFDO) is not authorized to carry a weapon during proving tests (refer to Title 49 of the Code of Federal Regulations (49 CFR) part 1544).

E.    Daily Preflight Briefings.

1)    FAA Team Lead. Before the start of each daily flight schedule, the CPT will meet to review the flight schedule, scenarios for that day, and specific team member assignments.

2)    FAA Team Lead. Meet with the applicant, crewmembers, and other applicable personnel to review the schedule and clear up any questions or concerns.

F.    Daily FAA‑Only Postflight Briefings. The CPT will determine if they have collected adequate EPI data and discuss the applicant’s performance. If it is unlikely the collected data will affirm element performance, the CPM/CTL will determine continuation in accordance with the following:

·    In the event of multiple system failures (air carrier systems) or severe system deficiencies, the CPM/CTL may terminate proving tests to allow the applicant to redesign systems.

·    In the event of a failure of an individual, or a minor system failure, the CPM/CTL may terminate that segment, but continue the proving tests.

·    The FAA may repeat scenarios, but the CPM/CTL will modify them.

·    If the FAA discontinues the proving tests, it will rescind the letter of approval.

G.    Daily Meeting With Applicant.

1)    Applicant. Provide the CPT with a self‑assessment of company performance.

2)    CPT. Debrief the applicant on each scenario and advise the status of continuance.

Figure 10‑69.  Proving‑Flight Test Scenario Worksheet

RESERVED. Paragraphs 10‑441 through 10‑461.