5/1/12

 

8900.1 CHG 210

Volume 10  Air Transportation Oversight System

Chapter 2  Procedures for Design and Performance Assessment

Section 8  Design Assessment Analysis and Assessment

Figure 10-52.    Module 7: Design Analysis and Assessment

Figure 10-52.    Module 7: Design Analysis and Assessment

10-258    INTRODUCTION. The analysis and assessment process module for Design Assessment (DA) determines if the air carrier’s or applicant’s system design meets the standards for acceptance or approval. The process uses data collected by Certificate Management Team (CMT) or Certification Project Team (CPT) members. The principal inspector (PI) or certification project manager (CPM) may use data from other sources to help make this bottom-line DA.

10-259    ANALYZE SAFETY ATTRIBUTE INSPECTION (SAI) DATA BY ELEMENT. After the collected SAI data have undergone a data quality review, the PI or CPM may request assistance from the Operations Research Analyst (ORA). The PI or CPM determines whether the air carrier’s or applicant’s system element design meets the requirements for initial or continued Federal Aviation Administration (FAA) approval or acceptance. This analysis involves looking at responses to SAI questions for that element, including “No” responses and explanations, “Yes” responses and comments, responses by question category and drop-down menu subjects, questions responded to as “Not Applicable,” and text entered in the “Inspector Action Taken” box. (See flowchart process step 7.1.)

A.    “Yes” Response. A “Yes” response means that the inspector observed conditions at the air carrier or applicant, or in its documentation that meets the criteria specified by the Data Collection Tool (DCT) question.

B.    “No” Response. A “No” response means that the inspector observed conditions at the air carrier or applicant, or in its documentation, that does not meet the criteria specified by the DCT question.

NOTE:  Drop-down menus are available when an inspector responds “No” to DCT questions. Select one or more options to narrow the suspected cause for identified deficiency.

C.    “Not Applicable” Response. A “Not Applicable” answer means that the DCT question does not apply to the evaluated air carrier because of the type of operation, aircraft, area of operation, etc.

10-260    ASSESS DATA IN THE AIR TRANSPORTATION OVERSIGHT SYSTEM (ATOS) DATA ANALYSIS PACKAGE DATA BY ELEMENT (CONDUCT RISK ASSESSMENT MEETING, IF REQUIRED). The data analysis package compares analyzed and assessed SAI data for the current DA with historical data for the current DA with historical SAI and other data for the element. (See flowchart process step 7.2.)

Indicates new/changed information.

A.    Consolidate and Summarize Relevant Data. You, the PI or CPM will use the ATOS data analysis package in the Safety Performance Analysis System (SPAS) to consolidate and summarize other relevant data needed to properly analyze and assess that element. By methodically assessing the data, you can make a bottom-line assessment of system design.

1)    For existing air carriers that desire to add or change a program, historical SAI data provide a useful source of information to consider before approving or accepting the program.
2)    For new applicants, no historical SAI data may exist to aid the CPT in determining program adequacy. Initial SAI data submission, along with the information from the ATOS data analysis package may be the only sources of information available for analysis.

B.    Additional Reports. There are various ways to construct other reports that ORAs or the SPAS provide to include the data necessary to complete the analysis and assessment process more effectively. The PI or CPM may request that the ORA generate additional analyses and reports to allow for adequate assessment of the element.

C.    DA Meeting. The PI or CPM may convene a DA meeting with other CMT or CPT members, as required, to review data related to the element and aid in making the bottom-line DA. This meeting may be held in conjunction with a performance risk assessment meeting. The PI or CPM can use the results of the DA meeting in decisionmaking, planning, and evaluating air carrier or applicant actions.

Indicates new/changed information.

10-261    MEETING THE REQUIREMENT FOR ACCEPTANCE OR APPROVAL. After assessing the ATOS data analysis package with input from other CMT or CPT members, as required, the PI or CPM determines whether the air carrier system design for that element meets the requirements for either continued approval or acceptance, or initial approval or acceptance. (See flowchart process step 7.3).

Indicates new/changed information.

10-262    DOCUMENT ACCEPTED/APPROVED DA. After completing the bottom-line assessment, the PI or CPM documents the decision to accept or approve the air carrier system design. (See Table 10-3, Bottom-line Design Assessment Categories.) The PI or CPM includes the rationale for bottom-line assessments categories 2 through 5 (green or yellow). No rationale is required for bottom-line assessment category 1 (green) for “No issues observed.” (See flowchart process step 7.4.)

10-263    DOCUMENT DA REJECTED. If the PI or CPM determines that the applicant’s or air carrier’s system design does not meet the requirements for approval or acceptance, the PI or CPM documents the decision to reject the system. The PI or CPM includes the rationale for the decision and also notes any issues or concerns. (See flowchart process step 7.5.)

Table 10-3.    Bottom-line Design Assessment Categories

1

Design Accepted/Approved

No issues observed

No action required

2

Design Accepted/Approved

Minor issues observed

No action required

3

Design Accepted/Approved

Minor issues observed

Mitigation required

4

Design Accepted/Approved

Major issues observed

Mitigation required

5

Design Accepted/Approved

Safety and/or regulatory issues observed

Mitigation required

6

Design Rejected

Systemic safety and/or regulatory issues observed

System reconfiguration by air carrier or applicant required

Legend

Green: 1–2

Yellow: 3–5

Red: 6

RESERVED. Paragraphs 10-264 through 10-278.