11/21/14

 

8900.1 CHG 368

Volume 10  AIR TRANSPORTATION OVERSIGHT SYSTEM

CHAPTER 2  PROCEDURES for Design and Performance Assessment

Section 8  Design Assessment Analysis and Assessment

Title: Change Bar - Description: Indicates new/changed information.

Figure 10-52.  Module 7: Design Analysis and Assessment

Figure 10-52. Module 7: Design Analysis and Assessment

10-258    INTRODUCTION. The analysis and assessment process module for Design Assessment (DA) determines if the air carrier’s or applicant’s system design meets the standards for acceptance or approval. The process uses data collected by Certificate Management Team (CMT) or Certification Project Team (CPT) members. The principal inspector (PI) or certification project manager (CPM) may use data from other sources to help make this bottom-line DA.

10-259    ANALYZE SAFETY ATTRIBUTE INSPECTION (SAI) DATA BY ELEMENT (see flowchart process step 7.1). After the collected SAI data have undergone a data quality review, the PI or CPM may request assistance from the Operations Research Analyst (ORA). The PI or CPM determines whether the air carrier’s or applicant’s system element design meets the requirements for initial or continued Federal Aviation Administration (FAA) approval or acceptance. This analysis involves looking at responses to SAI questions for that element, including “No” responses and explanations, “Yes” responses and comments, responses by question category and drop-down menu subjects, questions responded to as “Not Applicable,” and text entered in the “Inspector Action Taken” box.

A.    “Yes” Response. A “Yes” response means that the ASI observed conditions at the air carrier or applicant, or in its documentation, that meets the criteria specified by the Data Collection Tool (DCT) question.

B.    “No” Response. A “No” response means that the ASI observed conditions at the air carrier or applicant, or in its documentation, that does not meet the criteria specified by the DCT question.

NOTE:  Drop-down menus are available when an ASI responds “No” to DCT questions. Select one or more options to narrow the suspected cause for identified deficiency.

C.    “Not Applicable” Response. A “Not Applicable” answer means that the DCT question does not apply to the evaluated air carrier because of the type of operation, aircraft, area of operation, etc.

Title: Change Bar - Description: Indicates new/changed information.

10-260    ASSESS ASSESSMENT DATA PACKAGE BY ELEMENT (see flowchart process step 7.2). The assessment data package provides both current and historical data for the element.

A.    Consolidate and Summarize Relevant Data. The PI or CPM will use the Air Transportation Oversight System (ATOS) data analysis package in the Safety Performance Analysis System (SPAS) to consolidate and summarize other relevant data needed to properly analyze and assess that element. By methodically assessing the data, you can make a bottom-line assessment of system design.

1)    For existing air carriers that desire to add or change a program, historical SAI data provide a useful source of information to consider before approving or accepting the program.
2)    For new applicants, no historical SAI data may exist to aid the CPT in determining program adequacy. Initial SAI data submission, along with the information from the ATOS data analysis package may be the only sources of information available for analysis.

B.    Additional Reports. There are various ways to construct other reports that ORAs or the SPAS provide to include the data necessary to complete the analysis and assessment process more effectively. The PI or CPM may request that the ORA generate additional analyses and reports to allow for adequate assessment of the element.

Title: Change Bar - Description: Indicates new/changed information.

C.    Conduct Risk Assessment Meeting. The PI or CPM may convene a DA meeting with other CMT or CPT members, as required, to review data related to the element and assist the PI or CPM in making the bottom-line assessment. This meeting may be held in conjunction with a performance risk assessment meeting. The PI or CPM can use the results of the DA meeting in decisionmaking, planning, and evaluating air carrier or applicant actions.

Indicates new/changed information.

10-261    DOES THE ELEMENT MEET THE REQUIREMENT FOR ACCEPTANCE OR APPROVAL? (see flowchart process step 7.3). After assessing the assessment data package with input from other CMT or CPT members, as required, the PI or CPM determines whether the air carrier system design for that element meets the requirements for either continued approval or acceptance, or initial approval or acceptance.

Title: Change Bar - Description: Indicates new/changed information.

10-262    DOCUMENT ACCEPTED/APPROVED ASSESSMENT (see flowchart process step 7.4). After completing the bottom-line assessment, the PI or CPM documents the decision to accept or approve the air carrier system design. (See Table 10-3, Bottom-Line Assessment Categories.) The PI or CPM includes the rationale for bottom-line assessments categories 2 through 5 (green or yellow). No rationale is required for bottom-line assessment category 1 (green) for “No issues observed.”

Title: Change Bar - Description: Indicates new/changed information.

10-263    DOCUMENT DESIGN ASSESSMENT REJECTED (see flowchart process step 7.5). If the PI or CPM determines that the applicant’s or air carrier’s system design does not meet the requirements for approval or acceptance, the PI or CPM documents the decision to reject the system. The PI or CPM includes the rationale for the decision and also notes any issues or concerns.

Indicates new/changed information.

Table 10-3.  Bottom-line Design Assessment Categories

1

Design Accepted/Approved

No issues observed

No action required

2

Design Accepted/Approved

Minor issues observed

No action required

3

Design Accepted/Approved

Minor issues observed

Mitigation required

4

Design Accepted/Approved

Major issues observed

Mitigation required

5

Design Accepted/Approved

Safety and/or regulatory issues observed

Mitigation required

6

Design Rejected

Systemic safety and/or regulatory issues observed

System reconfiguration by air carrier or applicant required

Legend

Green: 1–2

Yellow: 3–5

Red: 6

RESERVED.  Paragraphs 10-264 through 10-278.