12/30/09

 

8900.1 CHG 81

Volume 10  Air Transportation Oversight System

Chapter 1  General

Section 2  Introduction to ATOS Business Process and Tools

10-20     PURPOSE. The Air Transportation Oversight System (ATOS) improves the certification and surveillance processes for air carriers. It assesses the safety of air carrier operating systems using system safety principles, safety attributes, risk management, and structured system engineering practices.

10-21     OVERSIGHT SYSTEM MODEL. ATOS exists in relation to the aviation system that an air carrier uses to produce its goods and services. The air carrier is the process owner of its aviation system, which is a production system in that it serves customers via products and services. The FAA is the process owner of the oversight system, which is a protection system. Protection systems are designed to protect customers from receiving inferior goods and services, and from potential harm of production activities. This includes potential harm from airplane accidents, occupational hazards, loss of equipment and other property, and damage to the environment. Safety and quality management systems are also protection systems. FAA and air carriers are the process owners of such complementary systems. The relationship between production and protection systems is a matter of exchanging information and exerting influence. Protection systems influence production systems by imposing controls. Figure 10‑1 shows this relationship at a high level.

Figure 10‑1, Oversight System Model—Level I

Figure 10-1, Oversight System Model--Level I

A.    Major Functions of ATOS. Three major functions further define the oversight system: design assessment (DA), performance assessment (PA), and risk management. The following Level II diagram shows these functions in a high-level model.

Figure 10‑2, Oversight System Model—Level II

Figure 10-2, Oversight System Model--Level II

B.    Design Assessment. DA is the ATOS function that ensures an air carrier’s operating systems comply with regulations and safety standards, including the requirement to provide service at the highest level of safety in the public interest. DA is the most important function of ATOS because safety is the outcome of a properly designed system. Poor system design compromises safety risk management. ATOS certification processes ensure that an air carrier’s operating systems comply with the intent of the regulations. ATOS uses standardized, systematic certification processes to determine an air carrier’s qualification for an operating certificate. FAA uses similar processes to approve or accept a new or changed air carrier program. The tools used in the certification processes also re-verify that an air carrier is meeting regulatory requirements during periodic program reviews or when environmental changes occur.

C.    Performance Assessment. FAA inspectors conduct PAs to confirm that an air carrier’s operating systems produce intended results, including mitigation or control of hazards and associated risks. ATOS uses time‑based PAs to detect latent, systemic failures that may occur due to subtle environmental changes. PA schedules are also adjustable based on known risks or safety priorities. Surveillance provides information for PAs and risk management. In this context, surveillance is synonymous with auditing. ATOS audits use the same tools as certification processes.

D.    Risk Management. Risk management identifies and controls hazards and manages FAA resources according to risk-based priorities. This function is accomplished through systematic risk assessments of an air carrier’s performance and environment. Hazards are defined in terms of their potential consequences. The likelihood and severity of a consequence determines risk. ATOS assesses the combined effects of likelihood and severity to determine priority when multiple risks are identified. Subsequent risk management action plans contain strategies to transfer, eliminate, accept, or mitigate the risk. This process validates the intended results of an action plan to ensure that a hazard is effectively eliminated or controlled. ATOS uses a modified version of Nicholas J. Bahr’s System Safety Process Model for hazard identification and risk management (see Figure 10-4). ATOS is concerned with the hazards and associated risks that are subject to regulatory controls such as enforcement actions, certificate amendments, and rulemaking. Hazards that are identified as the responsibility of an air carrier are tracked in the ATOS Risk Management Process until the air carrier satisfactorily resolves them.

10-22    BUSINESS PROCESS MODULES. A Level III model of the oversight system further defines ATOS processes at a procedural level.

A.    Definition of Business Process Modules. There are eight business process modules in the design and performance functions. The application of each process may be somewhat different depending on whether conducting a DA or PA, but the overall purpose of each module is as follows:

1)    Module 1—System Configuration. The System Configuration process assesses an air carrier’s or applicant’s request for a new or changed scope of operation to develop an oversight profile that contains all applicable elements.
2)    Module 2—Planning. Planning develops a risk‑based data collection plan for DAs and PAs.
3)    Module 3—Resource Management. Resource Management provides the resources, training, and funding to execute the data collection plan.
4)    Module 4—Data Collection. The Data Collection process collects the data requested in the data collection plan.
5)    Module 5—Data Reporting. The Data Reporting process transfers the collected data into the ATOS database.
6)    Module 6—Data Review. The Data Review process evaluates data in the ATOS database for compliance with the requirements in the data quality guidelines.
7)    Module 7—Analysis and Assessment.Analysis and Assessment makes a data‑based decision about whether to approve or accept or reject the design or performance of an air carrier’s or applicant’s programs.
8)    Module 8—Action Determination and Implementation. Action Determination and Implementation decides on and executes the appropriate course of action based on the decisions made during analysis and assessment.

B.     Flow Chart. The Level III model consists of cross-functional flowcharts for each of the business process modules. Cross‑functional flowcharts show responsibility for process steps using horizontal bands and vertical bands (swim lanes). Standard flowchart symbols used to depict ATOS processes include:

Figure 10‑3, Standard ATOS Business Process Flowchart Symbols

Figure 10-3, Standard ATOS Business Process Flowchart Symbols

10-23    APPLICATION OF SYSTEM SAFETY CONCEPTS. The overall purpose of a system safety‑based approach like ATOS is to identify, eliminate, or control hazards, and mitigate the associated risk. Below is the AFS System Safety Process Model, a modified version of the Nicholas J. Bahr system safety process model. Comparing the ATOS model to the System Safety Process Model demonstrates how ATOS is a system safety‑based approach to air carrier oversight.

Figure 10‑4, System Safety Process Model

Figure 10-4, System Safety Process Model

Figure 10‑5, Comparison of the ATOS Business Process and the System Safety Process

System Safety Process Step

ATOS Business Process

Define Objectives and Describe System.

Define the acceptable levels of safety.

How does the system work and how do its components interact?

Module 1, System Configuration.

Regulations and policy define the acceptable level of safety. System description begins with the Air Carrier Oversight Profile.

Hazard Identification.

Where are the hazards in the system?

What can go wrong?

Modules 2–6.

Identify areas where conditions in the system or operating environment may be creating hazards. Plan, report, and review data collection in those areas.

Risk Analysis and Assessment.

Determine the potential consequences that could result if hazards are not addressed or corrected.

Module 7, Analysis and Assessment.

PIs analyze data to identify hazards and determine the level of risk associated with a hazard. Hazards are assessed in terms of their severity and likelihood that an adverse outcome would result.

Decisionmaking.

What can be done to control the effects of the hazard and/or mitigate the associated levels of risk?

Module 8, Action Determination and Implementation. Take action to control the effects of the hazard and/or mitigate unacceptable levels of risk.

Validation of Controls.

Did the action work?

Module 8, Action Determination and Implementation. Determine if the action eliminated the hazard or lowered the level of risk to acceptable levels. If not, take additional action.

10-24    AIR CARRIER SYSTEMS, SUBSYSTEMS, AND ELEMENTS. ATOS uses a structured process to analyze how systems, subsystems, and elements interact. Seven air carrier systems form the basis for the ATOS system‑based approach. Each of these systems has a defined set of subsystems and elements. Elements are interrelated activities or actions completed to support air carrier subsystems and systems. Figures 10‑6 and 10‑7 at the end of this section identify each of the systems, subsystems, and elements (along with associated inspector specialties) used in planning and executing ATOS data collection. Below are the seven air carrier systems:

A.    Aircraft Configuration Control. An air carrier maintains the physical condition of the aircraft and its components using this system.

B.    Manuals System. This system controls the information and instructions to define and govern the air carrier activities.

C.    Flight Operations. This system pertains to aircraft movement.

D.    Personnel Training and Qualifications. Air carriers use processes to make sure that their personnel are trained and qualified.

E.    Route Structures. An air carrier uses this system to maintain facilities on approved routes.

F.    Airman and Crewmember Flight, Rest, and Duty Time. This system prescribes time limitations for air carrier employees.

G.    Technical Administration. Air carriers use this system to address other aspects of certification and operation, such as key management personnel.

10-25    ATOS TOOLS. ATOS standardizes the certification and surveillance processes. Its structured automation tools include:

A.    Air Carrier Configuration Checklist. The Air Carrier Configuration Checklist helps Certification Project Teams and Certificate Management Teams (CMT) to document the air carrier’s or applicant’s scope of operation including factors such as type of operations, aircraft, equipment and operations specifications. This information is used for automated filtering of the oversight profile.

B.    Air Carrier Oversight Profile. This profile is a tailored list of elements, Data Collection Tool (DCT) questions, and job task items that are based on the specific regulatory requirements that apply to the air carrier or applicant.

C.    The Air Carrier Assessment Tool (ACAT). This tool uses risk indicators to look for conditions that may be creating hazards in the air carrier’s systems. The results of the ACAT help to prioritize oversight activities.

D.    The Comprehensive Assessment Plan (CAP). This tool helps plan DAs and PAs.

E.    Outsource Oversight Prioritization Tool (OPT). The OPT is used for planning surveillance of air carrier outsource providers. Principal avionics inspectors and principal maintenance inspectors must use this tool during oversight planning. It allows for prioritization of outsource maintenance providers to help determine specific data collection requirements. The OPT will assist the principal inspector, other assigned inspectors, supervisors, and managers in identifying areas of concern or criticality about outsource providers and target resources toward the highest risk outsource maintenance providers.

F.    Corrective Action Tracking Tool (CATT). The CATT must be used to ensure the certificate holders meet schedules for completing corrective actions that result from DAs and PAs, or other oversight functions. The CATT may be used to track other CMT issues. The CATT does not replace documentation requirements specified in other guidance.

G.    Data Collection Tools (DCT). ASIs must ensure they are using the most current version of the DCT by checking for any temporary revisions in the ATOS News and Documentation DCT Temporary Revisions section. These ATOS tools support the assessments:

1)    Safety Attribute Inspection (SAI). Inspectors use the SAI questions to collect data for DA. Air carrier applicants use SAI DCTs during initial certification to document the results of their self audit.
2)    Element Performance Inspection (EPI). Inspectors use the EPI questions to collect data for PA.
3)    Constructed Dynamic Observation Report (ConDOR). This tool is used for focused, special inspections.
4)     Dynamic Observation Report (DOR). This tool allows inspectors to record safety observations outside the planned oversight process. These observations may involve any part 121 air carrier at any location, at any time.
5)    ATOS Random Inspections. These tools allow ATOS‑qualified part 121 air carrier inspectors to record unplanned cockpit en route inspections, cabin en route inspections, ramp inspections, or spot inspections. With management authorization, inspectors may perform these inspections on any part 121 air carrier at any location at any time. Random inspections are not included in CAPs or the National Flight Standards Work Program Guidelines.

NOTE:  ASIs that have access to ATOS automation can access random inspections on their home page. ASIs that do not have access to ATOS must request it from AFS‑900 by contacting the ATOS CMO Continual Improvement Section, Automation Team at ‘AVS‑AFS900‑ATOS‑Automation Team.

6)    Assessment Determination and Implementation Tool. The principal inspector or certification program manager uses this tool to document the bottom‑line DA or PA and the appropriate course of action for implementation.
7)    Off-Hour Surveillance Decision Aid. The CMT Comprehensive Assessment Plan (CAP) must be capable of detecting deficiencies in air carrier performance no matter when the activity is conducted. Based on information gathered during surveillance, the CMT will be prepared to evaluate the air carrier’s ability to manage activities conducted during off-hour periods and address any identified risks. Sufficient off-hour surveillance must occur to:
a)    Know what types and levels of activity are conducted during off hours.
b)    Understand how the air carrier is managing and supervising off-hour activities, especially the interface with outsource maintenance and other contracted activities.
c)    Determine if the air carrier’s processes and controls are sufficient to detect and correct any risks inherent with off-hour activities.
d)     Determine if the off-hour activities present a greater risk than activities done during normal FAA duty hours.

10-26    ATOS PROCESS FEEDBACK AND CONTINUOUS IMPROVEMENT. ATOS uses a feedback loop to aid in its effectiveness. Inspectors should submit their concerns or recommendations using the Problem Reporting and Feedback feature in ATOS automation.

Figure 10‑6, ATOS System/Subsystem/Element Chart—Airworthiness Elements

1.0 Aircraft Configuration Control

2.0 Manuals

1.1 Aircraft

2.1 Manual Management

1.1.3

Special Flight Permits

2.1.1

Manual Management

1.2 Records and Reporting Systems

 

 

1.2.1

Airworthiness Release/Maintenance Log Recording Requirements

 

 

1.2.4

Mechanical Interruption Summary (MIS)/Service Difficulty Report (SDR)

4.0 Personnel Training and Qualifications

1.3 Maintenance Organization

4.1 Maintenance Personnel Qualifications

1.3.1

Maintenance Program

4.1.1

Required Inspection Item Personnel

1.3.2

Maintenance/Inspection Schedule

4.1.2

Maintenance Certificate Requirements

1.3.3

Maintenance Facility/Main Maintenance Base

4.2 Training Program

1.3.4

Required Inspection Items

4.2.1

Maintenance/Required Inspection Item (RII) Training Program Requirements

1.3.5

Minimum Equipment List/Configuration Deviation List/Deferred Maintenance

 

 

1.3.6

Airworthiness Directives and Maintenance Record Requirements

 

 

1.3.7

Maintenance Providers

5.0 Route Structures

1.3.8

Control of Calibrated Tools and Test Equipment

5.1 Approved Routes and Areas

1.3.9

Major Repairs and Alterations

5.1.1

Line Stations (Service & Maintenance)

1.3.10

Aircraft Parts/Material Control

5.1.2

Weather Reporting/Supplemental Aviation Weather Reporting System

1.3.11

Continuous Analysis and Surveillance System (CASS)

5.1.3

Non‑Federal Navigational Aids

1.3.15

Reliability Program

5.1.8

Extended Operations

1.3.16

Fueling

5.1.9

Reduced Vertical Separation Minimum

1.3.17

Weight and Balance Program

7.0 Technical Administration

1.3.18

Deicing Program

7.1 Key Personnel

1.3.19

Lower Landing Minimums

7.1.1

Part 119 Required Personnel

1.3.23

Short‑Term Escalations

7.1.6

Maintenance Control

1.3.24

Coordinating Agencies for Suppliers Evaluation

 

 

1.3.25

Cargo Handling Equipment, Systems and Appliances

 

 

Figure 10‑7, ATOS System/Subsystem/Element Chart—Operations and Cabin Safety Elements

1.0 Aircraft Configuration Control

4.0 Personnel Training and Qualifications

1.1 Aircraft

4.2 Training Program

1.1.2

Appropriate Operational Equipment

4.2.3

Training of Flight Crewmembers

 

 

4.2.4

Training of Flight Attendants

2.0 Manuals

4.2.5

Training of Dispatchers

2.1 Manual Management

4.2.6

Training of Station Personnel

2.1.1

Manual Management

4.2.7

Training of Check Airman and Instructors

 

 

4.2.8

Simulators/Training Devices

 

 

4.2.9

Outsource Crewmember Training

 

 

4.2.11

Training of Flight Followers

 

 

4.2.12

Hazardous Materials Training

 

 

4.3 Crewmember and Dispatch Qualifications

3.0 Flight Operations

4.3.1

Pilot Operating Limitations/Recent Experience

3.1 Air Carrier Programs and Procedures

4.3.3

Advanced Qualification Program (AQP)

3.1.1

Passenger Handling

 

 

3.1.2

Crewmember Duties/Cabin Procedures

5.0 Route Structures

3.1.3

Airman Duties/Flight Deck Procedures

5.1 Approved Routes and Areas

3.1.4

Operational Control

5.1.5

Station Facilities

3.1.5

Carry‑on Baggage Program

5.1.6

Use of Approved Routes, Areas and Airports

3.1.6

Exit Seating Program

5.1.7

Special Navigation Areas of Operation

3.1.7

Deicing Program

5.1.8

Extended Operations

3.1.8

Carriage of Cargo

5.1.9

Reduced Vertical Separation Minimum Authorization

3.1.9

Airplane Performance Operating Limitations

6.0 Airman and Crewmember Flight, Rest and Duty Time

3.1.10

Category II and III Operations

6.1 Airman and Crewmember Limitations

3.1.11

Computer‑based Recordkeeping

6.1.2

Flight Crewmember Flight/Duty/Rest Time

3.1.12

Hazardous Materials

6.1.3

Flight Attendant Duty/Rest Time

3.1.13

Other Personnel with Operational Control

6.1.4

Dispatcher Duty/Rest Time

 

 

7.0 Technical Administration

3.2 Operational Release

7.1 Key Personnel

3.2.1

Dispatch or Flight Release

7.1.3

Part 119 Required Personnel

3.2.2

Flight/Load Manifest/Weight and Balance Control

7.2 Other Programs

3.2.3

Minimum Equipment List/Configuration Deviation List Procedures

7.2.1

Safety Program (Ground and Flight)

RESERVED. Paragraphs 10‑27 through 10‑41.