9/11/17

 

8900.1 CHG 465

VOLUME 17  SAFETY MANAGEMENT SYSTEM

CHAPTER 4  SAFETY MANAGEMENT SYSTEM VOLUNTARY PROGRAM

Section 3  Continued Operational Safety—Certificate Management Team Monitoring and Surveillance

17-4-3-1    PURPOSE. This section provides guidance for Certificate Management Teams (CMT) to perform continued oversight of a certificate holder’s applied safety management processes.

17-4-3-3    SAFETY MANAGEMENT SYSTEM VOLUNTARY PROGRAM (SMSVP) EXPECTATIONS. Volume 17, Chapter 4, Section 1, paragraph 17-4-1-13 clearly states that the CMT is expected to provide ongoing surveillance support to validate a certificate holder’s continued conformance to the SMSVP Standard. By doing so, it is anticipated that the CMT will realize significant benefits when performing its certificate holder oversight responsibilities. Regardless, failure of either the certificate holder or the CMT to adequately meet its obligations to SMSVP requirements may result in Safety Management System Program Office (SMSPO) withdrawal of “State recognition.”

17-4-3-5    APPLICATION OF SAFETY MANAGEMENT SYSTEM (SMS) TO CONTINUED OPERATIONAL SURVEILLANCE. Once safety management processes and activities have been integrated into the certificate holder’s technical processes, the CMT must broaden the scope of its normal surveillance to include the certificate holder’s SMS activities. Under a fully functioning SMS, when an inspector finds a regulatory violation or process nonconformance, his or her most important concern is, “Why didn’t the certificate holder’s SMS processes identify this problem, and if it was identified, why did the SMS not contain and/or correct this problem?” A certificate holder’s SMS increases organizational safety awareness and reduces “plausible excuses of ignorance” regarding systemic safety issues.

17-4-3-7    CMT SURVEILLANCE RECORDS. A CMT must record all safety management assessment activities to demonstrate certificate holder conformance with the SMSVP Standard. CMT surveillance activities, associated with safety management, must be recorded in the Safety Assurance System (SAS) data repository. This is accomplished by using Data Collection Tools (DCT) and associated questions sets designed into existing tools to assess safety management activities.

NOTE:  For those certificate holders not in SAS, the CMT will make Program Tracking and Reporting Subsystem (PTRS) entries as defined in the Continued Operational Safety (COS) job aids.

Figure 17-4-3A.  Safety Management System Voluntary Program Standard

1.    Purpose of This Attachment. The Safety Management System Voluntary Program (SMSVP) Standard, when properly applied, is the basis for formal State recognition of a certificate holder’s Safety Management System (SMS). The SMSVP Standard, while resembling Title 14 of the Code of Federal Regulations (14 CFR) Part 5, Safety Management Systems, is a separate document used by the Flight Standards Service (AFS) SMS Program Office (SMSPO) to evaluate SMSVP participants.

2.    Applicability. The SMSVP Standard details the minimum conformance expectations participants must maintain for State recognition of its SMS. Adherence to the SMSVP Standard does not replace compliance with other FAA regulatory requirements. The certificate holder may establish more stringent requirements in its system than those in this Standard.

SAFETY MANAGEMENT SYSTEM VOLUNTARY PROGRAM STANDARD

Subpart A—General.

5.1 Applicability.

5.3 General Requirements.

5.5 Definitions.

Subpart B—Safety Policy.

5.21 Safety Policy.

5.23 Safety Accountability and Authority.

5.25 Designation and Responsibilities of Required Safety Management Personnel.

5.27 Coordination of Emergency Response Planning.

Subpart C—Safety Risk Management.

5.51 Applicability.

5.53 System Analysis and Hazard Identification.

5.55 Safety Risk Assessment and Control.

Subpart D—Safety Assurance.

5.71 Safety Performance Monitoring and Measurement.

5.73 Safety Performance Assessment.

5.75 Continuous Improvement.

Subpart E—Safety Promotion.

5.91 Competencies and Training.

5.93 Safety Communication.

Subpart F—SMS Documentation and Recordkeeping.

5.95 SMS Documentation.

5.97 SMS Records.

Subpart A—General.

5.1 Applicability.

(a) A certificate holder desiring to implement an SMS must meet all requirements of this Standard and be found acceptable using the validation process as described in the Safety Management System Voluntary Program.

5.3 General Requirements.

(a) Any certificate holder required to have a Safety Management System under this Standard must submit the Safety Management System to the Administrator for acceptance. The SMS must be appropriate to the size, scope, and complexity of the certificate holder’s operation and include at least the following components:

(1) Safety policy in accordance with the requirements of subpart B of this Standard;

(2) Safety risk management in accordance with the requirements of subpart C of this Standard;

(3) Safety assurance in accordance with the requirements of subpart D of this Standard; and

(4) Safety promotion in accordance with the requirements of subpart E of this Standard.

(b) The Safety Management System must be maintained in accordance with the recordkeeping requirements in subpart F of this Standard.

(c) The Safety Management System must ensure compliance with the relevant regulatory standards in chapter I of Title 14 of the Code of Federal Regulations.

5.5 Definitions.

Hazard means a condition that could foreseeably cause or contribute to an aircraft accident as defined in Title 49 of the Code of Federal Regulations (49 CFR) part 830, § 830.2.

Risk means the composite of predicted severity and likelihood of the potential effect of a hazard.

Risk control means a means to reduce or eliminate the effects of hazards.

Safety assurance means processes within the SMS that function systematically to ensure the performance and effectiveness of safety risk controls and that the organization meets or exceeds its safety objectives through the collection, analysis, and assessment of information.

Safety objective means a measurable goal or desirable outcome related to safety.

Safety performance means realized or actual safety accomplishment relative to the organization’s safety objectives.

Safety policy means the certificate holder’s documented commitment to safety, which defines its safety objectives and the accountabilities and responsibilities of its employees in regards to safety.

Safety promotion means a combination of training and communication of safety information to support the implementation and operation of an SMS in an organization.

Safety Risk Management means a process within the SMS composed of describing the system, identifying the hazards, and analyzing, assessing and controlling risk.

Subpart B—Safety Policy.

5.21 Safety Policy.

(a) The certificate holder must have a safety policy that includes at least the following:

(1) The safety objectives of the certificate holder.

(2) A commitment of the certificate holder to fulfill the organization’s safety objectives.

(3) A clear statement about the provision of the necessary resources for the implementation of the SMS.

(4) A safety reporting policy that defines requirements for employee reporting of safety hazards or issues.

(5) A policy that defines unacceptable behavior and conditions for disciplinary action.

(6) An emergency response plan that provides for the safe transition from normal to emergency operations in accordance with the requirements of 5.27.

(b) The safety policy must be signed by the accountable executive described in 5.25.

(c) The safety policy must be documented and communicated throughout the certificate holder’s organization.

(d) The safety policy must be regularly reviewed by the accountable executive to ensure it remains relevant and appropriate to the certificate holder.

5.23 Safety Accountability and Authority.

(a) The certificate holder must define accountability for safety within the organization’s safety policy for the following individuals:

(1) Accountable executive, as described in 5.25.

(2) All members of management in regard to developing, implementing, and maintaining SMS processes within their area of responsibility, including, but not limited to:

(i) Hazard identification and safety risk assessment.
(ii) Assuring the effectiveness of safety risk controls.
(iii) Promoting safety as required in subpart E of this Standard.
(iv) Advising the accountable executive on the performance of the SMS and on any need for improvement.

(3) Employees relative to the certificate holder’s safety performance.

(b) The certificate holder must identify the levels of management with the authority to make decisions regarding safety risk acceptance.

5.25 Designation and Responsibilities of Required Safety Management Personnel.

(a) Designation of the accountable executive. The certificate holder must identify an accountable executive who, irrespective of other functions, satisfies the following:

(1) Is the final authority over operations authorized to be conducted under the certificate holder’s certificate(s).

(2) Controls the financial resources required for the operations to be conducted under the certificate holder’s certificate(s).

(3) Controls the human resources required for the operations authorized to be conducted under the certificate holder’s certificate(s).

(4) Retains ultimate responsibility for the safety performance of the operations conducted under the certificate holder’s certificate.

(b) Responsibilities of the accountable executive. The accountable executive must accomplish the following:

(1) Ensure that the SMS is properly implemented and performing in all areas of the certificate holder’s organization.

(2) Develop and sign the safety policy of the certificate holder.

(3) Communicate the safety policy throughout the certificate holder’s organization.

(4) Regularly review the certificate holder’s safety policy to ensure it remains relevant and appropriate to the certificate holder.

(5) Regularly review the safety performance of the certificate holder’s organization and direct actions necessary to address substandard safety performance in accordance with 5.75.

(c) Designation of management personnel. The accountable executive must designate sufficient management personnel who, on behalf of the accountable executive, are responsible for the following:

(1) Coordinate implementation, maintenance, and integration of the SMS throughout the certificate holder’s organization.

(2) Facilitate hazard identification and safety risk analysis.

(3) Monitor the effectiveness of safety risk controls.

(4) Ensure safety promotion throughout the certificate holder’s organization as required in subpart E of this Standard.

(5) Regularly report to the accountable executive on the performance of the SMS and on any need for improvement.

5.27 Coordination of Emergency Response Planning.

Where emergency response procedures are necessary, the certificate holder must develop and the accountable executive must approve as part of the safety policy, an emergency response plan that addresses at least the following:

(a) Delegation of emergency authority throughout the certificate holder’s organization;

(b) Assignment of employee responsibilities during the emergency; and

(c) Coordination of the certificate holder’s emergency response plans with the emergency response plans of other organizations it must interface with during the provision of its services.

Subpart C—Safety Risk Management.

5.51 Applicability.

A certificate holder must apply safety risk management to the following:

(a) Implementation of new systems.

(b) Revision of existing systems.

(c) Development of operational procedures.

(d) Identification of hazards or ineffective risk controls through the safety assurance processes in subpart D of this Standard.

5.53 System Analysis and Hazard Identification.

(a) When applying safety risk management, the certificate holder must analyze the systems identified in 5.51. Those system analyses must be used to identify hazards under paragraph (c) of this section, and in developing and implementing risk controls related to the system under 5.55(c).

(b) In conducting the system analysis, the following information must be considered:

(1) Function and purpose of the system.

(2) The system’s operating environment.

(3) An outline of the system’s processes and procedures.

(4) The personnel, equipment, and facilities necessary for operation of the system.

(c) The certificate holder must develop and maintain processes to identify hazards within the context of the system analysis.

5.55 Safety Risk Assessment and Control.

(a) The certificate holder must develop and maintain processes to analyze safety risk associated with the hazards identified in 5.53(c).

(b) The certificate holder must define a process for conducting risk assessment that allows for the determination of acceptable safety risk.

(c) The certificate holder must develop and maintain processes to develop safety risk controls that are necessary as a result of the safety risk assessment process under paragraph (b) of this section.

(d) The certificate holder must evaluate whether the risk will be acceptable with the proposed safety risk control applied, before the safety risk control is implemented.

Subpart D—Safety Assurance.

5.71 Safety Performance Monitoring and Measurement.

(a) The certificate holder must develop and maintain processes and systems to acquire data with respect to its operations, products, and services to monitor the safety performance of the organization. These processes and systems must include, at a minimum, the following:

(1) Monitoring of operational processes.

(2) Monitoring of the operational environment to detect changes.

(3) Auditing of operational processes and systems.

(4) Evaluations of the SMS and operational processes and systems.

(5) Investigations of incidents and accidents.

(6) Investigations of reports regarding potential noncompliance with regulatory standards or other safety risk controls established by the certificate holder through the safety risk management process established in subpart C of this Standard.

(7) A confidential employee reporting system in which employees can report hazards, issues, concerns, occurrences, incidents, as well as propose solutions and safety improvements.

(b) The certificate holder must develop and maintain processes that analyze the data acquired through the processes and systems identified under paragraph (a) of this section and any other relevant data with respect to its operations, products, and services.

5.73 Safety Performance Assessment.

(a) The certificate holder must conduct assessments of its safety performance against its safety objectives, which include reviews by the accountable executive, to:

(1) Ensure compliance with the safety risk controls established by the certificate holder.

(2) Evaluate the performance of the SMS.

(3) Evaluate the effectiveness of the safety risk controls established under 5.55(c) and identify any ineffective controls.

(4) Identify changes in the operational environment that may introduce new hazards.

(5) Identify new hazards.

(b) Upon completion of the assessment, if ineffective controls or new hazards are identified under paragraphs (a)(2) through (5) of this section, the certificate holder must use the safety risk management process described in subpart C of this Standard.

5.75 Continuous Improvement.

The certificate holder must establish and implement processes to correct safety performance deficiencies identified in the assessments conducted under 5.73.

Subpart E—Safety Promotion.

5.91 Competencies and Training.

The certificate holder must provide training to each individual identified in 5.23 to ensure the individuals attain and maintain the competencies necessary to perform their duties relevant to the operation and performance of the SMS.

5.93 Safety Communication.

The certificate holder must develop and maintain means for communicating safety information that, at a minimum:

(a) Ensures that employees are aware of the SMS policies, processes, and tools that are relevant to their responsibilities.

(b) Conveys hazard information relevant to the employee’s responsibilities.

(c) Explains why safety actions have been taken.

(d) Explains why safety procedures are introduced or changed.

Subpart F—SMS Documentation and Recordkeeping.

5.95 SMS Documentation.

The certificate holder must develop and maintain SMS documentation that describes the certificate holder’s:

(a) Safety policy.

(b) SMS processes and procedures.

5.97 SMS Records.

(a) The certificate holder must maintain records of outputs of safety risk management processes as described in subpart C of this Standard. Such records must be retained for as long as the control remains relevant to the operation.

(b) The certificate holder must maintain records of outputs of safety assurance processes as described in subpart D of this Standard. Such records must be retained for a minimum of 5 years.

(c) The certificate holder must maintain a record of all training provided under 5.91 for each individual. Such records must be retained for as long as the individual is employed by the certificate holder.

(d) The certificate holder must retain records of all communications provided under 5.93 for a minimum of 24 consecutive calendar-months.

Figure 17-4-3B.  SMS Safety Policy Design Validation

Certificate Holder Designator:

Date:

 

Process Area/Department Application:

 

SUPPLEMENTAL INFORMATION

Purpose: (Certificate Holder Responsibility) Implement and document a commitment to safety, which defines its safety objectives and employee safety accountabilities and responsibilities.

Objective: (FAA Responsibility) Validate that the certificate holder has effectively designed an SMS that incorporates a commitment to safety.

Related Code of Federal Regulations (CFR): Safety Management System Voluntary Program (SMSVP) Standard 5.21 through 5.27.

Related FAA Policy/Guidance: Advisory Circular (AC) 120-92, Safety Management Systems for Aviation Service Providers.

 

1.0 - Safety Policy

1.1 - Safety Policy

1)

Does the certificate holder’s SMS have a safety policy that includes at least the following minimum requirements:

    The certificate holder’s safety objectives;

    A commitment to fulfill the organization’s safety objectives;

    A clear statement to commit the necessary resources for implementation of the SMS;

    A safety reporting policy that defines requirements for employee reporting of safety hazards or issues;

    A policy that defines unacceptable behavior and conditions for disciplinary action; and

    An emergency response plan that provides for the safe transition from normal to emergency operations in accordance with the requirements of 5.27, Coordination of Emergency Response Planning?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.21(a)

Remarks:

2)

Does the certificate holder require that its safety policy be:

    In accordance with all applicable regulatory requirements in 14 CFR and must reflect the certificate holder’s commitment to safety (5.21(a));

    Be signed by the accountable executive described in 5.25 (5.21(b));

    Documented and communicated throughout their organization (5.21(c)); and

    Be regularly reviewed by the accountable executive to ensure it remains relevant and appropriate to the certificate holder (5.21(d))?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.21(b); 5.21(c); and 5.21(d)

Remarks:

1.2 - Safety Accountability and Authority

1)

Does the organization’s documentation define safety accountability for all organizational personnel, specifically:

    The accountable executive (described in 5.25);

    All members of management in regard to developing, implementing, and maintaining SMS processes within their area of responsibility; and

    Employees relative to the certificate holder’s safety performance?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.23(a)(1); 5.23(a)(2); 5.23(a)(3)

Remarks:

1.3 - Designation & Responsibility of Required Safety Management Personnel

1)

Does the certificate holder’s processes require that all members of management develop, implement, and maintain SMS processes within their area of responsibility to include, but not limited to:

    Hazard identification and safety risk assessment;

    Assuring the effectiveness of safety risk controls;

    Promoting safety as required in subpart E, Safety Promotion; and

    Advising the accountable executive on the performance of the SMS and on any need for improvement?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.23(a)(2)

Remarks:

2)

Do the certificate holder’s safety management processes identify the levels of management with the authority to make decisions regarding safety risk acceptance?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.23(b)

Remarks:

3)

Do the certificate holder’s safety management processes require the accountable executive to designate sufficient management personnel who, on behalf of the accountable executive, are responsible for:

    Coordinating implementation, maintenance, and integration of the SMS throughout the certificate holder’s organization;

    Facilitating hazard identification and safety risk analysis;

    Monitoring effectiveness of safety risk controls;

    Ensuring safety promotion is communicated throughout the certificate holder’s organization are required in subpart E, Safety Promotion; and

    Regularly reporting to the accountable executive on the performance of the SMS and any need for improvement?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.25(c)

Remarks:

1.4 - Coordination of Emergency Response Planning

1)

Where emergency response procedures are necessary, does the certificate holder develop and the accountable executive approve as part of the safety policy, an emergency response plan that addresses at least the following:

    Delegation of emergency authority throughout the organization;

    Assignment of employee responsibilities during the emergency; and

    Coordination of the emergency response plan with the emergency response plans of other affected organizations (e.g., code share partners, airports, contractors, affiliates, etc.)?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.21(a)(6); 5.27; 5.27(a); 5.27(b); 5.27(c)

Remarks:

1.5 - SMS Documentation

1)

Does the certificate holder have a process to develop and maintain SMS documentation that describes their safety policy, processes, and procedures?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.95(a); 5.95(b); 5.3(b)

Remarks:

Figure 17-4-3C.  SMS Safety Risk Management Design Validation

Certificate Holder Designator:

Date:

 

Process Area/Department Application:

 

SUPPLEMENTAL INFORMATION

Purpose: (Certificate Holder Responsibility) Incorporate a process within the SMS designed to identify, analyze, and assess the hazards to mitigate the associated risks.

Objective: (FAA Responsibility) Validate that the certificate holder has effectively designed an SMS which incorporates a process to identify, analyze, and assess the hazards to mitigate the associated risks.

Related Code of Federal Regulations (CFR): Safety Management System Voluntary Program (SMSVP) Standard 5.51 through 5.55.

Related FAA Policy/Guidance: Advisory Circular (AC) 120-92, Safety Management Systems for Aviation Service Providers.

 

2.0 - Safety Risk Management

2.1 Applicability

1)

Does the certificate holder’s SMS require that the organization apply the Safety Risk Management (SRM) process when any of the following conditions occur:

    Implementation of new systems;

    Revision of existing systems;

    Development of operational procedures; and

    Identification of hazards or ineffective risk controls identified through the safety assurance processes contained in the SMSVP Standard subpart D.

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.51(a), (b), (c), and (d)

Remarks:

2)

Does the certificate holder’s SMS define safety accountability for members of management, within their areas of responsibility and authority, regarding development, implementation, and maintenance of hazard identification and risk assessment processes?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.23(a)(2)(i)

Remarks:

3)

Does the certificate holder’s SMS identify management personnel responsible to facilitate hazard identification and safety risk analysis?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.25(c)(2)

Remarks:

2.2 - System Analysis and Hazard Identification

2.2.1  Process - System Description and Analysis

1)

When applying SRM, does the certificate holder have a process to describe and analyze the system for use in identifying hazards considering the following information:

    The function and purpose of the system;

    The system’s operating environment;

    An outline of the system’s processes and procedures; and

    The personnel, equipment, and facilities necessary for operation of the system?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.53(a) and (b)

Remarks:

2.2.2  Process - Hazard Identification

1)

Does the certificate holder’s SRM process(es) include specific processes to identify hazards within the context of the system analysis?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.53(c)

Remarks:

2.3 - Safety Risk Assessment and Control

2.3.1  Process - Analyze Safety Risk

1)

Does the certificate holder’s SRM include specific processes to analyze safety risk associated with hazards identified in 5.53(c)?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.55(a)

Remarks:

2.3.2  Process - Safety Risk Assessment

1)

Does the certificate holder’s SRM include specific processes for conducting risk assessment that allows for the determination of acceptable safety risk?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.55(b)

Remarks:

2)

Does the certificate holder’s SRM documentation clearly identify the levels of management with the authority to make decisions regarding safety risk acceptance for the company?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.23(b)

Remarks:

2.3.3  Process - Safety Risk Control

1)

Does the certificate holder’s SRM include specific processes to ensure that risk controls are developed which are necessary as a result of the safety risk assessment process?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.53(a), 5.55(c)

Remarks:

2)

Does the certificate holder evaluate, prior to SRM risk control implementation, that the identified risk will be acceptable with the risk control applied?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.55(d)

Remarks:

3)

Does the certificate holder’s risk management process evaluate the effectiveness of implemented safety risk controls, which includes reviews by the accountable executive?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.73(a)(3)

Remarks:

2.4 - SMS Documentation and Recordkeeping

1)

Does the certificate holder have a process to develop and maintain SMS documentation that describes their SRM processes and procedures?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.95(b)

Remarks:

2)

Does the certificate holder’s SMS require the organization have a process to maintain records of their SRM outputs for as long as the control(s) remain relevant to their operation, to include:

    Records of identified hazards or no hazard risk acceptance;

    Records of associated risks with identified hazards, as applicable;

    Records of analysis for each risk, as applicable; and

    Records of new risk controls approved to mitigate unacceptable risks, as applicable?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.97(a)

Remarks:

Figure 17-4-3D.  SMS Safety Assurance Design Validation

Certificate Holder Designator:

Date:

 

Process Area/Department Application:

 

SUPPLEMENTAL INFORMATION

Purpose: (Certificate Holder Responsibility) Incorporate processes that ensure effective safety risk controls which meet or exceed safety objectives through the collection, analysis, and assessment of data.

Objective: (FAA Responsibility) Validate that the certificate holder has effectively designed processes that ensure effective safety risk controls which meet or exceed safety objectives through the collection, analysis, and assessment of data.

Related Code of Federal Regulations (CFR): Safety Management System Voluntary Program (SMSVP) Standard 5.71 through 5.75.

Related FAA Policy/Guidance: Advisory Circular (AC) 120-92, Safety Management Systems for Aviation Service Providers.

 

3.0 - Safety Assurance

3.1 - Safety Performance Monitoring and Measurement

1)

Does the certificate holder’s SMS have processes to acquire and monitor data within the operational environment to detect changes related to the safety performance of the organization including:

    Products and services; and

    Operational processes?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.71(a)(1); 5.71(a)(2)

Remarks:

3.1.1  Process - Auditing Operational Processes & Systems

2)

Does the certificate holder’s SMS have processes to audit the safety performance of its operational processes, systems, products, and services?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.71(a)(3)

Remarks:

3.1.2  Process - Evaluations of SMS, Operational Processes & Systems

3)

Does the certificate holder’s SMS have processes to evaluate the safety performance of its operational processes, systems, products, and services?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.71(a)(4)

Remarks:

3.1.3  Process - Investigations of Incidents, Accidents & Potential Noncompliance

4)

Does the certificate holder’s SMS have processes to investigate its operational processes, systems, products, and services that include:

    Incidents and accidents; and

    Reports regarding potential noncompliance or other safety risk controls established in subpart C?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.71(a)(5); 5.71(a)(6)

Remarks:

3.1.4  Process - Confidential Employee Reporting System

5)

Does the certificate holder’s SMS have a confidential reporting system(s) to monitor safety performance that allows employees to:

    Report hazards, issues, concerns, occurrences, and incidents; and

    Propose solutions and safety improvements?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.71(a)(7)

Remarks:

Indicates new/changed information.

3.1.5  Process - Analysis of Data

6)

Does the certificate holder’s SMS have procedures to analyze data acquired from their safety assurance processes described in 5.71(a)(1) through (7), and any other relevant data with respect to its operations, products, and services, including at a minimum:

    Monitoring of operational processes;

    Monitoring of the operational environment to detect changes;

    Auditing of operational process and systems;

    Evaluations of the SMS and operational processes and systems;

    Investigations of incidents and accidents;

    Investigations of reports regarding noncompliance with regulations or risk controls established under subpart C, SRM; and

    Confidential safety reporting from employees on hazards, concerns, incidents, etc.?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.71(b)

Remarks:

3.2 - Process - Safety Performance Assessment

1)

Does the certificate holder’s SMS require the organization to regularly review and report on the system’s safety performance and does the Accountable Executive review these reports to:

    Ensure compliance with their established safety risk controls;

    Evaluate the performance of the SMS;

    Evaluate the safety risk control effectiveness established under 5.55(c) with identification of ineffective controls;

    Identify changes in the organization’s operational environment that may introduce new hazards; and

    Identify new hazards?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.25(b) and (c); 5.73(a); 5.75

Remarks:

Indicates new/changed information. Indicates new/changed information. Indicates new/changed information. Indicates new/changed information.

2)

Does the certificate holder’s organization define accountability for assuring the effectiveness of safety risk controls for all managers in their areas of responsibility?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.23(a)

Remarks:

3)

Does the certificate holder’s SMS designate management personnel who, on behalf of the accountable executive, are responsible for:

    Coordinating implementation, maintenance, and integration of the SMS throughout their organization;

    Facilitating hazard identification and safety risk analysis;

    Monitoring the effectiveness of safety risk controls;

    Ensuring safety promotion throughout their organization as required in subpart E; and

    Regularly reporting to the accountable executive on the performance of the SMS and on any need for improvement?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.25(c)

Remarks:

3.3 - Continuous Improvement

1)

Does the certificate holder have a process to ensure that the accountable executive directs actions necessary to address substandard safety performance in the system?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.25(b); 5.75

Remarks:

3.4 - SMS Documentation and Recordkeeping

1)

Does the certificate holder have a process to develop and maintain SMS documentation that describes their safety assurance processes and procedures?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.95(b)

Indicates new/changed information.

Remarks:

2)

Does the certificate holder’s SMS contain a process to maintain records of their safety assurance process outputs for a minimum of 5 years?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.3(b); 5.97(b)

Remarks:

3)

Do the certificate holder’s processes and procedures ensure that for ineffective risk controls or hazards identified during safety performance assessments, safety risk management is applied as described in subpart C?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.73(b)

Remarks:

Figure 17-4-3E.  SMS Safety Promotion Design Validation

Certificate Holder Designator:

Date:

 

Process Area/Department Application:

 

SUPPLEMENTAL INFORMATION

Purpose: (Certificate Holder Responsibility) Incorporate a combination of training and communication of safety information to support the implementation and operation of an SMS in an organization.

Objective: (FAA Responsibility) Validate that the certificate holder has effectively designed an SMS that incorporates training and communication of safety information throughout the organization.

Related Code of Federal Regulations (CFR): Safety Management System Voluntary Program (SMSVP) Standard 5.91 through 5.93.

Related FAA Policy/Guidance: Advisory Circular (AC) 120-92, Safety Management Systems for Aviation Service Providers.

 

4.0 - Safety Promotion

4.1 - General Expectations

1)

Does the certificate holder’s SMS define accountability for all members of management to promote safety within their area of responsibility in regards to developing, implementing, and maintaining SMS processes?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.23(a)(2)(iii)

Remarks:

4.2 - Competencies and Training

1)

Does the certificate holder’s SMS provide training to each individual identified in 5.23 that ensures the individuals attain and maintain the competencies necessary to perform their duties relevant to the operation and performance of the SMS?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.23(a), 5.91

Remarks:

2)

Does the certificate holder’s SMS specify that the accountable executive designate management personnel who, on behalf of the accountable executive, ensure that safety is promoted throughout the organization as required by subpart E?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.25(c)(4)

Remarks:

4.3 - Safety Communication

1)

Does the certificate holder have a process to develop and maintain a means for communicating safety information that:

    Ensures employees are aware of the SMS policies, processes, and tools relevant to their responsibilities;

    Conveys hazard information relevant to the employee’s responsibilities;

    Explains why safety actions have been taken; and

    Explains why safety procedures are introduced or changed?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.93

Remarks:

4.4 - SMS Documentation and Recordkeeping

1)

Does the certificate holder have a process to develop and maintain documentation that describes the organization’s SMS processes and procedures?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.95(b)

Remarks:

2)

Does the certificate holder maintain employee records of all safety management-related training provided under 5.91 for each individual and retain such records for as long as the individual is employed by the certificate holder?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.3(b), 5.97(c)

Remarks:

3)

Does the certificate holder retain the records of all safety communications provided under 5.93 for a minimum of 24 consecutive calendar-months?

☐ Yes

☐ No

SMSVP Standard 03-09-2015: 5.3(b), 5.97(d)

Remarks:

Figure 17-4-3F.  SMS Safety Policy Design Demonstration

Certificate Holder Designator:

Date:

Design Job Aid Reference Policy

Process Area/Department Application:

Performance Objective:

Certificate Management Teams (CMT) need to validate, to the extent necessary, that the organization’s safety policy has been conveyed to employees throughout the organization to include:

    A safety reporting policy for employee reporting of safety hazards or issues;

    A policy that defines unacceptable behavior and conditions for disciplinary action;

    Safety accountability within the organization.

Directions:

There are five basic subobjectives associated with this design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

During design validation, the CMT reviewed and accepted the certificate holder’s safety policy. The CMT must now confirm that the certificate holder has communicated this policy to employees applying or supporting its technical operations. The CMT should substantiate:

1)    The certificate holder’s communication guidance is being followed; and

2)    The effectiveness of the certificate holder’s communication strategy (i.e., employees understand how they can directly support safety policy in their day-to-day work activities).

A certificate holder’s safety policy must also define its process for reporting “safety hazards or issues.” Safety policy validation can be undertaken during regularly scheduled surveillance activities, or independently. Validating safety policy reporting and communications procedures can be done by interviewing employees at all levels of an organization.

Criteria:

    The certificate holder’s process must effectively communicate its safety policy at all levels of the organization to existing, new, and temporary employees, as applicable.

    All levels of management should be aware of their responsibility and accountability for safety in their organization. Individual managers are responsible for developing, implementing, and maintaining SMS processes within their technical areas. Members of management must be aware of their accountability and competence at:

    Hazard identification and safety risk assessment;

    Assuring the effectiveness of safety risk controls;

    Promoting safety; and

    Advising the accountable executive on the performance of the SMS and any need for improvement.

    All employees at all levels must know what is acceptable and unacceptable behavior and conditions for disciplinary action.

    All employees at all levels must know or be able to find the process for safety hazard and issue reporting (employee reporting). Several validation samples of personnel actually completing a hard copy or electronic sample submission should be accomplished.

NOTE:  Processing identified hazards may be accomplished as a separate validation activity or as a part of the safety policy validation, if the certificate holder’s process is not complex. If the certificate holder uses a corrective or preventive action process to resolve hazard reports, the CMT may wish to review the hazard report processing when it validates the corrective or preventive action process. The CMT should determine that the record includes information on the source of the input (e.g., Hazard Reporting Process Department) (see Figure 17-4-3N, SMS Continuous Improvement Process Design Demonstration).

Validation Repeatability: It is recommended that this validation be repeated in as many technical operational areas as necessary to ensure that organization’s communications mechanisms effectively accomplish the stated objective of this test. It is further recommended that the CMT add these validations to its regular surveillance activities and not expend resources on independent “SMS only” validation work for an area/department, unless the area/department sampling demonstrates failure.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

SUPPLEMENTAL INFORMATION

Purpose: (Certificate Holder Responsibility) Implement a safety policy that includes the detection and reporting of unacceptable behavior and the conditions for the disciplinary action and accountability of the safety within their organization.

Objective: (FAA Responsibility) Confirm through design demonstration that the implementation of the safety policy has been effectively conveyed to all employees throughout the organization.

Data Collection Tool Questions Record of Results:                                                  YES      NO

Indicates new/changed information. Indicates new/changed information. Indicates new/changed information.

1

Do employees throughout the organization demonstrate awareness of their system for employee reporting of safety hazards or issues?

Note(s): The demonstration of employee awareness is assessed from employee interviews.

Ref: SMSVP Standard 03-09-2015, 5.21(a)(4)

 

 

2

Do employees throughout the organization demonstrate awareness of unacceptable safety behavior and conditions for disciplinary action?

Note(s): The demonstration of employee awareness is assessed from employee interviews.

Ref: SMSVP Standard 03-09-2015, 5.21(a)(5)

 

 

3

Do employees throughout the organization demonstrate awareness of their defined safety accountability (i.e., can they relate safety objective(s) to their job)?

Note(s): The demonstration of employee awareness is assessed from employee interviews.

Ref: SMSVP Standard 03-09-2015, 5.21(a)(1); 5.23(a)

 

 

Figure 17-4-3G.  SMS Emergency Preparedness/Response Design Demonstration

Certificate Holder Designator:

Date:

Design Job Aid Reference Policy

Process Area/Department Application:

Performance Objective:

Certificate Management Teams (CMT) need to validate, to the extent necessary, that the certificate holder can effectively transition from normal operations to emergency operations without compromising safety. A secondary objective is to ensure that managers in contact with other organizations also having emergency response plans (ERP) have documented evidence that their respective ERPs are coordinated.

Directions:

There are five basic subobjectives associated with this design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

The certificate holder’s ERP process was reviewed and accepted during CMT design validation. The CMT must now ensure these processes are effective by validating the certificate holder’s ability to move select individuals out of normal daily operations and that those operations continue to effectively function during their absence.

It is important to verify that when key decisionmakers are unavailable to fulfill their responsibilities, the certificate holder has position proxies or a backup plan to maintain the affected processes. The CMT must ensure that the organization’s “backup strategy” (people and processes) will work.

A certificate holder’s ERP documentation should identify substitutes for those that must participate in emergency activities and are unavailable to perform normal duties. The CMT may test:

1)    How the person is notified of their additional duties;

2)    That, as a proxy, they have the competencies (training) to perform the additional duties; and

3)    That the person is knowledgeable of these duties or can identify appropriate guidance required for performance.

The CMT will require evidence that the certificate holder is coordinating its ERP with other organizations’ emergency plans. Evidence of coordinated ERPs may be located in meeting minutes, documents, and/or supplier contracts. When the certificate holder documents its ERP coordination in proprietary documents (e.g., contracts, etc.) it may provide excerpts (redacted information) as proof of performance.

Criteria: The CMT uses a certificate holder’s ERP to identify several key samples for testing.

    That proxies for risk decisionmakers have been identified, that have been removed from normal operations to conduct emergency operations.

    The limitation of the authority of those proxies is defined.

    A proxy has the authorities and competencies (training) required by the organization to make safety‑related decisions for the process area (e.g., Safety Risk Management (SRM) activities, corrective action oversight, etc.).

    The organization shows satisfactory documentation that ERPs are coordinated with external business partners that have ERPs (e.g., code share partners, airports, etc.).

Validation Repeatability: Validations for proxies should be sampled using the ERP to identify process area samples. The CMT may test each process area individually during normal, routine surveillance or the CMT may perform a single “tabletop” activity to verify proxies’ knowledge of their duties and responsibilities. Certificate holder’s ERP coordination with external parties may be validated as a single activity if they have developed a common record repository.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

SUPPLEMENTAL INFORMATION

Purpose: (Certificate Holder Responsibility) Implement an ERP as necessary, without compromise to safety including documented organizational interfaces.

Objective: (FAA Responsibility) Confirm through design demonstration that the certificate holder can effectively transition from normal operations to emergency operations without compromising safety.

Data Collection Tool Questions Record of Results:                                                  YES      NO

1

Does the certificate holder clearly identify “proxies” and the assignment and limitations of their authority to perform safety management responsibilities when select individuals are moved from daily into emergency operations?

Note(s): A proxy is delegated emergency authority to represent and perform duties of an individual during their absence.

Ref: SMSVP Standard 03-09-2015, 5.27(a); 5.27(b)

 

 

2

Does the proxy understand their defined limitations and authority as documented by the certificate holder for instances where emergency authority is delegated?

Ref: SMSVP Standard 03-09-2015, 5.27(a)

 

 

3

Does the certificate holder have documentation that those identified with delegated authority (proxies) have the competencies (i.e., qualification, training, knowledge, and experience) required by the organization to make safety-related decisions for their process area (e.g., SRM activities, corrective action oversight, etc.)?

Ref: SMSVP Standard 03-09-2015, 5.91

 

 

4

Does the certificate holder have documentation that emergency response plans are coordinated with external business partners that have emergency response plans (e.g., code share partners, airports, etc.)?

Ref: SMSVP Standard 03-09-2015, 5.27(c)

 

 

Figure 17-4-3H.  SMS SRM (Process/Department Owner) Design Demonstration

Certificate Holder Designator:

Date:

Design Job Aid Reference Safety Risk Management

Process Area/Department Application:

Performance Objective:

Certificate Management Teams (CMT) need to validate, to the extent necessary, that those individuals or groups: accept supplier guidance materials into their process area; and/or have the authority to draft and approve new or revised procedural changes for their process area, can effectively apply the organization’s Safety Risk Management (SRM) process to those process procedures.

NOTE:  There is another validation test for the corporate level SRM Process (see Figure 17-4-3J, SMS SRM (Organizational) Design Demonstration). In this test, multiple process areas are affected and process owners must interact determining the perceived risks and mitigations (e.g., adding a new aircraft fleet, implementing new multifaceted software solution across process areas, etc.).

Directions:

There are five basic subobjectives associated with this design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

The certificate holder’s SRM process was reviewed and accepted during CMT design validation. Next, the CMT must ensure that process owners throughout the system can perform SRM. Since SRM is applicable only to “design change,” the CMT should evaluate how guidance documents, used by the certificate holder’s workforce, are revised. The CMT should identify those drafting and authorizing guidance document changes. During this evaluation the CMT may find that a manager does not always draft guidance changes or review supplier provided documents for acceptance into the system. It is, therefore, important to identify who is actually performing work associated with the SRM process steps and determine what risk acceptance authority they have under the certificate holder’s defined process.

Understanding that new or revision of personnel guidance is vital to SRM applications, the CMT should concentrate on the organization’s application of SRM in their document control and approval process. Realizing that any design change in these documents has an SRM recording requirement attached to it, it is important that the CMT validate:

1)    What person(s) is making decisions regarding design change for the process area;

2)    Is there documented evidence that this person(s) has been trained to perform these duties; and

3)    Who has been designated to sign off on the document (accept risk) for the process area?

In smaller organizations, a single person may have the authority to perform the entire SRM process steps. In larger organizations or organizations with complex process areas (e.g., maintenance department for large airlines, etc.), the authority to perform specific aspects of the SRM process may be delegated to subordinates. In these situations, the CMT needs to identify the first SRM decisionmaker (hazard identification) and trace the process up through the chain of command until the person authorized to “accept risk” (i.e., sign off the design change) is identified.

For SRM samples that result in the development of new controls added to a process procedure, there should be a documented record of the outputs for the following processes:

1)    Identified hazards;

2)    Associated risks for each hazard;

3)    Analysis for each risk; and

4)    Any new control(s).

Criteria:

    The person conducting an SRM required activity is given that authority by the certificate holder. Training is documented to demonstrate competency to perform the specified activity(ies).

    The required records for each required SRM activity are complete (minimum record is a “no hazard” signoff for new or revised process/procedural change). When decisionmakers identify risks and new controls, the required records are:

    List of hazards;

    List of risks associated with each hazard;

    Analysis of each risk; and

    Record of mitigation (controls).

    Escalation and Traceability when a single person is not responsible for all decisions related to the SRM process, the “decisionmaking chain of command” must be evaluated to ensure:

    Persons performing some, but not all, SRM process activities are authorized by the organization to do so and competent (trained) to perform those activities.

    Escalation interfaces of SRM activities from one level of process manager to a higher level of process manager allows a positive transfer to occur.

    Escalation of SRM process activities is traceable from one process owner to another.

    Transference of SRM process steps between authorized personnel is monitored to prevent failure of the transfer process.

Validation Repeatability: The CMT shall repeat the validations in all process areas and for as many process owners or process owner escalations as the CMT finds necessary to ensure full integration of the SRM process to the lowest levels of decisionmaking within a process area. Since SRM is one of the most critical SMS components, SRM process owner validation must be very thorough.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

SUPPLEMENTAL INFORMATION

Process Purpose: (Certificate Holder Responsibility) Implement safety risk of all safety‑critical processes at the process owner and/or department level.

Objective: (FAA Responsibility) Confirm through design demonstration that the certificate holder can effectively apply the organization’s Safety Risk Management (SRM) process to all safety-critical processes within the process owner’s department.

Data Collection Tool Questions Record of Results:                                                  YES      NO

1

Do individuals or groups that accept supplier guidance materials into their process area(s) understand that updates or changes to these materials requires safety risk management be conducted before it is used in the system?

Ref: SMSVP Standard 03-09-2015, 5.51

 

 

2

Do individuals or groups that have the authority to draft and approve new or revised process and procedural changes for their process area(s), understand their responsibility to conduct safety risk management on those changes/materials before they are used in the system?

Ref: SMSVP Standard 03-09-2015, 5.51

 

 

3

Does the certificate holder clearly define individuals or groups that are performing safety risk management process steps and accepting risk for the process area(s) being assessed?

Ref: SMSVP Standard 03-09-2015, 5.23(a) and (b)

 

 

4

Does the certificate holder have documentation showing the individuals who complete safety risk management-related process steps have the competencies (i.e., qualification, training, knowledge, and experience) to properly perform those activities?

Ref: SMSVP Standard 03-09-2015, 5.91

 

 

5

When the organization has identified hazards or ineffective risk controls, can the SRM process documentation be traced to ensure the following recording requirements are met:

    Record(s) of identified hazards or lack of hazards;

    A list of risks associated with each existing hazard;

    Analysis of each risk;

    Record of mitigation (controls) for unacceptable risks;

    Record of safety risk acceptance decision(s) by authorized individual/group; and

    Verification of safety risk control effectiveness prior to final risk acceptance?

Ref: SMSVP Standard 03-09-2015, 5.3(a)(2), 5.51(d), 5.73(a)(3)

 

 

Figure 17-4-3J.  SMS SRM (Organizational) Design Demonstration

Certificate Holder Designator:

Date:

Design Job Aid Reference Safety Risk Management

Process Area/Department Application:

Performance Objective:

Certificate Management Teams (CMT) must validate, to the extent necessary, the certificate holder’s process for conducting integrated Safety Risk Management (SRM) when multiple departments are affected by a system change.

NOTE:  This SRM test is not to be confused with a process owner/department level SRM, if the certificate holder defines different process steps for “multidepartment” SRM (see Figure 17-4-3H, SMS SRM (Process/Department Owner) Design Demonstration). It is highly recommended that process owner/department SRM be assessed before testing the corporate SRM process.

Directions:

There are five basic subobjectives associated with this design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

The organization’s corporate SRM process was reviewed and accepted during CMT design review validation. The CMT next ensures that process owners/department representatives can perform corporate level SRM.

The certificate holder has a process to identify hazards and associated risks, analyze risks, and develop new risk controls that affect multiple process owner/departments within its organization. SRM decisionmaking and recording requirements are the same as those for “process owner/department SRM,” except there are more complex interfaces between departments and require process owner/department leadership to coordinate the required risk mitigations. In addition, final risk acceptance for an organization may be made at a management level above the process owner or by a committee. It is important that the CMT understand and validate these differences between the corporate and process area SRM processes, as applicable.

The CMT will determine whether the corporate level interfaces allow for all required SRM activities to be completed and documented. The CMT will ensure that those conducting corporate SRM activities have the authority and competencies (training) required. It is recommended that corporate SRM validation follow the process owner SRM validations. This allows the CMT to identify how individual process owners process SRM risk decisions within their technical area before they participate in the “higher level” corporate SRM process.

The corporate level SRM performance validation test is one of two final validation tests jointly conducted by the CMT, SMS Program Office, and certificate holder.

Criteria:

    The person(s) conducting the corporate level SRM activities have been given the authority by the certificate holder and it is documented the person(s) are competent to perform the specified activity(ies).

    The records for each required SRM activity are complete.

    The certificate holder has included, through documented record, each process owner stakeholder who must contribute to a collective risk decision and their respective inputs have been recorded as required by the corporate SRM process (e.g., meeting minutes with attendance rosters, required process owner submissions attached to meeting minutes, etc.).

Validation Repeatability: This performance validation only needs to be conducted once. It is normally one of the last validation tests before the certificate holder’s SMS is accepted. The CMT and Safety Management System Program Officer (SMSPO) will perform this validation jointly. It is highly recommended that a corporate SRM test include as many process owner areas/departments as possible. If the test sample does not include all process owner areas, the CMT should require that all process owners/departments are represented during a test (i.e., during a tabletop exercise). This sequence allows the CMT and SMSPO to ask pertinent questions.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

SUPPLEMENTAL INFORMATION

Purpose: (Certificate Holder Responsibility) Integrate SRM across multiple departments when affected by changes to their environment/systems.

Objective: (FAA Responsibility) Confirm through design demonstration, the certificate holder is capable of conducting integrated SRM when multiple departments are affected by a system change.

Data Collection Tool Questions Record of Results:                                                  YES      NO

Indicates new/changed information. Indicates new/changed information. Indicates new/changed information.

1

When multiple departments are affected by a system change, is there clear documentation that affected process owners or their proxies participate in a collective (organizational) risk assessment?

Ref: SMSVP Standard 03-09-2015, 5.51

 

 

2

When the organization has identified hazards or ineffective risk controls, can the SRM process documentation be traced to ensure the following recording requirements are met:

    Record(s) of identified hazards or lack of hazards;

    A list of risks associated with each existing hazard;

    Analysis of each risk;

    Record of mitigation (controls) for unacceptable risks;

    Record of safety risk acceptance decision(s) by authorized individual/group; and

    Verification of safety risk control effectiveness prior to final risk acceptance?

Ref: SMSVP Standard 03-09-2015, 5.3(a)(2), 5.51(d), 5.73(a)(3)

 

 

3

Does the certificate holder have documentation showing the individuals or group who complete the organizational safety risk management-related process steps have the competencies (i.e., qualification, training, knowledge, and experience) to properly perform those activities?

Ref: SMSVP Standard 03-09-2015, 5.91

 

 

4

Does the certificate holder clearly document that the individual(s), who have the authority to accept risk for the organizational SRM process, are performing that responsibility?

Ref: SMSVP Standard 03-09-2015, 5.23(b), 5.55(b)

 

 

5

Is there documentation that certificate holder personnel have actively participated as required by the organization’s SRM process?

Note(s): Inputs can include meeting minutes with attendance rosters, required process owner submissions attached to meeting minutes, etc.

Ref: SMSVP Standard 03-09-2015, 5.55(b)

 

 

Figure 17-4-3K.  SMS Audit Process Design Demonstration

Certificate Holder Designator:

Date:

Design Job Aid Reference Safety Assurance

Process Area/Department Application:

Performance Objective:

Certificate Management Teams (CMT) need to validate, to the extent necessary, that the certificate holder is periodically conducting audits to assess process function against defined process requirements. The CMT must ensure that the organization uses competent auditors, their reviews are system-wide, and there is an effective process to identify and correct nonconformance.

Directions:

There are five basic subobjectives associated with this design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

The certificate holder is expected to conduct audits to monitor the system to ensure that it functions as designed. Audits should be conducted by personnel with requisite competencies in the process area being reviewed to ensure an in-depth and detailed audit is performed. Often, audits are conducted by auditors independent of the process area. However, an auditor that does not have detailed knowledge of the process requirements, and the intended outcomes, usually provides only obvious process nonconformance.

It is important that the CMT ensure that audits are performed on all operational processes and systems. It is also important that the certificate holder identifies the minimum baseline frequency of assessments to satisfactorily monitor the process area and may develop an audit schedule to facilitate this. However, the organization may elect to perform additional process audits for a variety of reasons (e.g., effectiveness validation of a corrective action, a mitigation activity for a risk being monitored, an independent assessment by the evaluations team, etc.).

The CMT must ensure that auditor-identified nonconformance items are acted upon. The CMT may confirm correction of the nonconformance by determining if the certificate holder is using a corrective action tracking log or other method. Whatever the certificate holder uses, the audit should not be closed out until nonconformance items are transferred to the appropriate resolution process.

Criteria:

    Each critical process area/department is within the scope of the audit process and there is a strategy or audit schedule for periodic monitoring to occur.

    Audits are conducted by qualified personnel with competencies in the audit areas.

    Audit findings of nonconformance are appropriately tracked and corrective or preventive action (negative trends), and any associated action plans, are appropriately closed out.

    Corrective or preventive actions resulting from audits are not closed without effectiveness verification by qualified personnel.

    Corrective or preventive actions resulting from audits were spot checked by the CMT to ensure all proposed actions were implemented prior to closing the action. The CMT should choose as many verification samples as it feels appropriate to ensure process owners are following through on proposed actions. Often a CMT will choose its sampling based on identified process risks or process criticality.

    For corrective or preventive actions resulting from audits that identify a procedural change, there must be appropriate objective evidence of SRM being conducted (see Figure 17-4-3H, SMS SRM (Process/Department Owner) Design Demonstration).

Validation Repeatability: The CMT may wish to assess the completeness of the audit process as a single validation activity if a specified person or group in the organization manages the audit program. If validated in this manner, the CMT may pick specific audit findings of nonconformance for multiple process owner areas to validate the audit process from the data collection phase through the correction phase.

The organization’s audit process should require individual process owners to conduct their own internal process audits. The CMT should validate the completeness of the process owner audits using multiple validation activities (process owner by process owner).

Regardless of how audits are organized, it is recommended that the certificate holder’s audit outputs be compared against the CMT assessments to discern whether the audit yielded outputs “equal to” or “better than” the CMT assessment outputs. The certificate holder’s audits should always be more thorough than that of an external assessor, including those of the FAA.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

SUPPLEMENTAL INFORMATION

Purpose: (Certificate Holder Responsibility) Perform periodic audits to assess process performance against defined process requirements, and process nonconformance identification and correction procedures.

Objective: (FAA Responsibility) Confirm through design demonstration that the certificate holder is periodically performing audits to assess process performance against the defined requirements.

Data Collection Tool Questions Record of Results:                                                  YES      NO

Indicates new/changed information. Indicates new/changed information. Indicates new/changed information.

1

For the process area being assessed, is the certificate holder completing its planned audits on safety processes to gather data for use in assessing system performance?

Ref: SMSVP Standard 03-09-2015, 5.71(a)(3)

 

 

2

Are the certificate holder’s process area audits being conducted by personnel who have the identified competencies (i.e., qualification, training, knowledge, and experience) to appropriately assess the assigned process?

Ref: SMSVP Standard 03-09-2015, 5.91

 

 

3

Do the certificate holder’s audit findings clearly identify conformances and non-conformances?

Ref: SMSVP Standard 03-09-2015, 5.71(a)

 

 

4

When certificate holder non-conformances exist, are they appropriately assigned and corrected?

Ref: SMSVP Standard 03-09-2015, 5.71(a)

 

 

Figure 17-4-3L.  SMS Evaluation Process Design Demonstration

Certificate Holder Designator:

Date:

Design Job Aid Reference Safety Assurance

Process Area/Department Application:

Performance Objective:

Certificate Management Teams (CMT) need to validate, to the extent necessary, that a person or group within the certificate holder organization is analyzing aggregate data to measure and evaluate process area performance. These evaluations must include the status of defined organizational objectives and the status of process owner compliance with required safety management activities. Evaluations are independently reported to executive management.

Directions:

There are five basic subobjectives associated with design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

The certificate holder is expected to conduct evaluations to monitor performance across the system. Evaluations should be conducted by an individual or team independent of the process owners/department managers. Evaluations should target aggregate data from multiple data sources including: results of audits, trend data from department records generated, records required to measure progress towards defined safety objectives, corrective action/preventive action effectiveness, observations, or any other relevant data.

The individual or group should have unrestricted access to executive management as an independent reporting source. The CMT will assess the certificate holder’s ability to manage safety through independent evaluations of processes and activities. It is important that the CMT understand the inputs used for evaluations to ensure that evaluations are being appropriately applied across the system.

Criteria:

    Ensure each process area/department is within the scope of the evaluations process.

    Ensure that the evaluation person/team reports to executive management independent of process owner/department management to validate process performance claims by those managers.

    Ensure that evaluation reports assess whether the organization is meeting its safety objectives.

    An effective evaluation process should consider the following inputs:

    Results of audits;

    Results of investigations;

    Results of corrective or preventive actions to include effectiveness evaluations;

    Results of actions directed by executive management reviews;

    Results of continuous monitoring activities directed by process owners;

    Results of hazard reporting; and

    Results of new control effectiveness that were implemented by process owners since the last evaluations reporting period.

Validation Repeatability: The CMT may wish to validate the completeness of the evaluations process as a single validation activity after all SMS expectations have been implemented system-wide and this data is available for evaluation. This ensures that there is enough aggregate data from all process owner areas to ensure evaluation completeness. Once the CMT is confident that evaluations are being conducted system-wide, it may only be necessary to validate one evaluation. The CMT should review how the results of the evaluations are reported to executive management (reporting mechanism) and how the certificate holder ensures repeatability (e.g., a management review type process which may include evaluation reports, etc.).

Conversely, the CMT may wish to conduct several validation activities if they determine that independent process area evaluations reviews would offer greater flexibility to the CMT during the validation process.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

SUPPLEMENTAL INFORMATION

Purpose: (Certificate Holder Responsibility) Measure, evaluate, and report to executive management process area data on performance and compliance of required safety management activities.

Objective: (FAA Responsibility) Confirm through design demonstration that the certificate holder measured, evaluated and reported to executive management, the process area data on performance and compliance of required safety management activities.

Data Collection Tool Questions Record of Results:                                                  YES      NO

1

Does the certificate holder conduct evaluations to monitor safety-related performance across its systems and operational processes?

Ref: SMSVP Standard 03-09-2015, 5.71(a)(1)

 

 

2

Does the certificate holder review and analyze the aggregate data acquired from various safety assurance input sources such as:

    Audits;

    Investigations;

    Corrective/preventive actions including effectiveness evaluations;

    Actions directed by executive management reviews;

    Continuous monitoring activities directed by process owners;

    Hazard reporting; and

    New control effectiveness after implementation?

Ref: SMSVP Standard 03-09-2015, 5.71(b)

 

 

3

Do the certificate holder’s evaluation reports assess whether the organization is meeting its defined safety objectives?

Ref: SMSVP Standard 03-09-2015, 5.73(a)

 

 

4

Does the person/team who performs safety evaluations within the certificate holder’s organization report directly to executive management to independently validate process area safety performance?

Note(s): These evaluations are to be separate from process owner/department management reports.

Ref: SMSVP Standard 03-09-2015, 5.23(a)(2)(iv), 5.25(c)(5)

 

 

Figure 17-4-3M.  SMS Investigation Process Design Demonstration

Certificate Holder Designator:

Date:

Design Job Aid Reference Safety Assurance

Process Area/Department Application:

Performance Objective:

Certificate Management Teams (CMT) need to validate, to the extent necessary, that those persons/positions assigned to conduct investigations of incidents and accidents are capable of performing those duties. The CMT will determine if a certificate holder’s investigation process follows a formal process to collect and analyze target specific data (e.g., accidents, incidents, regulatory violations, etc.). The CMT will assess if the process determines causal factors and develop process corrections, as necessary, to correct system deficiencies and improve the safety performance of the organization.

Directions:

There are five basic subobjectives associated with design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

The steps of an investigations process is not substantially different than the process steps associated with a certificate holder’s corrective action processes. Investigations are focused on defined events (e.g., accident, incident, etc.) and may require special data collection activities to aid process owners in their analysis and subsequent corrective actions (e.g., one investigatory practice may include interview information from event witnesses). The required investigation process steps should be defined by the organization in its guidance documents. The CMT only needs to ensure that personnel, authorities, competencies, and process steps are understood and/or demonstrated in defined accident or incident documentation.

Therefore, it is important that investigation records identify “who” conducted certain activities so the CMT can validate authorities and competencies of those individuals.

Criteria:

    The investigation process steps should be understood by those persons/positions defined by the organization.

    Any accident or incident investigation process steps should be completed using actual samples.

    The person/position responsible to complete the investigation includes any documentation required by the certificate holder.

    Investigations are implemented in a timely manner to preserve evidence associated with the event.

    Any investigation activities requiring an interface with other processes used to maintain system integrity (e.g., SRM, Preventive Action/Preventive Action, Voluntary Self Disclosure, etc.) are complete and traceable to the associated investigation.

    Investigations should not be fully closed until the certificate holder has validated all required actions required by the certificate holder investigation process were implemented.

    Required actions must be evaluated for effectiveness before the investigation is considered complete (determine whether system deficiencies have been corrected to improve the safety performance of the organization).

Validation Repeatability: The CMT may wish to validate the investigations process by selecting samples from completed accident or incident investigation records to ensure process steps were completed by authorized personnel. The CMT may wish to combine the investigations process validation with other, similar, corrective action processes or independently validate the investigations process. If a specific person or team coordinates investigations for the entire organization, the validation may be completed as a one-time event by interviewing the coordinator and reviewing documentation samples. If the certificate holder identifies multiple process owners as having investigation authority, more samples may be warranted. To save time, the CMT may wish to perform a tabletop exercise with parties responsible to conduct investigations on behalf of the certificate holder and then sample associated records as a separate validation event.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

SUPPLEMENTAL INFORMATION

Purpose: (Certificate Holder Responsibility) Implement a formal process for investigating incidents and accidents including determination of causal factors and a process for developing corrective actions to improve the safety performance of the organization.

Objective: (FAA Responsibility) Confirm through design demonstration that the formal process for investigating incidents and accidents determines causal factors and develops corrective actions to improve the safety performance of the organization.

Data Collection Tool Questions Record of Results:                                                   YES     NO

1

Do personnel that conduct investigations of incidents, accidents or other certificate holder defined events have the competencies (i.e., qualification, training, knowledge, and experience) to perform their safety management‑related duties and responsibilities?

Ref: SMSVP Standard 03-09-2015, 5.71(a)(5)

 

 

2

Do personnel that are qualified to conduct investigations of incidents, accidents, or other certificate holder defined events follow the organization’s process to collect and analyze investigatory data?

Ref: SMSVP Standard 03-09-2015, 5.71(a)(5)

 

 

3

Are corrective actions resulting from the investigatory process being evaluated for effectiveness (i.e., determine whether system deficiencies and ineffective controls have been corrected to improve the safety performance of the organization)?

Note(s): Before the investigation is considered complete, system deficiencies and ineffective controls must be corrected.

Ref: SMSVP Standard 03-09-2015, 5.73(a)(3), 5.75

 

 

4

As a result of an investigation leading to new or revised processes or procedures, does the certificate holder have clear documentation showing that the safety risk management process was completed prior to deployment into the system?

Ref: SMSVP Standard 03-09-2015, 5.51(a), (b), and (c)

 

 

Figure 17-4-3N.  SMS Continuous Improvement Process Design Demonstration

Certificate Holder Designator:

Date:

Design Job Aid Reference Safety Assurance

Process Area/Department Application:

Performance Objective:

Certificate Management Teams (CMT) need to validate, to the extent necessary, that technical process integrity is being managed to correct substandard safety performance by implementing corrective or preventive action when necessary. It is important that the CMT ensures that the certificate holder takes defined action when a process nonconformance has occurred or negative trends suggest a potential nonconformance will occur.

Directions:

There are five basic subobjectives associated with design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

The certificate holder is expected to monitor its system processes in a variety of ways (e.g., audit, evaluations, hazard reporting, investigations, daily/weekly/monthly record reviews, etc.). When any monitoring mechanism identifies actual or potential process failure, the certificate holder must take action to correct or prevent a nonconformance and maintain process integrity to its original design expectation. The CMT will validate that process owners responsible for these actions complete all the required process steps in the certificate holder’s process, provide proof of action implementation, and have not closed the action until an “effectiveness evaluation” has been completed.

The effectiveness evaluation should be defined by the process owner during the action determination phase of the process and should be documented on a tracking record to direct the follow-up evaluation. The effectiveness evaluation may be conducted by the process owner/proxy or another person/group/department in the organization that can understand the follow-up evaluation requirements.

Corrections and preventions should be closed in a timely manner. (“Timely” means that the organization has proof that they are actively moving toward resolution or they have set targeted objectives and recorded progress on those objectives.) Often lengthy corrections/preventions are associated with complex or expensive solutions. If noncomplex corrections and/or preventions are not making progress toward final solution, the CMT should discuss the issue(s) with the process owner to determine causes for the delays. It should be noted that the organization should implement temporary risk mitigations (e.g., cease an operation, use communication backup plans, perform frequent checks, etc.) until the final action plan is fully implemented. The CMT should also question the integrity of the temporary mitigations that were put in place until the corrective or preventive action is implemented.

One way to ensure an effective preventive/corrective action process is to use a tracking system. Some attributes and activities associated with an effective preventive/corrective action tracking process are as follows:

    The document used to track preventive/corrective action has sufficient “general information” to identify the input source (e.g., audit finding, employee report, etc.), date opened, unique tracking number for traceability reference, and the identification of the responsible process owner who will oversee the process activities, and other process owner interfaces.

    The tracking document provides the immediate actions used to “contain” the problem, allowing the process to continue functioning safely until a final solution is implemented.

    The tracking document provides a location to record root cause analysis associated with the process.

    The action plan is not closed without an effectiveness evaluation by qualified personnel.

    In addition to reviewing the status of a large sample of tracking documents for specific process owners/departments, specific action plans should be selected by the CMT representative to validate that all process steps identified in the action plan were fully implemented. There should be sufficient evidence to verify full implementation of the selected samples.

    For corrective or preventive actions leading to a process design change, there should be clear, traceable evidence to a completed Safety Risk Management (SRM) process record.

Criteria:

    The certificate holder must establish and implement processes to correct identified substandard safety performance.

Validation Repeatability: The CMT may decide to validate the corrective or preventive action process independently or add to a regularly planned assessment where records would be easily accessed. The CMT may also decide to perform the validation in two phases:

1)    Perform a high level validation of the corrective or preventive action process by thorough examination of associated records and validating signature authorities, process training records, and timely closure of the process action plans; and

2)    Select specific samples that require onsite validations and add these validation activities to regular surveillance activities for specific process owners/departments.

The CMT may decide to add this validation to selected, prescheduled, process area surveillance activities. During surveillance activity, the inspector should ask to see the process owner’s documentation as required by the certificate holder’s process (e.g., tracking records from audits, management review, employee reports, investigations, continuous monitoring, etc.) and complete the review defined in the previous paragraph. Basically, the only difference in this approach is the CMT’s preference as to how it wishes to initiate the assessment.

Regardless of technique, it is very important that the CMT performs enough validation activities to ensure the consistency of process owners “follow through” across the organization.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

SUPPLEMENTAL INFORMATION

Purpose: (Certificate Holder Responsibility) Manage technical process integrity through corrective or preventive actions, including current and future nonconformance.

Objective: (FAA Responsibility) Confirm through design demonstration that the certificate holder managed its technical process integrity through corrective or preventive actions, including current and future nonconformance.

Indicates new/changed information.

Data Collection Tool Questions Record of Results:                                                   YES     NO

Indicates new/changed information. Indicates new/changed information. Indicates new/changed information. Indicates new/changed information.

1

Is there clear documentation that the certificate holder’s management contribute mitigation strategies to correct negative safety trends or potential nonconformance within the system?

Note(s): Levels of organizational management can be found on an organizational chart.

Ref: SMSVP Standard 03-09-2015, 5.23(a), 5.25(a), (b), and (c), 5.75

 

 

2

Do the certificate holder’s members of management and other personnel have the competencies required by the organization to perform those functions required of them by the safety management system processes (i.e., qualification, training, knowledge, and experience)?

Ref: SMSVP Standard 03-09-2015, 5.23(a) 5.91

 

 

3

Does the certificate holder analyze the quality of all relevant data outputs of continuous improvement actions at the appropriate levels of the organization?

Ref: SMSVP Standard 03-09-2015, 5.71(b)

 

 

4

For corrective or preventive actions leading to new or revised process design, does the certificate holder have clear documentation showing that the safety risk management process was completed prior to deployment into the system?

Ref: SMSVP Standard 03-09-2015, 5.51, 5.55(c)

 

 

Figure 17-4-3P.  SMS Accountable Executive Review Design Demonstration

Certificate Holder Designator:

Date:

Design Job Aid Reference Policy

Process Area/Department Application:

Performance Objective:

Certificate Management Teams (CMT) must validate, to the extent necessary, that the certificate holder’s accountable executive is involved in the system-wide safety management efforts. The accountable executive must have adequate knowledge to play an active role in directing actions relevant to resolving safety performance deficiencies in the system.

Directions:

There are five basic subobjectives associated with design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

The certificate holder’s accountable executive was identified using a SMSVP job aid during design validation. The accountable executive is defined as a key leadership individual in the organization’s business tier that has ultimate authority over safety operations and organizational resources. As a result, it is important that the accountable executive is aware of safety performance data and information collected from the system so that he/she may direct any necessary actions and/or resources to support safety initiatives.

It is important that the accountable executive:

1)    Hold periodic meetings to review collected data and information to assess the safety performance of the organization;

2)    At a minimum, review key data/information inputs defined by the SMSVP Standard; and

3)    Direct appropriate action, as warranted.

Accountable executive directed actions should be processed in the same manner as other corrections made in system processes. These methods include corrective and/or preventive action, investigations, SRM process corrections, etc.

Criteria:

    The organization has a process for the accountable executive review (e.g., management review).

    Objective evidence can be obtained to support that executive management reviews are being performed.

    Management reviews should include those required by the accountable executive, but at minimum:

    Information on the effectiveness of safety risk controls (usually results from audits for each process owner/department, external audits, continuous monitoring outputs, voluntary disclosure reporting program, etc.).

    Information on the effectiveness of safety risk controls established since the last reporting period (these reports are usually the results of the effectiveness evaluations from corrective or preventive actions and SRM).

    Information on changes to operational environments and associated new hazards (e.g., things not in control of the certificate holder: regulatory changes, airport configuration changes, changes to approach or en route procedures, vendor status changes, etc.).

    Information on new hazards identified throughout the system through any safety assurance mechanism used by the organization.

    Other aggregate information, that relates to the effectiveness of the organization’s safety management efforts towards meeting its stated safety objectives.

NOTE:  Meeting minutes from accountable executive reviews are convenient recording locations for revalidation or edits to the organization’s safety policy. This record is sufficient evidence of a “signed safety policy,” which is required to be communicated throughout the organization.

Validation Repeatability: This validation need only be conducted once and as one of the final CMT validation process activities. However, this final test must be conducted with the SMSPO. It is important that the certificate holder has completed full SMS implementation, so it can define what system reports are appropriate for the management review process(es). Finally, the accountable executive must take appropriate action to address any substandard safety performance. This validation may be repeated if the certificate holder does not follow its defined process and the minimum data/information detailed above was not included during the CMT validation assessment.

NOTE:  It is often difficult to identify directed actions resulting from meeting minutes unless a template is used to list defined actions to be carried forward to the next management review. Using this technique removes the “guess work” associated with deciphering discussions contained in meeting minutes.

Certificate holder use of a template or “actions table” for the meeting minutes is strongly encouraged.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

SUPPLEMENTAL INFORMATION

Purpose: (Certificate Holder Responsibility) Designate an accountable executive who is involved in the system-wide safety management efforts.

Objective: (FAA Responsibility) Confirm through design demonstration that the certificate holder accountable executive has adequate knowledge and plays an active role in directing actions relevant to resolving safety performance deficiencies in the system.

Data Collection Tool Questions Record of Results:                                                  YES      NO

1

Does the certificate holder have documentation showing that the accountable executive is periodically reviewing and assessing the organization’s safety management performance?

Ref: SMSVP Standard 03-09-2015, 5.25(b)(5), 5.73

 

 

2

Does the certificate holder have documentation showing that the accountable executive directs actions to address substandard safety performance?

Ref: SMSVP Standard 03-09-2015, 5.25(b)(5)

 

 

3

Does the certificate holder have documentation showing the directives of the accountable executive are tracked and reported upon at the next regular review or as required?

Ref: SMSVP Standard 03-09-2015, 5.25(b), 5.73, 5.97

 

 

Figure 17-4-3Q.  SMS Records Retention Process Design Demonstration

Certificate Holder Designator:

Date:

Design Job Aid Reference Policy, Safety Risk Management and Safety Assurance

Process Area/Department Application:

Performance Objective:

Certificate Management Teams (CMT) will validate, to the extent necessary, that the organization has record retention capability conforming to the SMSVP Standard in either paper or electronic media. The ability of the organization to retrieve archived records shall be tested.

Directions:

There are five basic subobjectives associated with design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

The certificate holder is required to maintain records demonstrating conformance with applicable SMSVP standards and provide historical reference documents for ongoing decisionmaking.

The CMT must ensure that the certificate holder is capable of storing data for the required time periods defined in the SMSVP Standard and those required to retrieve stored data can do so in a “timely” manner. For paper records, access, protection from damage and misfiling are components of a good process. For electronic records, access, backup and protection from loss or overwrite are components of a good process. The CMT should test the certificate holder’s record systems by requesting evidence that stored historical data matches the maximum retention period requirement.

For example: If today’s date is 12/01/13 and there is a 24-month retention requirement, the certificate holder should be able to produce records from 12/01/11. If today’s date is 12/01/18 and the retention requirement is unlimited, then records must be accessible back to the initial date of creation.

NOTE:  If there is no “master record tracking document” defining the initial inception date of record, there is no standard to measure the historical completeness of a given record.

It is difficult to determine if something is missing from recorded history if one does not know what is supposed to be in the historic file in the first place. Therefore, the CMT must always identify the evaluation standard it will use to test the records retention system before examining individual records or files.

For example: For personnel records, the CMT should pick individuals from actual surveillance activities to determine required training from the sample. Since training records are required to be retained as long as the individual is employed, ask the management representative for employee records of individuals who are working in the process area. If employees have SMS training modules in their job description, this should be documented in a training matrix detailing the requirement and process area. The CMT, for process procedure records, should identify a specific process for assessment with a revisions log or document history. The CMT evaluator should then check the archived documents by composition or approval date to validate the document retention requirement.

Criteria:

    “Unlimited” record retention requirement: records of SRM outputs for as long as the control remains relevant to the operation (i.e., each revision level of a process procedure should have SRM records from the date of original SMS acceptance). Employee competencies and training records must be retained as long as the individual is employed.

    Five-year record retention requirement: Safety Assurance outputs (e.g., investigations, audits, corrective and preventive action, continuous process monitoring records (whether by day, week, or month) and employee hazard reports).

    Twenty-four-month record retention requirement: Safety communications, (e.g., the “why” documentation that includes bulletins, training records/curricula, records of corrective or preventive actions that require retraining of employees, meeting or briefing notes where “why” is explained, checklists of items reviewed at production meetings, etc.).

NOTE:  While it is commendable that a certificate holder can control its documents in an orderly fashion, if records are not being used for their intended purpose, then the records retention process is just a compliance drill. To prevent this, the CMT should ensure training records are periodically audited by the certificate holder to validate that its training process is working. When SRM is conducted, records from past SRM decisions should be reviewed as part of the analysis process.

Validation Repeatability: This validation is applicable to all process areas. In a large organization, the CMT may wish to select specific samples from process area subgroups and perform a one-time check; applying those samples to all record media used by that process group.

If the organization is smaller, management often requires its individual process owners to maintain records applicable to their area/department. In these situations, the CMT may wish to perform multiple validations on process areas with several process owners.

Regardless of the certificate holder’s size, it is important the CMT identify the records custodian(s) and perform enough validation activities to feel confident in the certificate holder’s ability to meet the SMSVP records retention requirements.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

SUPPLEMENTAL INFORMATION

Purpose: (Certificate Holder Responsibility) Implement a record retention process to comply with all regulatory record requirements.

Objective: (FAA Responsibility) Confirm through design demonstration that the certificate holder has a record retention process that complies with all regulatory record requirements.

Data Collection Tool Questions Record of Results:                                                  YES      NO

1

Did the certificate holder’s personnel adequately demonstrate that they can retrieve required safety management records (both current and historical) as defined in their records process to include:

    Safety risk management outputs as long as the control remains relevant to the operation (5.97(a));

    Five-year record retention requirement for the outputs of its safety assurance processes (5.97(b));

    Record of training for each individual to be retained for as long as they are employed by the certificate holder (5.97(c)); and

    Twenty-four-calendar-month record retention requirement for safety communications (5.97(d))?

Ref: SMSVP Standard 03-09-2015, 5.97

 

 

Figure 17-4-3R.  SMS Safety Communications Design Demonstration

Certificate Holder Designator:

Date:

Design Job Aid Reference Safety Promotion

Process Area/Department Application:

Performance Objective:

Certificate Management Teams (CMT) will validate, to the extent necessary, that the certificate holder has communicated safety information throughout its organization to ensure that employees are aware of their safety‑related responsibilities, and other critical safety-related information.

Directions:

There are five basic subobjectives associated with design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

The certificate holder is required to communicate safety-related information to its employees.

The SMSVP Standard identifies three communications requirements:

1)    Communications between management and employees ensures awareness of their specific safety management duties and responsibilities (e.g., employee guidance documents, manuals, training records and curricula, bulletins, etc.).

2)    Communications between management and employees resulting from identified hazard information that impacts specific employee groups (e.g., bulletins, production meetings, training records and curricula, etc.).

3)    Communications explaining why safety actions were taken to include why the addition of new controls or imposed corrective actions were implemented to correct process nonconformance or negative trends.

When a new process or procedural control is implemented, the affected employees (revised procedure) need to know why the new control was implemented. In other words, employees affected by the change should understand the basic objectives of the new control. By communicating the “why” behind a change, employees are better able to help management reach the proposed objectives.

NOTE:  The intent of this requirement is reinforcing to the certificate holder that it cannot expect employees to support desired outcomes if they don’t know what they are. Employees will often not remember the “why” when questioned about a changed process but should be aware they contribute to the overall safety of their organization. The CMT should also question the integrity of temporary mitigations before the mitigation is implemented. The CMT will have to sample enough employees to assess whether it believes the organization’s communication method is effective and meets the intent of the SMSVP communications requirement.

Criteria:

    The organization’s process must ensure that all employees throughout the organization are aware of the safety management system.

    The organization’s process must ensure that any safety-critical information is conveyed to the appropriate lines of business.

    The organization must have a process that ensures that an explanation is communicated to employees on why particular safety actions are taken.

    The organization must have a process that ensures that an explanation is communicated to employees on why a safety procedure is introduced or changed.

Validation Repeatability: This validation is applicable to all process areas. The CMT may wish to select samples from process area subgroups in a large organization and perform a one-time check to access communication media used by the certificate holder. In a smaller organization all communication may be company-wide. Communications in a small, medium, or large organization may be in the form of newsletters, safety bulletins, training media, meetings, etc.

Regardless of the certificate holder’s size, it is important that the CMT identify the processes used for communicating safety information and performs enough validation activities to feel confident in the organization’s ability to meet the SMSVP communications requirement.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

SUPPLEMENTAL INFORMATION

Purpose: (Certificate Holder Responsibility) Implement a process for communicating safety-critical information throughout its organization to ensure that employees are aware of their safety-related responsibilities.

Objective: (FAA Responsibility) Confirm through design demonstration that the certificate holder communicates safety information throughout its organization to its employees, including their safety-related responsibilities, and other critical safety-related information.

Data Collection Tool Questions Record of Results:                                                  YES      NO

Indicates new/changed information. Indicates new/changed information.

1

Does the certificate holder demonstrate that:

    Safety-critical information is communicated at appropriate personnel levels; and

    Employees have received an explanation as to why particular company safety actions are taken (i.e., new or revised policies/procedures or changes that impact their working conditions)?

Note: The demonstration of employee awareness is assessed from employee interviews and documentation.

Ref: SMSVP Standard 03-09-2015, 5.93(a), (b), (c), and (d), 5.97(d)

 

 

2

Does the certificate holder’s safety communication process explain to employees: safety policies, processes, procedures and actions relevant to their responsibilities?

Note: The demonstration of employee awareness is assessed from employee interviews.

Ref: SMSVP Standard 03-09-2015, 5.93

 

 

Figure 17-4-3S.  Transitioning from SMS Pilot Project to the SMS Voluntary Program

SMS Pilot Project (SMSPP) and the SMS Voluntary Program (SMSVP). Since 2007, the SMSPP has provided the FAA and certificate holders significant experience and lessons learned for good safety management implementation strategies.

Establishing a permanent way for certificate holders to have their SMS integrated into day-to-day operations or recognized for international operations is a logical evolution of the SMSPP. For that reason, the Flight Standards Service National Field Office (AFS-900) has created the SMSVP. As a result, all current SMSPP participants are automatically in the SMSVP, unless required by regulation to establish an SMS. In those cases, certificate holders will follow issued regulations and referenced advisory materials.

While certificate holders are “automatically” entered into the SMSVP, their SMS implementation efforts must correspond to the SMSVP structure. The Safety Management System Program Office (SMSPO) and certificate management teams (CMT) will use design validations to measure progress. The SMSPO and certificate holders’ CMTs will work to ensure that progress is properly acknowledged and past work is not lost.

This figure describes the process that SMSPP participants will use to transition to the SMSVP. This process will be discontinued once all SMSPP participants have transitioned to the SMSVP. If a certificate holder does not wish to make this transition, they may withdraw from the SMSPP and any Flight Standards Service (AFS) acknowledgement letters will be null and void.

1.    Phase 1 Certificate holder’s Implementation Transition. Certificate holders will revise their implementation plans to the SMSVP Standard (Figure 17-4-3A). It is recommended that the certificate holder and CMT become familiar with the SMSVP Standard to realize the few differences between the Advisory Circular (AC) 120-92A Framework and the SMSVP Standard (see Figure 17-4-3T). After familiarization, the SMSPO recommends that the certificate holder take the following steps to revise its implementation efforts:

a.    The certificate holder should identify any new SMSVP expectations that are different from its original implementation plan conceived under AC 120-92A, Appendix 1, Aviation Service Provider Safety Management System Framework: Functional Expectations.

b.    The certificate holder should determine what changes or modifications/revisions will have to be implemented to meet the new expectations.

c.    The status of each expectation should be annotated on a revised Implementation Plan. This may be as simple as adding a “status” column to the existing plan and annotating whether the expectation:

i.    Has been met;

ii.   Requires revision; or

iii.  Remains in progress (the expectation is still under initial development).

d.    All remaining work on the revised implementation plan should include:

i.    Any revised manual references;

ii.   The person responsible; and

iii.  Anticipated completion dates for documentation and full implementation. These dates will be used by the CMT to develop its validation plan for expectations still under development.

e.    The certificate holder will re-submit its revised SMSVP Implementation Plan to the CMT, using the revision process formally agreed upon between the certificate holder and CMT.

NOTE:  While the certificate holder may develop its Implementation Plan in any form or manner it chooses, the plan must be acceptable to the CMT. Under the SMSVP, the plan must include dates that the certificate holder expects its documentation to be completed and target dates when documented requirements will be implemented into its system. The CMT will use these dates to develop its validation project plan required under the SMSVP (see Volume 17, Chapter 4, Section 2).

2.    CMT Review and Acknowledging of the Certificate holder’s Revised Implementation Plan. While the certificate holder’s revised plan will contain relatively few changes, the CMT verification activities will shift to design validations using the SMSVP validation tools. The following process steps will be used by the CMT to accomplish the transition:

a.    The CMT will use the attached “Bridging Document” to familiarize themselves with the changes between the AC 120-92A Framework and the SMSVP Standard. The CMT shall ensure that the certificate holder has revised its plan to address the appropriate SMSVP Standard references and has made a status determination for each requirement on the revised plan.

b.    The CMT will review the certificate holder’s status claim and decide if:

i.    The expectation has been met;

ii.   The expectation requires revision; or

iii.  The expectation remains “in-progress” (still under initial development).

c.    The CMT will provide the certificate holder written notification of any status disagreement and upon acceptable correction by the certificate holder, accept the revised plan as formal conversion to the SMSVP.

NOTE:  Although the SMSPO is the final authority on the SMSVP standards differences of opinion over revised plan suitability between the CMT and certificate holder may be referred to the SMS Regional Point of Contact (RPOC) for resolution. The CMT and/or RPOC may request assistance from the SMSPO to answer any technical questions, or request a meeting in facilitating the transition process.

From the certificate holder’s revised Implementation Plan, the CMT will develop its Validation Project Plan using the guidance contained in this chapter (see Volume 17, Chapter 4, Section 2, subparagraph 17-4-2-3E) and complete all remaining validation work in accordance with this document.

NOTE:  The CMT may request assistance from the SMSPO to assist with development of the validation project plan.

3.    PTRS Procedures. The person with “transition plan oversight” will open a PTRS record to record CMT completion of the SMS transition from SMSPP to SMSVP activities:

i.    Enter activity number 1045/3045/5045 as appropriate;

ii.   Enter “SMSVPIPT” (SMS Implementation Plan Transition) in the “National Use” box; and

iii.  Record any additional information in the Comments section, as required.

Figure 17-4-3T.  Bridging Document Differences Between AC 120-92A and the SMSVP Standard

The following table lists the differences between the Advisory Circular (AC) 120-92A, Appendix 1 Framework and the SMS Voluntary Program Standard.

The document is intended to assist CMT’s transition from the SMS Pilot Project to the SMSVP expectations. It may also be used by a certificate holder to assist in documenting changed expectations in its SMS Implementation Plan.

Disclaimer: FAA certificate-holding offices are not obligated to accept or reject a certificate holder submission using this document. The SMSVP Standard is the primary reference to be used in the SMSVP.

Primary Reference to be Used in the SMSVP.

SMSPP Process Based on AC 120-92A

SMSVP Standard Based on NPRM Part 5

Differences Between SMSPP Framework to SMSVP Standard

Redline Changes from Previous SMSVP Standard to Revised SMSVP Standard Based on Part 5 Final Rule

If a certificate holder has implemented or is implementing an SMS using AC 120-92A, Appendix 1, these are the processes they have developed or are developing:

Title 14 CFR part 5 is the foundation document for the SMSVP Standard, but the Standard, not 14 CFR part 5, is how participants are evaluated:

As follows are the noted differences between AC 120-92A and the SMSVP Standard. It is important to the CMT to focus on the processes impact of the change to ensure conformance to the Standard.

Words in bold are key words to focus your review.

No change

Component 1.0 Safety Policy and Objectives

SMSVP Standard 5.3(a)(1) and 5.21—5.27

  Ensure that the company designates an accountable executive and replaces the term “Top Management” with the term “Accountable Executive” in their manuals and documentation.

[Wording and management concept change]

5.3 was updated to read as follows:

(a) Any certificate holder required to have a Safety Management System under this Standard must submit the Safety Management System to the Administrator for acceptance. The SMS must be appropriate to the size, scope, and complexity of the certificate holder’s operation and include at least the following components:

(1) Safety policy in accordance with the requirements of subpart B of this Standard part

(2) Safety risk management in accordance with the requirements of subpart C of this Standard part;

(3) Safety assurance in accordance with the requirements of subpart D of this Standard part; and

(4) Safety promotion in accordance with the requirements of subpart E of this Standard part.

 

5.5 Definitions

 

Hazard means a condition that could foreseeably cause or contribute to an aircraft accident as defined in 49 CFR 830.2.

Element 1.1 Safety Policy

SMSVP Standard 5.21(a)(2) and 5.23

  Ensure that the company’s Safety Policy contains a commitment to fulfill the organization’s safety objectives.

[Bold text not addressed in AC 120-92A]

Changed to require signature by accountable executive.

(b) The safety policy must be in accordance with all applicable regulatory requirements in Chapter I of Title 14 of the Code of Federal Regulations and must reflect the certificate holder’s commitment to safety.

(b) The safety policy must be signed by the accountable executive described in 5.25.

(c) The safety policy must be documented and communicated throughout the certificate holder’s organization.

(d) The safety policy must be regularly reviewed by the accountable executive to ensure it remains relevant and appropriate to the certificate holder.

Element 1.1(2)(e)

SMSVP Standard 5.21(a)(4)

  Ensure that the company’s Safety Policy defines requirements (replaces “encourages”) for employee reporting of safety hazards or issues.

[Wording and process change. Review existing process to ensure conformance with the SMSVP Standard conformance.]

No change

Element 1.1(b)(2)(f)

SMSVP Standard 5.21(a)(5)

  Ensure that the company’s safety policy defines unacceptable behavior and conditions for disciplinary action.

[Change from AC 120-92A (Element 1.1b(2)(f)]

No change

Element 1.4

SMSVP Standard 5.21(a)(6)

  Ensure that the company’s Safety Policy contains an emergency response plan which provides for the safe transition from normal to emergency operations in accordance with the requirements of 5.27.

[Bold text not addressed in AC 120-92A]

No change

Element 1.1(2)(k)

SMSVP Standard 5.21(d)

  Ensure that the company’s Safety Policy requires regular reviews by the accountable executive (replaces “organization/company/etc.”) to ensure that it remains relevant and appropriate to the certificate holder.

[Wording and process change]

No change

Element 1.2

SMSVP Standard 5.23(a)(2)

  Ensure that the company’s Safety Policy defines management’s accountability for safety for SMS processeswithin their area of responsibility, including, but not limited to:

(i) Hazard identification and safety risk assessment.

(ii) Assuring the effectiveness of safety risk controls.

[Bold text not addressed in AC 120-92A]

No change

Element 1.2 Management Commitment and Safety Accountabilities

SMSVP Standard 5.23 and 5.25

  Ensure that the company has documentation that identifies an accountable executive who, irrespective of other functions, satisfies the following:

(1) Is the final authority over operations authorized to be conducted under the certificate(s).

(2) Controls the financial resources required for the operations to be conducted under the certificate(s).

(3) Controls the human resources required for the operations authorized to be conducted under the certificate(s).

(4) Retains ultimate responsibility for the safety performance of the operations conducted under the certificate.

[Bold text not addressed in AC 120-92A]

No change

Element 1.2(3)(a)

SMSVP Standard 5.25(b)(2)

  Ensure that the company requires the accountable executive to accomplish the development (replaces the term “define”) and sign the organization’s Safety Policy.

[Wording and process change]

No change

Element 1.2

SMSVP Standard 5.25(b)(5)

  Ensure that the company requires the accountable executive (replaces the term “management”) to assess the SMS performance, to review the safety performance and direct actions to address substandard performance.

[Wording and process change not addressed in AC 120-92A]

No change

Element 1.3 Key Safety Personnel

SMSVP Standard 5.25(c)

  Ensure that the company requires the accountable executive (replaces the term “top management”) must designate a management representative (replaces the term “a member of management”) who must be responsible for the following:

[Wording and process change]

(1) Facilitating hazard identification and safety risk analysis; and

(2) Monitoring the effectiveness of safety risk controls.

[Bold text not addressed in AC 120-92A]

Replaced management representative with management personnel and adjusted job responsibilities.

(c) Designation of management personnel. The accountable executive must designate sufficient management personnel who, on behalf of the accountable executive, are responsible for the following:

(1) Coordinate implementation, maintenance, and integration of the SMS throughout the certificate holder’s organization.

(2) Facilitate hazard identification and safety risk analysis.

(3) Monitor the effectiveness of safety risk controls.

(4) Ensure safety promotion throughout the certificate holder’s organization as required in subpart E of this Standard.

(5) Regularly report to the accountable executive on the performance of the SMS and on any need for improvement.

Element 1.4 Emergency Preparedness and Response

SMSVP Standard 5.27

  Where emergency procedures are necessary, the accountable executive and management representative must develop as part of the Safety Policy of the certificate holder, an emergency response plan that addresses at least the following:

(1) Delegation of emergency authority throughout the organization; and

(2) Assignment of employee responsibilities during the emergency.

[Bold text not addressed in AC 120-92A]

Changed to the following:

Where emergency response procedures are necessary, the certificate holder must develop and the accountable executive must approve as part of the safety policy, an emergency response plan that addresses at least the following:

Element 1.5 SMS Documentation and Records

SMSVP Standard 5.95 and 5.97

  Ensure that the company requires the following record retention times:

(1) Outputs of SRM must be retained as long as controls are relevant;

(2) Outputs of SA records must be retained for a minimum of 5 years;

(3) Training records must be retained for a minimum of 24 consecutive calendar-months; and

(4) Records of all communications provided under 5.93 for a minimum of 24 consecutive calendar-months.

[Bold text not addressed in AC 120-92A]

5.97(c) has been updated to read:

(c) The certificate holder must maintain a record of all training provided under 5.91 for each individual. Such records must be retained for as long as the individual is employed by the certificate holder.

Component 2.0 Safety Risk Management (SRM)

Subpart C, Safety Risk Management, SMSVP Standard 5.3(a)(2), SMSVP Standard 5.51, 5.53, and 5.55

Intentionally left blank.

Deleted some language for clarification.

A certificate holder must apply safety risk management to the following:

Element 2.1 Hazard Identification and Analysis

Intentionally left blank.

Intentionally left blank.

No change

Process 2.1.1 System Description and Task Analysis

SMSVP Standard 5.53(a) and (b), System Analysis and Hazard Identification

  Ensure that procedures are in place, when conducting the system analysis, to require consideration of:

(1) Function and purpose of the system.

(2) The system’s operating environment.

(3) An outline of the system’s processes and procedures.

(4) The personnel, equipment, and facilities necessary for operation of the system.

[Bold text not addressed in AC 120-92A]

5.53(a) changed as follows:

(a) When applying safety risk management, the certificate holder must analyze the systems identified in 5.51. Those system analyses must be used to identify hazards under paragraph (c) of this section, and in developing and implementing risk controls related to the system under 5.55(c).

Process 2.1.2 Identify Hazards

SMSVP Standard 5.53(c), System Analysis and Hazard Identification

No change noted.

No change

Element 2.2 Risk Assessment and Control

Intentionally left blank.

Intentionally left blank.

No change

Process 2.2.1 Analyze Safety Risk

SMSVP Standard 5.55(a), Safety Risk Assessment and Control

No change noted.

No change

Process 2.2.2 Assess Safety Risk

SMSVP Standard 5.55(b), Safety Risk Assessment and Control

The certificate holder must define a process for conducting risk assessment that allows for the determination of acceptable safety risk.

[Bold text not addressed in AC 120-92A]

5.55(b) updated by deleting the following sentence:

“Acceptable safety risk must, at a minimum, comply with the applicable regulatory requirements set forth in Chapter I of Title 14 of the Code of Federal Regulations.”

Process 2.2.3 Control/Mitigate Safety Risk

SMSVP Standard 5.55(c), Safety Risk Assessment and Control

The certificate holder must develop and maintain processes to develop safety risk controls that are necessary as a result of the safety risk assessment process under paragraph (b) of this section.

[Bold text not addressed in AC 120-92A]

Renumbered 5.55(c)(1) to 5.55(d).

Deleted:

5.55(c)(2) The safety risk controls must, at a minimum, comply with the applicable regulatory requirements set forth in Chapter I of title 14 of the Code of Federal Regulations.

Component 3.0 Safety Assurance

Subpart D, Safety Assurance, SMSVP Standard 5.3(a)(3), 5.71, 5.73, and 5.75

Intentionally left blank.

No change

Element 3.1 Safety Performance Monitoring and Measurement

Intentionally left blank.

Intentionally left blank.

No change

Process 3.1.1 Continuous Monitoring

SMSVP Standard 5.71(a)(1) and (2), Safety Performance Monitoring and Measurement

  The certificate holder must develop and maintain processes and systems to acquire data with respect to its operations, products, and services to monitor the safety performance of the organization. These processes and systems must include, at a minimum, the following:

(1)  Continuous monitoring of operational processes; and

(2)  Periodic monitoring of the operational environment to detect changes.

[Bold text not addressed in AC 120-92A]

Changed 5.71(a)(1) and (2) to read as follows:

(a) The certificate holder must develop and maintain processes and systems to acquire data with respect to its operations, products, and services to monitor the safety performance of the organization. These processes and systems must include, at a minimum, the following:

(1) Continuous Monitoring of operational processes.

(2) Continuous Monitoring of the operational environment to detect changes.

Process 3.1.2 Internal Audits by Operational Departments

SMSVP Standard 5.71(a)(3), Safety Performance Monitoring and Measurement

  The certificate holder must develop and maintain processes and systems to acquire data with respect to its operations, products, and services to monitor the safety performance of the organization. These processes and systems must include, at a minimum, auditing of operational processes and systems.

[The term “Systems” is not addressed in AC 120-92A in reference to this process. This is an optional wording change as there are no functional differences in the processes.]

No change

Process 3.1.3 Internal Evaluation

SMSVP Standard 5.71(a)(4), Safety Performance Monitoring and Measurement

  The certificate holder must develop and maintain processes and systems to acquire data with respect to its operations, products, and services to monitor the safety performance of the organization. These processes and systems must include, at a minimum, evaluations of the SMS and operational processes and systems.

[The term “Systems” is not addressed in AC 120-92A in reference to this process. This is an optional wording change as there are no functional differences in the processes.]

No change

Process 3.1.4 External Auditing of the SMS

SMSVP Standard 5.71(a)(3), Safety Performance Monitoring and Measurement

  There is no wording or functional change required with this process. This process is included in the SMSVP 5.71(a)(3).

[Combining of processes]

No change

Process 3.1.5 Investigation

SMSVP Standard 5.71(a)(5) and (6), Safety Performance Monitoring and Measurement

No change noted.

No change

Process 3.1.6 Employee Reporting and Feedback System

SMSVP Standard 5.71(a)(7), Safety Performance Monitoring and Measurement

  The term “Employee Reporting and Feedback System,” has been replaced with the term “Confidential Employee Reporting System

[This is an optional wording change as there are no functional differences in the processes.] and

  The certificate holder must develop and maintain processes and systems to acquire data with respect to its operations, products, and services to monitor the safety performance of the organization. These processes and systems must include, at a minimum, the following:

(7) A confidential employee reporting system in which employees can report including, but not limited to hazards, issues, concerns, occurrences, incidents, as well as propose solutions and safety improvements.

[Bold text is not addressed in AC 120-92A with reference to the Employee Reporting System]

Changed 5.71(a)(7) to read as follows:

(7) A confidential employee reporting system in which employees can report, including, but not limited to: Hazards, issues, concerns, occurrences, incidents, as well as propose solutions and safety improvements.

(7) A confidential employee reporting system in which employees can report hazards, issues, concerns, occurrences, incidents, as well as propose solutions and safety improvements.

Process 3.1.7 Analysis of Data

SMSVP Standard 5.71(b), Safety Performance Monitoring and Measurement

  The certificate holder must develop and maintain processes and systems to acquire data with respect to its operations, products, and services to monitor the safety performance of the organization. These processes and systems must include, at a minimum, processes, the following:

(8) The certificate holder must develop and maintain processes that analyze the data acquired through the processes and systems identified under paragraph (a) of this section and any other relevant data with respect to its operations, products, and services.

[Bold text is not addressed in AC 120-92A. There is a requirement for this process in the AC but it only refers to “operations.” Review existing process, if “operations” includes products and services, no change is required.]

No change

Process 3.1.8 System Assessment

SMSVP Standard 5.73(a)(1), Safety Performance Assessment

No change noted.

5.73(a)(1) has been changed to read:

(1) Ensure the certificate holder’s compliance with the applicable regulatory requirements in Chapter I of title 14 of the Code of Federal Regulations and additional safety risk controls established by the certificate holder.

(1) Ensure compliance with the safety risk controls established by the certificate holder.

5.73(a)(5) has been changed to read:

(5) Identify potential new hazards.

Element 3.2 Management of Change

SMSVP Standard 5.73(a)(4), Safety Performance Assessment

  This process has been included with 5.73(a)(4).

[Combining of processes]

No change

Element 3.3 Continuous Improvement

SMSVP Standard 5.75, Continuous Improvement

No change noted.

5.75 has been changed to read:

The certificate holder must establish and implement processes to correct safety performance substandard deficiencies identified in the assessments conducted under 5.73.

Process 3.3.1 Preventive/Corrective Action

SMSVP Standard 5.75, Continuous Improvement

No change noted.

No change

Process 3.3.2 Management Review

SMSVP Standard 5.73(a)(4), Safety Performance Assessment

  This process has been included with 5.73(a)(4).

[Combining of processes]

No change

Component 4.0 Safety Promotion

Subpart E, Safety Promotion, SMSVP Standard 5.3(a)(4)

Intentionally left blank.

No change

Element 4.1 Competencies and Training

SMSVP Standard 5.91, Competencies and Training

No change noted.

5.91 has been changed by deleting the word qualifications and replacing with the word competencies.

The certificate holder must provide training to each individual identified in 5.23 to ensure the individuals attain and maintain the qualifications competencies necessary to perform their duties relevant to the operation and performance of the SMS.

Process 4.1.1 Personnel Expectations (Competence)

SMSVP Standard 5.91, Competencies and Training

No change noted.

No change

Process 4.1.2 Training

SMSVP Standard 5.91, Competencies and Training

No change noted.

No change

Element 4.2 Communication and Awareness

SMSVP Standard:

5.21(d) The safety policy must be documented and communicated throughout the certificate holder organization.

5.25(b)(3) [the accountable executive will] Communicate the safety policy throughout the certificate holder’s organization.

AC 120-92A, Appendix 1

Element 1.1b(2)(j): Be communicated with visible management endorsement to all employees and responsible parties.

No change

Figure 17-4-3U.  Definitions

A.    Causal Factors. Causal factors are that set of elements that affect an event’s outcome. A causal factor is not necessarily a root cause, because whereas removing a causal factor can benefit an outcome, it does not with certainty prevent recurrence of an undesirable event. (See “root cause” and “root cause analysis.”)

B.    Conformance. Means agreement in nature or form of a presented document, process, or system.

C.    Continued Operational Safety (COS). Routine recurring Performance Assessments (i.e., routine surveillance through safety inspections). Also includes certificate management, the management of major changes in operation (i.e., system configuration changes).

D.    Corporate Safety Risk Management (SRM). As used in this document is a process to identify hazards and associated risks, analyze risks, and develop new risk controls affecting multiple process owner areas/departments within the organization. Final risk acceptance for Corporate SRM may be accomplished at a management level above the process owner/department level, or by a committee.

E.    Corrective Action. A corrective action addresses a nonconformity that has occurred.

F.    Design Demonstration. An activity that demonstrates, for purposes of validation, that a certificate holder’s design of safety management processes function in an operational environment.

G.    Design Review. Determines if a certificate holder’s safety management processes conform to the Safety Management System Voluntary Program (SMSVP) Standard.

H.    Gap Analysis. Compares existing processes, procedures, programs, and activities to the SMSVP Standard.

I.    Hazard. Means a condition that can foreseeably cause or contribute to an aircraft accident as defined in Title 49 of the Code of Federal Regulations (49 CFR) part 830, § 830.2.

J.    Preventive Action. A preventive action addresses the potential for a nonconformity to occur.

K.    Risk. Means the composite of predicted severity and likelihood of the potential effect of a hazard.

L.    Risk Control. A means to reduce or eliminate the effects of hazards.

M.    Root Cause. The root cause of a nonconformity or undesirable event is that factor that would with certainty result in the event not occurring were it not present.

N.    Root Cause Analysis (RCA). A method for identifying the underlying causal factor of a nonconformity or undesirable event. A causal factor is considered the root cause if its removal from the event sequence prevents the undesirable event from recurring.

O.    Root Cause Analysis Corrective Action Plan. A formalized plan to eliminate the causal factor that resulted in a nonconformity or undesirable event by addressing the factor determined to be the root cause.

P.    Safety Assurance. Means processes within the SMS that function systematically to ensure the performance and effectiveness of safety risk controls and that the organization meets or exceeds its safety objectives through the collection, analysis, and assessment of information.

Q.    Safety Management System (SMS). Means the formal, top-down, organization-wide approach to managing safety risk and assuring the effectiveness of safety risk controls. It includes systematic procedures, practices, and policies for the management of safety risk.

R.    Safety Objective. Means a measurable goal or desirable outcome related to safety.

S.    Safety Performance. Means realized or actual safety accomplishment relative to the organization’s safety objectives.

T.    Safety Policy. Means the certificate holder’s documented commitment to safety, which defines its safety objectives and the accountabilities and responsibilities of its employees in regards to safety.

U.    Safety Promotion. Means a combination of training and communication of safety information to support the implementation and operation of an SMS in an organization.

V.    Safety Risk Management (SRM). Means a process within the SMS composed of describing the system, identifying the hazards, and analyzing, assessing and controlling safety risk.

W.    System. Means a group of interacting, interrelated, or interdependent elements forming a complete whole.

X.    Validation. CMT activities involving observations, audits, and certificate management functions that provide sufficient information for the CMT to assess whether a certificate holder’s system design achieves stated objectives and meets published SMS standards.

Y.    Validation Plan. Means a forecast of resources needed to perform applicable assessments to confirm a certificate holder’s safety management activities and processes.

17-4-3-9 through 17-4-3-23 RESERVED.