VOLUME 17  SAFETY MANAGEMENT SYSTEM

CHAPTER 4  SAFETY MANAGEMENT SYSTEM VOLUNTARY PROGRAM

Section 3  Continued Operational Safety—Certificate Management Team Monitoring and Surveillance

17-4-3-1    PURPOSE. This section provides guidance for Certificate Management Teams (CMT) to perform continued oversight of a certificate holder’s applied safety management processes.

17-4-3-3    SAFETY MANAGEMENT SYSTEM VOLUNTARY PROGRAM (SMSVP) EXPECTATIONS. Volume 17, Chapter 4, Section 1, paragraph 17-4-1-13 clearly states that the CMT is expected to provide ongoing surveillance support to validate a certificate holder’s continued conformance to the SMSVP Standard. By doing so, it is anticipated that the CMT will realize significant benefits when performing its certificate holder oversight responsibilities. Regardless, failure of either the certificate holder or the CMT to adequately meet its obligations to SMSVP requirements may result in Safety Management System Program Office (SMSPO) withdrawal of “State recognition.”

17-4-3-5    APPLICATION OF SAFETY MANAGEMENT SYSTEM (SMS) TO CONTINUED OPERATIONAL SURVEILLANCE. Once safety management processes and activities have been integrated into the certificate holder’s technical processes, the CMT must broaden the scope of its normal surveillance to include the certificate holder’s SMS activities. Under a fully functioning SMS, when an inspector finds a regulatory violation or process nonconformance, his or her most important concern is, “Why didn’t the certificate holder’s SMS processes identify this problem, and if it was identified, why did the SMS not contain and/or correct this problem?” A certificate holder’s SMS increases organizational safety awareness and reduces “plausible excuses of ignorance” regarding systemic safety issues.

17-4-3-7    CMT SURVEILLANCE RECORDS. A CMT must record all safety management assessment activities to demonstrate certificate holder conformance with the SMSVP Standard. CMT surveillance activities, associated with safety management, must be recorded in the Safety Assurance System (SAS) data repository. This is accomplished by using Data Collection Tools (DCT) and associated questions sets designed into existing tools to assess safety management activities.

NOTE:  For those certificate holders not in SAS, the CMT will make Program Tracking and Reporting Subsystem (PTRS) entries as defined in the Continued Operational Safety (COS) job aids.

Figure 17-4-3A.  Safety Management System Voluntary Program Standard

1.    Purpose of This Attachment. The Safety Management System Voluntary Program (SMSVP) Standard, when properly applied, is the basis for formal State recognition of a certificate holder’s Safety Management System (SMS). The SMSVP Standard, while resembling Title 14 of the Code of Federal Regulations (14 CFR) Part 5, Safety Management Systems, is a separate document used by the Flight Standards Service (AFS) SMS Program Office (SMSPO) to evaluate SMSVP participants.

2.    Applicability. The SMSVP Standard details the minimum conformance expectations participants must maintain for State recognition of its SMS. Adherence to the SMSVP Standard does not replace compliance with other FAA regulatory requirements. The certificate holder may establish more stringent requirements in its system than those in this Standard.

SAFETY MANAGEMENT SYSTEM VOLUNTARY PROGRAM STANDARD

Subpart A—General.

5.1 Applicability.

5.3 General Requirements.

5.5 Definitions.

Subpart B—Safety Policy.

5.21 Safety Policy.

5.23 Safety Accountability and Authority.

5.25 Designation and Responsibilities of Required Safety Management Personnel.

5.27 Coordination of Emergency Response Planning.

Subpart C—Safety Risk Management.

5.51 Applicability.

5.53 System Analysis and Hazard Identification.

5.55 Safety Risk Assessment and Control.

Subpart D—Safety Assurance.

5.71 Safety Performance Monitoring and Measurement.

5.73 Safety Performance Assessment.

5.75 Continuous Improvement.

Subpart E—Safety Promotion.

5.91 Competencies and Training.

5.93 Safety Communication.

Subpart F—SMS Documentation and Recordkeeping.

5.95 SMS Documentation.

5.97 SMS Records.

Subpart A—General.

5.1 Applicability.

(a) A certificate holder desiring to implement an SMS must meet all requirements of this Standard and be found acceptable using the validation process as described in the Safety Management System Voluntary Program.

5.3 General Requirements.

(a) Any certificate holder required to have a Safety Management System under this Standard must submit the Safety Management System to the Administrator for acceptance. The SMS must be appropriate to the size, scope, and complexity of the certificate holder’s operation and include at least the following components:

(1) Safety policy in accordance with the requirements of subpart B of this Standard;

(2) Safety risk management in accordance with the requirements of subpart C of this Standard;

(3) Safety assurance in accordance with the requirements of subpart D of this Standard; and

(4) Safety promotion in accordance with the requirements of subpart E of this Standard.

(b) The Safety Management System must be maintained in accordance with the recordkeeping requirements in subpart F of this Standard.

(c) The Safety Management System must ensure compliance with the relevant regulatory standards in chapter I of Title 14 of the Code of Federal Regulations.

5.5 Definitions.

Hazard means a condition that could foreseeably cause or contribute to an aircraft accident as defined in Title 49 of the Code of Federal Regulations (49 CFR) part 830, § 830.2.

Risk means the composite of predicted severity and likelihood of the potential effect of a hazard.

Risk control means a means to reduce or eliminate the effects of hazards.

Safety assurance means processes within the SMS that function systematically to ensure the performance and effectiveness of safety risk controls and that the organization meets or exceeds its safety objectives through the collection, analysis, and assessment of information.

Safety objective means a measurable goal or desirable outcome related to safety.

Safety performance means realized or actual safety accomplishment relative to the organization’s safety objectives.

Safety policy means the certificate holder’s documented commitment to safety, which defines its safety objectives and the accountabilities and responsibilities of its employees in regards to safety.

Safety promotion means a combination of training and communication of safety information to support the implementation and operation of an SMS in an organization.

Safety Risk Management means a process within the SMS composed of describing the system, identifying the hazards, and analyzing, assessing and controlling risk.

Subpart B—Safety Policy.

5.21 Safety Policy.

(a) The certificate holder must have a safety policy that includes at least the following:

(1) The safety objectives of the certificate holder.

(2) A commitment of the certificate holder to fulfill the organization’s safety objectives.

(3) A clear statement about the provision of the necessary resources for the implementation of the SMS.

(4) A safety reporting policy that defines requirements for employee reporting of safety hazards or issues.

(5) A policy that defines unacceptable behavior and conditions for disciplinary action.

(6) An emergency response plan that provides for the safe transition from normal to emergency operations in accordance with the requirements of 5.27.

(b) The safety policy must be signed by the accountable executive described in 5.25.

(c) The safety policy must be documented and communicated throughout the certificate holder’s organization.

(d) The safety policy must be regularly reviewed by the accountable executive to ensure it remains relevant and appropriate to the certificate holder.

5.23 Safety Accountability and Authority.

(a) The certificate holder must define accountability for safety within the organization’s safety policy for the following individuals:

(1) Accountable executive, as described in 5.25.

(2) All members of management in regard to developing, implementing, and maintaining SMS processes within their area of responsibility, including, but not limited to:

(i) Hazard identification and safety risk assessment.

(ii) Assuring the effectiveness of safety risk controls.

(iii) Promoting safety as required in subpart E of this Standard.

(iv) Advising the accountable executive on the performance of the SMS and on any need for improvement.

(3) Employees relative to the certificate holder’s safety performance.

(b) The certificate holder must identify the levels of management with the authority to make decisions regarding safety risk acceptance.

5.25 Designation and Responsibilities of Required Safety Management Personnel.

(a) Designation of the accountable executive. The certificate holder must identify an accountable executive who, irrespective of other functions, satisfies the following:

(1) Is the final authority over operations authorized to be conducted under the certificate holder’s certificate(s).

(2) Controls the financial resources required for the operations to be conducted under the certificate holder’s certificate(s).

(3) Controls the human resources required for the operations authorized to be conducted under the certificate holder’s certificate(s).

(4) Retains ultimate responsibility for the safety performance of the operations conducted under the certificate holder’s certificate.

(b) Responsibilities of the accountable executive. The accountable executive must accomplish the following:

(1) Ensure that the SMS is properly implemented and performing in all areas of the certificate holder’s organization.

(2) Develop and sign the safety policy of the certificate holder.

(3) Communicate the safety policy throughout the certificate holder’s organization.

(4) Regularly review the certificate holder’s safety policy to ensure it remains relevant and appropriate to the certificate holder.

(5) Regularly review the safety performance of the certificate holder’s organization and direct actions necessary to address substandard safety performance in accordance with 5.75.

(c) Designation of management personnel. The accountable executive must designate sufficient management personnel who, on behalf of the accountable executive, are responsible for the following:

(1) Coordinate implementation, maintenance, and integration of the SMS throughout the certificate holder’s organization.

(2) Facilitate hazard identification and safety risk analysis.

(3) Monitor the effectiveness of safety risk controls.

(4) Ensure safety promotion throughout the certificate holder’s organization as required in subpart E of this Standard.

(5) Regularly report to the accountable executive on the performance of the SMS and on any need for improvement.

5.27 Coordination of Emergency Response Planning.

Where emergency response procedures are necessary, the certificate holder must develop and the accountable executive must approve as part of the safety policy, an emergency response plan that addresses at least the following:

(a) Delegation of emergency authority throughout the certificate holder’s organization;

(b) Assignment of employee responsibilities during the emergency; and

(c) Coordination of the certificate holder’s emergency response plans with the emergency response plans of other organizations it must interface with during the provision of its services.

Subpart C—Safety Risk Management.

5.51 Applicability.

A certificate holder must apply safety risk management to the following:

(a) Implementation of new systems.

(b) Revision of existing systems.

(c) Development of operational procedures.

(d) Identification of hazards or ineffective risk controls through the safety assurance processes in subpart D of this Standard.

5.53 System Analysis and Hazard Identification.

(a) When applying safety risk management, the certificate holder must analyze the systems identified in 5.51. Those system analyses must be used to identify hazards under paragraph (c) of this section, and in developing and implementing risk controls related to the system under 5.55(c).

(b) In conducting the system analysis, the following information must be considered:

(1) Function and purpose of the system.

(2) The system’s operating environment.

(3) An outline of the system’s processes and procedures.

(4) The personnel, equipment, and facilities necessary for operation of the system.

(c) The certificate holder must develop and maintain processes to identify hazards within the context of the system analysis.

5.55 Safety Risk Assessment and Control.

(a) The certificate holder must develop and maintain processes to analyze safety risk associated with the hazards identified in 5.53(c).

(b) The certificate holder must define a process for conducting risk assessment that allows for the determination of acceptable safety risk.

(c) The certificate holder must develop and maintain processes to develop safety risk controls that are necessary as a result of the safety risk assessment process under paragraph (b) of this section.

(d) The certificate holder must evaluate whether the risk will be acceptable with the proposed safety risk control applied, before the safety risk control is implemented.

Subpart D—Safety Assurance.

5.71 Safety Performance Monitoring and Measurement.

(a) The certificate holder must develop and maintain processes and systems to acquire data with respect to its operations, products, and services to monitor the safety performance of the organization. These processes and systems must include, at a minimum, the following:

(1) Monitoring of operational processes.

(2) Monitoring of the operational environment to detect changes.

(3) Auditing of operational processes and systems.

(4) Evaluations of the SMS and operational processes and systems.

(5) Investigations of incidents and accidents.

(6) Investigations of reports regarding potential noncompliance with regulatory standards or other safety risk controls established by the certificate holder through the safety risk management process established in subpart C of this Standard.

(7) A confidential employee reporting system in which employees can report hazards, issues, concerns, occurrences, incidents, as well as propose solutions and safety improvements.

(b) The certificate holder must develop and maintain processes that analyze the data acquired through the processes and systems identified under paragraph (a) of this section and any other relevant data with respect to its operations, products, and services.

5.73 Safety Performance Assessment.

(a) The certificate holder must conduct assessments of its safety performance against its safety objectives, which include reviews by the accountable executive, to:

(1) Ensure compliance with the safety risk controls established by the certificate holder.

(2) Evaluate the performance of the SMS.

(3) Evaluate the effectiveness of the safety risk controls established under 5.55(c) and identify any ineffective controls.

(4) Identify changes in the operational environment that may introduce new hazards.

(5) Identify new hazards.

(b) Upon completion of the assessment, if ineffective controls or new hazards are identified under paragraphs (a)(2) through (5) of this section, the certificate holder must use the safety risk management process described in subpart C of this Standard.

5.75 Continuous Improvement.

The certificate holder must establish and implement processes to correct safety performance deficiencies identified in the assessments conducted under 5.73.

Subpart E—Safety Promotion.

5.91 Competencies and Training.

The certificate holder must provide training to each individual identified in 5.23 to ensure the individuals attain and maintain the competencies necessary to perform their duties relevant to the operation and performance of the SMS.

5.93 Safety Communication.

The certificate holder must develop and maintain means for communicating safety information that, at a minimum:

(a) Ensures that employees are aware of the SMS policies, processes, and tools that are relevant to their responsibilities.

(b) Conveys hazard information relevant to the employee’s responsibilities.

(c) Explains why safety actions have been taken.

(d) Explains why safety procedures are introduced or changed.

Subpart F—SMS Documentation and Recordkeeping.

5.95 SMS Documentation.

The certificate holder must develop and maintain SMS documentation that describes the certificate holder’s:

(a) Safety policy.

(b) SMS processes and procedures.

5.97 SMS Records.

(a) The certificate holder must maintain records of outputs of safety risk management processes as described in subpart C of this Standard. Such records must be retained for as long as the control remains relevant to the operation.

(b) The certificate holder must maintain records of outputs of safety assurance processes as described in subpart D of this Standard. Such records must be retained for a minimum of 5 years.

(c) The certificate holder must maintain a record of all training provided under 5.91 for each individual. Such records must be retained for as long as the individual is employed by the certificate holder.

(d) The certificate holder must retain records of all communications provided under 5.93 for a minimum of 24 consecutive calendar-months.

Figure 17-4-3B.  SMS Safety Policy Design Validation

 

 

Figure 17-4-3C.  SMS Safety Risk Management Design Validation

 

 

Figure 17-4-3D.  SMS Safety Assurance Design Validation

 

 

Figure 17-4-3E.  SMS Safety Promotion Design Validation

 

 

Figure 17-4-3F.  SMS Safety Policy Design Demonstration

Performance Objective:

Certificate Management Teams (CMT) need to validate, to the extent necessary, that the organization’s safety policy has been conveyed to employees throughout the organization to include:

•    A safety reporting policy for employee reporting of safety hazards or issues;

•    A policy that defines unacceptable behavior and conditions for disciplinary action;

•    Safety accountability within the organization.

Directions:

There are five basic subobjectives associated with this design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

During design validation, the CMT reviewed and accepted the certificate holder’s safety policy. The CMT must now confirm that the certificate holder has communicated this policy to employees applying or supporting its technical operations. The CMT should substantiate:

1)    The certificate holder’s communication guidance is being followed; and

2)    The effectiveness of the certificate holder’s communication strategy (i.e., employees understand how they can directly support safety policy in their day-to-day work activities).

A certificate holder’s safety policy must also define its process for reporting “safety hazards or issues.” Safety policy validation can be undertaken during regularly scheduled surveillance activities, or independently. Validating safety policy reporting and communications procedures can be done by interviewing employees at all levels of an organization.

Criteria:

•    The certificate holder’s process must effectively communicate its safety policy at all levels of the organization to existing, new, and temporary employees, as applicable.

•    All levels of management should be aware of their responsibility and accountability for safety in their organization. Individual managers are responsible for developing, implementing, and maintaining SMS processes within their technical areas. Members of management must be aware of their accountability and competence at:

•    Hazard identification and safety risk assessment;

•    Assuring the effectiveness of safety risk controls;

•    Promoting safety; and

•    Advising the accountable executive on the performance of the SMS and any need for improvement.

•    All employees at all levels must know what is acceptable and unacceptable behavior and conditions for disciplinary action.

•    All employees at all levels must know or be able to find the process for safety hazard and issue reporting (employee reporting). Several validation samples of personnel actually completing a hard copy or electronic sample submission should be accomplished.

NOTE:  Processing identified hazards may be accomplished as a separate validation activity or as a part of the safety policy validation, if the certificate holder’s process is not complex. If the certificate holder uses a corrective or preventive action process to resolve hazard reports, the CMT may wish to review the hazard report processing when it validates the corrective or preventive action process. The CMT should determine that the record includes information on the source of the input (e.g., Hazard Reporting Process – Department) (see Figure 17-4-3N, SMS Continuous Improvement Process Design Demonstration).

Validation Repeatability: It is recommended that this validation be repeated in as many technical operational areas as necessary to ensure that organization’s communications mechanisms effectively accomplish the stated objective of this test. It is further recommended that the CMT add these validations to its regular surveillance activities and not expend resources on independent “SMS only” validation work for an area/department, unless the area/department sampling demonstrates failure.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

Data Collection Tool Questions – Record of Results:                                                  YES      NO

Figure 17-4-3G.  SMS Emergency Preparedness/Response Design Demonstration

Performance Objective:

Certificate Management Teams (CMT) need to validate, to the extent necessary, that the certificate holder can effectively transition from normal operations to emergency operations without compromising safety. A secondary objective is to ensure that managers in contact with other organizations also having emergency response plans (ERP) have documented evidence that their respective ERPs are coordinated.

Directions:

There are five basic subobjectives associated with this design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

The certificate holder’s ERP process was reviewed and accepted during CMT design validation. The CMT must now ensure these processes are effective by validating the certificate holder’s ability to move select individuals out of normal daily operations and that those operations continue to effectively function during their absence.

It is important to verify that when key decisionmakers are unavailable to fulfill their responsibilities, the certificate holder has position proxies or a backup plan to maintain the affected processes. The CMT must ensure that the organization’s “backup strategy” (people and processes) will work.

A certificate holder’s ERP documentation should identify substitutes for those that must participate in emergency activities and are unavailable to perform normal duties. The CMT may test:

1)    How the person is notified of their additional duties;

2)    That, as a proxy, they have the competencies (training) to perform the additional duties; and

3)    That the person is knowledgeable of these duties or can identify appropriate guidance required for performance.

The CMT will require evidence that the certificate holder is coordinating its ERP with other organizations’ emergency plans. Evidence of coordinated ERPs may be located in meeting minutes, documents, and/or supplier contracts. When the certificate holder documents its ERP coordination in proprietary documents (e.g., contracts, etc.) it may provide excerpts (redacted information) as proof of performance.

Criteria: The CMT uses a certificate holder’s ERP to identify several key samples for testing.

•    That proxies for risk decisionmakers have been identified, that have been removed from normal operations to conduct emergency operations.

•    The limitation of the authority of those proxies is defined.

•    A proxy has the authorities and competencies (training) required by the organization to make safety‑related decisions for the process area (e.g., Safety Risk Management (SRM) activities, corrective action oversight, etc.).

•    The organization shows satisfactory documentation that ERPs are coordinated with external business partners that have ERPs (e.g., code share partners, airports, etc.).

Validation Repeatability: Validations for proxies should be sampled using the ERP to identify process area samples. The CMT may test each process area individually during normal, routine surveillance or the CMT may perform a single “tabletop” activity to verify proxies’ knowledge of their duties and responsibilities. Certificate holder’s ERP coordination with external parties may be validated as a single activity if they have developed a common record repository.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

Data Collection Tool Questions – Record of Results:                                                  YES      NO

Figure 17-4-3H.  SMS SRM (Process/Department Owner) Design Demonstration

Performance Objective:

Certificate Management Teams (CMT) need to validate, to the extent necessary, that those individuals or groups: accept supplier guidance materials into their process area; and/or have the authority to draft and approve new or revised procedural changes for their process area, can effectively apply the organization’s Safety Risk Management (SRM) process to those process procedures.

NOTE:  There is another validation test for the corporate level SRM Process (see Figure 17-4-3J, SMS SRM (Organizational) Design Demonstration). In this test, multiple process areas are affected and process owners must interact determining the perceived risks and mitigations (e.g., adding a new aircraft fleet, implementing new multifaceted software solution across process areas, etc.).

Directions:

There are five basic subobjectives associated with this design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

The certificate holder’s SRM process was reviewed and accepted during CMT design validation. Next, the CMT must ensure that process owners throughout the system can perform SRM. Since SRM is applicable only to “design change,” the CMT should evaluate how guidance documents, used by the certificate holder’s workforce, are revised. The CMT should identify those drafting and authorizing guidance document changes. During this evaluation the CMT may find that a manager does not always draft guidance changes or review supplier provided documents for acceptance into the system. It is, therefore, important to identify who is actually performing work associated with the SRM process steps and determine what risk acceptance authority they have under the certificate holder’s defined process.

Understanding that new or revision of personnel guidance is vital to SRM applications, the CMT should concentrate on the organization’s application of SRM in their document control and approval process. Realizing that any design change in these documents has an SRM recording requirement attached to it, it is important that the CMT validate:

1)    What person(s) is making decisions regarding design change for the process area;

2)    Is there documented evidence that this person(s) has been trained to perform these duties; and

3)    Who has been designated to sign off on the document (accept risk) for the process area?

In smaller organizations, a single person may have the authority to perform the entire SRM process steps. In larger organizations or organizations with complex process areas (e.g., maintenance department for large airlines, etc.), the authority to perform specific aspects of the SRM process may be delegated to subordinates. In these situations, the CMT needs to identify the first SRM decisionmaker (hazard identification) and trace the process up through the chain of command until the person authorized to “accept risk” (i.e., sign off the design change) is identified.

For SRM samples that result in the development of new controls added to a process procedure, there should be a documented record of the outputs for the following processes:

1)    Identified hazards;

2)    Associated risks for each hazard;

3)    Analysis for each risk; and

4)    Any new control(s).

Criteria:

•    The person conducting an SRM required activity is given that authority by the certificate holder. Training is documented to demonstrate competency to perform the specified activity(ies).

•    The required records for each required SRM activity are complete (minimum record is a “no hazard” signoff for new or revised process/procedural change). When decisionmakers identify risks and new controls, the required records are:

•    List of hazards;

•    List of risks associated with each hazard;

•    Analysis of each risk; and

•    Record of mitigation (controls).

•    Escalation and Traceability – when a single person is not responsible for all decisions related to the SRM process, the “decisionmaking chain of command” must be evaluated to ensure:

•    Persons performing some, but not all, SRM process activities are authorized by the organization to do so and competent (trained) to perform those activities.

•    Escalation interfaces of SRM activities from one level of process manager to a higher level of process manager allows a positive transfer to occur.

•    Escalation of SRM process activities is traceable from one process owner to another.

•    Transference of SRM process steps between authorized personnel is monitored to prevent failure of the transfer process.

Validation Repeatability: The CMT shall repeat the validations in all process areas and for as many process owners or process owner escalations as the CMT finds necessary to ensure full integration of the SRM process to the lowest levels of decisionmaking within a process area. Since SRM is one of the most critical SMS components, SRM process owner validation must be very thorough.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

Data Collection Tool Questions – Record of Results:                                                  YES      NO

Figure 17-4-3J.  SMS SRM (Organizational) Design Demonstration

Performance Objective:

Certificate Management Teams (CMT) must validate, to the extent necessary, the certificate holder’s process for conducting integrated Safety Risk Management (SRM) when multiple departments are affected by a system change.

NOTE:  This SRM test is not to be confused with a process owner/department level SRM, if the certificate holder defines different process steps for “multidepartment” SRM (see Figure 17-4-3H, SMS SRM (Process/Department Owner) Design Demonstration). It is highly recommended that process owner/department SRM be assessed before testing the corporate SRM process.

Directions:

There are five basic subobjectives associated with this design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

The organization’s corporate SRM process was reviewed and accepted during CMT design review validation. The CMT next ensures that process owners/department representatives can perform corporate level SRM.

The certificate holder has a process to identify hazards and associated risks, analyze risks, and develop new risk controls that affect multiple process owner/departments within its organization. SRM decisionmaking and recording requirements are the same as those for “process owner/department SRM,” except there are more complex interfaces between departments and require process owner/department leadership to coordinate the required risk mitigations. In addition, final risk acceptance for an organization may be made at a management level above the process owner or by a committee. It is important that the CMT understand and validate these differences between the corporate and process area SRM processes, as applicable.

The CMT will determine whether the corporate level interfaces allow for all required SRM activities to be completed and documented. The CMT will ensure that those conducting corporate SRM activities have the authority and competencies (training) required. It is recommended that corporate SRM validation follow the process owner SRM validations. This allows the CMT to identify how individual process owners process SRM risk decisions within their technical area before they participate in the “higher level” corporate SRM process.

The corporate level SRM performance validation test is one of two final validation tests jointly conducted by the CMT, SMS Program Office, and certificate holder.

Criteria:

•    The person(s) conducting the corporate level SRM activities have been given the authority by the certificate holder and it is documented the person(s) are competent to perform the specified activity(ies).

•    The records for each required SRM activity are complete.

•    The certificate holder has included, through documented record, each process owner stakeholder who must contribute to a collective risk decision and their respective inputs have been recorded as required by the corporate SRM process (e.g., meeting minutes with attendance rosters, required process owner submissions attached to meeting minutes, etc.).

Validation Repeatability: This performance validation only needs to be conducted once. It is normally one of the last validation tests before the certificate holder’s SMS is accepted. The CMT and Safety Management System Program Officer (SMSPO) will perform this validation jointly. It is highly recommended that a corporate SRM test include as many process owner areas/departments as possible. If the test sample does not include all process owner areas, the CMT should require that all process owners/departments are represented during a test (i.e., during a tabletop exercise). This sequence allows the CMT and SMSPO to ask pertinent questions.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

Data Collection Tool Questions – Record of Results:                                                  YES      NO

Figure 17-4-3K.  SMS Audit Process Design Demonstration

Performance Objective:

Certificate Management Teams (CMT) need to validate, to the extent necessary, that the certificate holder is periodically conducting audits to assess process function against defined process requirements. The CMT must ensure that the organization uses competent auditors, their reviews are system-wide, and there is an effective process to identify and correct nonconformance.

Directions:

There are five basic subobjectives associated with this design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

The certificate holder is expected to conduct audits to monitor the system to ensure that it functions as designed. Audits should be conducted by personnel with requisite competencies in the process area being reviewed to ensure an in-depth and detailed audit is performed. Often, audits are conducted by auditors independent of the process area. However, an auditor that does not have detailed knowledge of the process requirements, and the intended outcomes, usually provides only obvious process nonconformance.

It is important that the CMT ensure that audits are performed on all operational processes and systems. It is also important that the certificate holder identifies the minimum baseline frequency of assessments to satisfactorily monitor the process area and may develop an audit schedule to facilitate this. However, the organization may elect to perform additional process audits for a variety of reasons (e.g., effectiveness validation of a corrective action, a mitigation activity for a risk being monitored, an independent assessment by the evaluations team, etc.).

The CMT must ensure that auditor-identified nonconformance items are acted upon. The CMT may confirm correction of the nonconformance by determining if the certificate holder is using a corrective action tracking log or other method. Whatever the certificate holder uses, the audit should not be closed out until nonconformance items are transferred to the appropriate resolution process.

Criteria:

•    Each critical process area/department is within the scope of the audit process and there is a strategy or audit schedule for periodic monitoring to occur.

•    Audits are conducted by qualified personnel with competencies in the audit areas.

•    Audit findings of nonconformance are appropriately tracked and corrective or preventive action (negative trends), and any associated action plans, are appropriately closed out.

•    Corrective or preventive actions resulting from audits are not closed without effectiveness verification by qualified personnel.

•    Corrective or preventive actions resulting from audits were spot checked by the CMT to ensure all proposed actions were implemented prior to closing the action. The CMT should choose as many verification samples as it feels appropriate to ensure process owners are following through on proposed actions. Often a CMT will choose its sampling based on identified process risks or process criticality.

•    For corrective or preventive actions resulting from audits that identify a procedural change, there must be appropriate objective evidence of SRM being conducted (see Figure 17-4-3H, SMS SRM (Process/Department Owner) Design Demonstration).

Validation Repeatability: The CMT may wish to assess the completeness of the audit process as a single validation activity if a specified person or group in the organization manages the audit program. If validated in this manner, the CMT may pick specific audit findings of nonconformance for multiple process owner areas to validate the audit process from the data collection phase through the correction phase.

The organization’s audit process should require individual process owners to conduct their own internal process audits. The CMT should validate the completeness of the process owner audits using multiple validation activities (process owner by process owner).

Regardless of how audits are organized, it is recommended that the certificate holder’s audit outputs be compared against the CMT assessments to discern whether the audit yielded outputs “equal to” or “better than” the CMT assessment outputs. The certificate holder’s audits should always be more thorough than that of an external assessor, including those of the FAA.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

Data Collection Tool Questions – Record of Results:                                                  YES      NO

Figure 17-4-3L.  SMS Evaluation Process Design Demonstration

Performance Objective:

Certificate Management Teams (CMT) need to validate, to the extent necessary, that a person or group within the certificate holder organization is analyzing aggregate data to measure and evaluate process area performance. These evaluations must include the status of defined organizational objectives and the status of process owner compliance with required safety management activities. Evaluations are independently reported to executive management.

Directions:

There are five basic subobjectives associated with design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

The certificate holder is expected to conduct evaluations to monitor performance across the system. Evaluations should be conducted by an individual or team independent of the process owners/department managers. Evaluations should target aggregate data from multiple data sources including: results of audits, trend data from department records generated, records required to measure progress towards defined safety objectives, corrective action/preventive action effectiveness, observations, or any other relevant data.

The individual or group should have unrestricted access to executive management as an independent reporting source. The CMT will assess the certificate holder’s ability to manage safety through independent evaluations of processes and activities. It is important that the CMT understand the inputs used for evaluations to ensure that evaluations are being appropriately applied across the system.

Criteria:

•    Ensure each process area/department is within the scope of the evaluations process.

•    Ensure that the evaluation person/team reports to executive management independent of process owner/department management to validate process performance claims by those managers.

•    Ensure that evaluation reports assess whether the organization is meeting its safety objectives.

•    An effective evaluation process should consider the following inputs:

•    Results of audits;

•    Results of investigations;

•    Results of corrective or preventive actions to include effectiveness evaluations;

•    Results of actions directed by executive management reviews;

•    Results of continuous monitoring activities directed by process owners;

•    Results of hazard reporting; and

•    Results of new control effectiveness that were implemented by process owners since the last evaluations reporting period.

Validation Repeatability: The CMT may wish to validate the completeness of the evaluations process as a single validation activity after all SMS expectations have been implemented system-wide and this data is available for evaluation. This ensures that there is enough aggregate data from all process owner areas to ensure evaluation completeness. Once the CMT is confident that evaluations are being conducted system-wide, it may only be necessary to validate one evaluation. The CMT should review how the results of the evaluations are reported to executive management (reporting mechanism) and how the certificate holder ensures repeatability (e.g., a management review type process which may include evaluation reports, etc.).

Conversely, the CMT may wish to conduct several validation activities if they determine that independent process area evaluations reviews would offer greater flexibility to the CMT during the validation process.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

Data Collection Tool Questions – Record of Results:                                                  YES      NO

Figure 17-4-3M.  SMS Investigation Process Design Demonstration

Performance Objective:

Certificate Management Teams (CMT) need to validate, to the extent necessary, that those persons/positions assigned to conduct investigations of incidents and accidents are capable of performing those duties. The CMT will determine if a certificate holder’s investigation process follows a formal process to collect and analyze target specific data (e.g., accidents, incidents, regulatory violations, etc.). The CMT will assess if the process determines causal factors and develop process corrections, as necessary, to correct system deficiencies and improve the safety performance of the organization.

Directions:

There are five basic subobjectives associated with design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

The steps of an investigations process is not substantially different than the process steps associated with a certificate holder’s corrective action processes. Investigations are focused on defined events (e.g., accident, incident, etc.) and may require special data collection activities to aid process owners in their analysis and subsequent corrective actions (e.g., one investigatory practice may include interview information from event witnesses). The required investigation process steps should be defined by the organization in its guidance documents. The CMT only needs to ensure that personnel, authorities, competencies, and process steps are understood and/or demonstrated in defined accident or incident documentation.

Therefore, it is important that investigation records identify “who” conducted certain activities so the CMT can validate authorities and competencies of those individuals.

Criteria:

•    The investigation process steps should be understood by those persons/positions defined by the organization.

•    Any accident or incident investigation process steps should be completed using actual samples.

•    The person/position responsible to complete the investigation includes any documentation required by the certificate holder.

•    Investigations are implemented in a timely manner to preserve evidence associated with the event.

•    Any investigation activities requiring an interface with other processes used to maintain system integrity (e.g., SRM, Preventive Action/Preventive Action, Voluntary Self Disclosure, etc.) are complete and traceable to the associated investigation.

•    Investigations should not be fully closed until the certificate holder has validated all required actions required by the certificate holder investigation process were implemented.

•    Required actions must be evaluated for effectiveness before the investigation is considered complete (determine whether system deficiencies have been corrected to improve the safety performance of the organization).

Validation Repeatability: The CMT may wish to validate the investigations process by selecting samples from completed accident or incident investigation records to ensure process steps were completed by authorized personnel. The CMT may wish to combine the investigations process validation with other, similar, corrective action processes or independently validate the investigations process. If a specific person or team coordinates investigations for the entire organization, the validation may be completed as a one-time event by interviewing the coordinator and reviewing documentation samples. If the certificate holder identifies multiple process owners as having investigation authority, more samples may be warranted. To save time, the CMT may wish to perform a tabletop exercise with parties responsible to conduct investigations on behalf of the certificate holder and then sample associated records as a separate validation event.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

Data Collection Tool Questions – Record of Results:                                                   YES     NO

Figure 17-4-3N.  SMS Continuous Improvement Process Design Demonstration

Performance Objective:

Certificate Management Teams (CMT) need to validate, to the extent necessary, that technical process integrity is being managed to correct substandard safety performance by implementing corrective or preventive action when necessary. It is important that the CMT ensures that the certificate holder takes defined action when a process nonconformance has occurred or negative trends suggest a potential nonconformance will occur.

Directions:

There are five basic subobjectives associated with design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

The certificate holder is expected to monitor its system processes in a variety of ways (e.g., audit, evaluations, hazard reporting, investigations, daily/weekly/monthly record reviews, etc.). When any monitoring mechanism identifies actual or potential process failure, the certificate holder must take action to correct or prevent a nonconformance and maintain process integrity to its original design expectation. The CMT will validate that process owners responsible for these actions complete all the required process steps in the certificate holder’s process, provide proof of action implementation, and have not closed the action until an “effectiveness evaluation” has been completed.

The effectiveness evaluation should be defined by the process owner during the action determination phase of the process and should be documented on a tracking record to direct the follow-up evaluation. The effectiveness evaluation may be conducted by the process owner/proxy or another person/group/department in the organization that can understand the follow-up evaluation requirements.

Corrections and preventions should be closed in a timely manner. (“Timely” means that the organization has proof that they are actively moving toward resolution or they have set targeted objectives and recorded progress on those objectives.) Often lengthy corrections/preventions are associated with complex or expensive solutions. If noncomplex corrections and/or preventions are not making progress toward final solution, the CMT should discuss the issue(s) with the process owner to determine causes for the delays. It should be noted that the organization should implement temporary risk mitigations (e.g., cease an operation, use communication backup plans, perform frequent checks, etc.) until the final action plan is fully implemented. The CMT should also question the integrity of the temporary mitigations that were put in place until the corrective or preventive action is implemented.

One way to ensure an effective preventive/corrective action process is to use a tracking system. Some attributes and activities associated with an effective preventive/corrective action tracking process are as follows:

•    The document used to track preventive/corrective action has sufficient “general information” to identify the input source (e.g., audit finding, employee report, etc.), date opened, unique tracking number for traceability reference, and the identification of the responsible process owner who will oversee the process activities, and other process owner interfaces.

•    The tracking document provides the immediate actions used to “contain” the problem, allowing the process to continue functioning safely until a final solution is implemented.

•    The tracking document provides a location to record root cause analysis associated with the process.

•    The action plan is not closed without an effectiveness evaluation by qualified personnel.

•    In addition to reviewing the status of a large sample of tracking documents for specific process owners/departments, specific action plans should be selected by the CMT representative to validate that all process steps identified in the action plan were fully implemented. There should be sufficient evidence to verify full implementation of the selected samples.

•    For corrective or preventive actions leading to a process design change, there should be clear, traceable evidence to a completed Safety Risk Management (SRM) process record.

Criteria:

•    The certificate holder must establish and implement processes to correct identified substandard safety performance.

Validation Repeatability: The CMT may decide to validate the corrective or preventive action process independently or add to a regularly planned assessment where records would be easily accessed. The CMT may also decide to perform the validation in two phases:

1)    Perform a high level validation of the corrective or preventive action process by thorough examination of associated records and validating signature authorities, process training records, and timely closure of the process action plans; and

2)    Select specific samples that require onsite validations and add these validation activities to regular surveillance activities for specific process owners/departments.

The CMT may decide to add this validation to selected, prescheduled, process area surveillance activities. During surveillance activity, the inspector should ask to see the process owner’s documentation as required by the certificate holder’s process (e.g., tracking records from audits, management review, employee reports, investigations, continuous monitoring, etc.) and complete the review defined in the previous paragraph. Basically, the only difference in this approach is the CMT’s preference as to how it wishes to initiate the assessment.

Regardless of technique, it is very important that the CMT performs enough validation activities to ensure the consistency of process owners “follow through” across the organization.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

Data Collection Tool Questions – Record of Results:                                                   YES     NO

Figure 17-4-3P.  SMS Accountable Executive Review Design Demonstration

Performance Objective:

Certificate Management Teams (CMT) must validate, to the extent necessary, that the certificate holder’s accountable executive is involved in the system-wide safety management efforts. The accountable executive must have adequate knowledge to play an active role in directing actions relevant to resolving safety performance deficiencies in the system.

Directions:

There are five basic subobjectives associated with design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

The certificate holder’s accountable executive was identified using a SMSVP job aid during design validation. The accountable executive is defined as a key leadership individual in the organization’s business tier that has ultimate authority over safety operations and organizational resources. As a result, it is important that the accountable executive is aware of safety performance data and information collected from the system so that he/she may direct any necessary actions and/or resources to support safety initiatives.

It is important that the accountable executive:

1)    Hold periodic meetings to review collected data and information to assess the safety performance of the organization;

2)    At a minimum, review key data/information inputs defined by the SMSVP Standard; and

3)    Direct appropriate action, as warranted.

Accountable executive directed actions should be processed in the same manner as other corrections made in system processes. These methods include corrective and/or preventive action, investigations, SRM process corrections, etc.

Criteria:

•    The organization has a process for the accountable executive review (e.g., management review).

•    Objective evidence can be obtained to support that executive management reviews are being performed.

•    Management reviews should include those required by the accountable executive, but at minimum:

•    Information on the effectiveness of safety risk controls (usually results from audits for each process owner/department, external audits, continuous monitoring outputs, voluntary disclosure reporting program, etc.).

•    Information on the effectiveness of safety risk controls established since the last reporting period (these reports are usually the results of the effectiveness evaluations from corrective or preventive actions and SRM).

•    Information on changes to operational environments and associated new hazards (e.g., things not in control of the certificate holder: regulatory changes, airport configuration changes, changes to approach or en route procedures, vendor status changes, etc.).

•    Information on new hazards identified throughout the system through any safety assurance mechanism used by the organization.

•    Other aggregate information, that relates to the effectiveness of the organization’s safety management efforts towards meeting its stated safety objectives.

NOTE:  Meeting minutes from accountable executive reviews are convenient recording locations for revalidation or edits to the organization’s safety policy. This record is sufficient evidence of a “signed safety policy,” which is required to be communicated throughout the organization.

Validation Repeatability: This validation need only be conducted once and as one of the final CMT validation process activities. However, this final test must be conducted with the SMSPO. It is important that the certificate holder has completed full SMS implementation, so it can define what system reports are appropriate for the management review process(es). Finally, the accountable executive must take appropriate action to address any substandard safety performance. This validation may be repeated if the certificate holder does not follow its defined process and the minimum data/information detailed above was not included during the CMT validation assessment.

NOTE:  It is often difficult to identify directed actions resulting from meeting minutes unless a template is used to list defined actions to be carried forward to the next management review. Using this technique removes the “guess work” associated with deciphering discussions contained in meeting minutes.

Certificate holder use of a template or “actions table” for the meeting minutes is strongly encouraged.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

Data Collection Tool Questions – Record of Results:                                                  YES      NO

Figure 17-4-3Q.  SMS Records Retention Process Design Demonstration

Performance Objective:

Certificate Management Teams (CMT) will validate, to the extent necessary, that the organization has record retention capability conforming to the SMSVP Standard in either paper or electronic media. The ability of the organization to retrieve archived records shall be tested.

Directions:

There are five basic subobjectives associated with design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

The certificate holder is required to maintain records demonstrating conformance with applicable SMSVP standards and provide historical reference documents for ongoing decisionmaking.

The CMT must ensure that the certificate holder is capable of storing data for the required time periods defined in the SMSVP Standard and those required to retrieve stored data can do so in a “timely” manner. For paper records, access, protection from damage and misfiling are components of a good process. For electronic records, access, backup and protection from loss or overwrite are components of a good process. The CMT should test the certificate holder’s record systems by requesting evidence that stored historical data matches the maximum retention period requirement.

For example: If today’s date is 12/01/13 and there is a 24-month retention requirement, the certificate holder should be able to produce records from 12/01/11. If today’s date is 12/01/18 and the retention requirement is unlimited, then records must be accessible back to the initial date of creation.

NOTE:  If there is no “master record tracking document” defining the initial inception date of record, there is no standard to measure the historical completeness of a given record.

It is difficult to determine if something is missing from recorded history if one does not know what is supposed to be in the historic file in the first place. Therefore, the CMT must always identify the evaluation standard it will use to test the records retention system before examining individual records or files.

For example: For personnel records, the CMT should pick individuals from actual surveillance activities to determine required training from the sample. Since training records are required to be retained as long as the individual is employed, ask the management representative for employee records of individuals who are working in the process area. If employees have SMS training modules in their job description, this should be documented in a training matrix detailing the requirement and process area. The CMT, for process procedure records, should identify a specific process for assessment with a revisions log or document history. The CMT evaluator should then check the archived documents by composition or approval date to validate the document retention requirement.

Criteria:

•    “Unlimited” record retention requirement: records of SRM outputs for as long as the control remains relevant to the operation (i.e., each revision level of a process procedure should have SRM records from the date of original SMS acceptance). Employee competencies and training records must be retained as long as the individual is employed.

•    Five-year record retention requirement: Safety Assurance outputs (e.g., investigations, audits, corrective and preventive action, continuous process monitoring records (whether by day, week, or month) and employee hazard reports).

•    Twenty-four-month record retention requirement: Safety communications, (e.g., the “why” documentation that includes bulletins, training records/curricula, records of corrective or preventive actions that require retraining of employees, meeting or briefing notes where “why” is explained, checklists of items reviewed at production meetings, etc.).

NOTE:  While it is commendable that a certificate holder can control its documents in an orderly fashion, if records are not being used for their intended purpose, then the records retention process is just a compliance drill. To prevent this, the CMT should ensure training records are periodically audited by the certificate holder to validate that its training process is working. When SRM is conducted, records from past SRM decisions should be reviewed as part of the analysis process.

Validation Repeatability: This validation is applicable to all process areas. In a large organization, the CMT may wish to select specific samples from process area subgroups and perform a one-time check; applying those samples to all record media used by that process group.

If the organization is smaller, management often requires its individual process owners to maintain records applicable to their area/department. In these situations, the CMT may wish to perform multiple validations on process areas with several process owners.

Regardless of the certificate holder’s size, it is important the CMT identify the records custodian(s) and perform enough validation activities to feel confident in the certificate holder’s ability to meet the SMSVP records retention requirements.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

Data Collection Tool Questions – Record of Results:                                                  YES      NO

Figure 17-4-3R.  SMS Safety Communications Design Demonstration

Performance Objective:

Certificate Management Teams (CMT) will validate, to the extent necessary, that the certificate holder has communicated safety information throughout its organization to ensure that employees are aware of their safety‑related responsibilities, and other critical safety-related information.

Directions:

There are five basic subobjectives associated with design demonstration:

1)    The certificate holder has applied its design requirements to system operations;

2)    Certificate holder personnel have the competencies to perform their safety management-related duties and responsibilities (i.e., qualification, training, knowledge, and experience);

3)    Process area/department personnel are appropriately applying the documented process procedures approved in their guidance documents;

4)    The expected outputs of the process application are achieved; and

5)    Gaps in the certificate holder’s process design are identified during CMT validations.

The certificate holder is required to communicate safety-related information to its employees.

The SMSVP Standard identifies three communications requirements:

1)    Communications between management and employees ensures awareness of their specific safety management duties and responsibilities (e.g., employee guidance documents, manuals, training records and curricula, bulletins, etc.).

2)    Communications between management and employees resulting from identified hazard information that impacts specific employee groups (e.g., bulletins, production meetings, training records and curricula, etc.).

3)    Communications explaining why safety actions were taken to include why the addition of new controls or imposed corrective actions were implemented to correct process nonconformance or negative trends.

When a new process or procedural control is implemented, the affected employees (revised procedure) need to know why the new control was implemented. In other words, employees affected by the change should understand the basic objectives of the new control. By communicating the “why” behind a change, employees are better able to help management reach the proposed objectives.

NOTE:  The intent of this requirement is reinforcing to the certificate holder that it cannot expect employees to support desired outcomes if they don’t know what they are. Employees will often not remember the “why” when questioned about a changed process but should be aware they contribute to the overall safety of their organization. The CMT should also question the integrity of temporary mitigations before the mitigation is implemented. The CMT will have to sample enough employees to assess whether it believes the organization’s communication method is effective and meets the intent of the SMSVP communications requirement.

Criteria:

•    The organization’s process must ensure that all employees throughout the organization are aware of the safety management system.

•    The organization’s process must ensure that any safety-critical information is conveyed to the appropriate lines of business.

•    The organization must have a process that ensures that an explanation is communicated to employees on why particular safety actions are taken.

•    The organization must have a process that ensures that an explanation is communicated to employees on why a safety procedure is introduced or changed.

Validation Repeatability: This validation is applicable to all process areas. The CMT may wish to select samples from process area subgroups in a large organization and perform a one-time check to access communication media used by the certificate holder. In a smaller organization all communication may be company-wide. Communications in a small, medium, or large organization may be in the form of newsletters, safety bulletins, training media, meetings, etc.

Regardless of the certificate holder’s size, it is important that the CMT identify the processes used for communicating safety information and performs enough validation activities to feel confident in the organization’s ability to meet the SMSVP communications requirement.

Organizational Manual Reference(s) Used for the Process Area Assessed in This Validation Test:

Data Collection Tool Questions – Record of Results:                                                  YES      NO

Figure 17-4-3S.  Transitioning from SMS Pilot Project to the SMS Voluntary Program

SMS Pilot Project (SMSPP) and the SMS Voluntary Program (SMSVP). Since 2007, the SMSPP has provided the FAA and certificate holders significant experience and lessons learned for good safety management implementation strategies.

Establishing a permanent way for certificate holders to have their SMS integrated into day-to-day operations or recognized for international operations is a logical evolution of the SMSPP. For that reason, the Flight Standards Service National Field Office (AFS-900) has created the SMSVP. As a result, all current SMSPP participants are automatically in the SMSVP, unless required by regulation to establish an SMS. In those cases, certificate holders will follow issued regulations and referenced advisory materials.

While certificate holders are “automatically” entered into the SMSVP, their SMS implementation efforts must correspond to the SMSVP structure. The Safety Management System Program Office (SMSPO) and certificate management teams (CMT) will use design validations to measure progress. The SMSPO and certificate holders’ CMTs will work to ensure that progress is properly acknowledged and past work is not lost.

This figure describes the process that SMSPP participants will use to transition to the SMSVP. This process will be discontinued once all SMSPP participants have transitioned to the SMSVP. If a certificate holder does not wish to make this transition, they may withdraw from the SMSPP and any Flight Standards Service (AFS) acknowledgement letters will be null and void.

1.    Phase 1 – Certificate holder’s Implementation Transition. Certificate holders will revise their implementation plans to the SMSVP Standard (Figure 17-4-3A). It is recommended that the certificate holder and CMT become familiar with the SMSVP Standard to realize the few differences between the Advisory Circular (AC) 120-92A Framework and the SMSVP Standard (see Figure 17-4-3T). After familiarization, the SMSPO recommends that the certificate holder take the following steps to revise its implementation efforts:

a.    The certificate holder should identify any new SMSVP expectations that are different from its original implementation plan conceived under AC 120-92A, Appendix 1, Aviation Service Provider Safety Management System Framework: Functional Expectations.

b.    The certificate holder should determine what changes or modifications/revisions will have to be implemented to meet the new expectations.

c.    The status of each expectation should be annotated on a revised Implementation Plan. This may be as simple as adding a “status” column to the existing plan and annotating whether the expectation:

i.    Has been met;

ii.   Requires revision; or

iii.  Remains in progress (the expectation is still under initial development).

d.    All remaining work on the revised implementation plan should include:

i.    Any revised manual references;

ii.   The person responsible; and

iii.  Anticipated completion dates for documentation and full implementation. These dates will be used by the CMT to develop its validation plan for expectations still under development.

e.    The certificate holder will re-submit its revised SMSVP Implementation Plan to the CMT, using the revision process formally agreed upon between the certificate holder and CMT.

NOTE:  While the certificate holder may develop its Implementation Plan in any form or manner it chooses, the plan must be acceptable to the CMT. Under the SMSVP, the plan must include dates that the certificate holder expects its documentation to be completed and target dates when documented requirements will be implemented into its system. The CMT will use these dates to develop its validation project plan required under the SMSVP (see Volume 17, Chapter 4, Section 2).

2.    CMT Review and Acknowledging of the Certificate holder’s Revised Implementation Plan. While the certificate holder’s revised plan will contain relatively few changes, the CMT verification activities will shift to design validations using the SMSVP validation tools. The following process steps will be used by the CMT to accomplish the transition:

a.    The CMT will use the attached “Bridging Document” to familiarize themselves with the changes between the AC 120-92A Framework and the SMSVP Standard. The CMT shall ensure that the certificate holder has revised its plan to address the appropriate SMSVP Standard references and has made a status determination for each requirement on the revised plan.

b.    The CMT will review the certificate holder’s status claim and decide if:

i.    The expectation has been met;

ii.   The expectation requires revision; or

iii.  The expectation remains “in-progress” (still under initial development).

c.    The CMT will provide the certificate holder written notification of any status disagreement and upon acceptable correction by the certificate holder, accept the revised plan as formal conversion to the SMSVP.

NOTE:  Although the SMSPO is the final authority on the SMSVP standards differences of opinion over revised plan suitability between the CMT and certificate holder may be referred to the SMS Regional Point of Contact (RPOC) for resolution. The CMT and/or RPOC may request assistance from the SMSPO to answer any technical questions, or request a meeting in facilitating the transition process.

From the certificate holder’s revised Implementation Plan, the CMT will develop its Validation Project Plan using the guidance contained in this chapter (see Volume 17, Chapter 4, Section 2, subparagraph 17-4-2-3E) and complete all remaining validation work in accordance with this document.

NOTE:  The CMT may request assistance from the SMSPO to assist with development of the validation project plan.

3.    PTRS Procedures. The person with “transition plan oversight” will open a PTRS record to record CMT completion of the SMS transition from SMSPP to SMSVP activities:

i.    Enter activity number 1045/3045/5045 as appropriate;

ii.   Enter “SMSVPIPT” (SMS Implementation Plan Transition) in the “National Use” box; and

iii.  Record any additional information in the Comments section, as required.

Figure 17-4-3T.  Bridging Document Differences Between AC 120-92A and the SMSVP Standard

The following table lists the differences between the Advisory Circular (AC) 120-92A, Appendix 1 Framework and the SMS Voluntary Program Standard.

The document is intended to assist CMT’s transition from the SMS Pilot Project to the SMSVP expectations. It may also be used by a certificate holder to assist in documenting changed expectations in its SMS Implementation Plan.

Disclaimer: FAA certificate-holding offices are not obligated to accept or reject a certificate holder submission using this document. The SMSVP Standard is the primary reference to be used in the SMSVP.

Figure 17-4-3U.  Definitions

A.    Causal Factors. Causal factors are that set of elements that affect an event’s outcome. A causal factor is not necessarily a root cause, because whereas removing a causal factor can benefit an outcome, it does not with certainty prevent recurrence of an undesirable event. (See “root cause” and “root cause analysis.”)

B.    Conformance. Means agreement in nature or form of a presented document, process, or system.

C.    Continued Operational Safety (COS). Routine recurring Performance Assessments (i.e., routine surveillance through safety inspections). Also includes certificate management, the management of major changes in operation (i.e., system configuration changes).

D.    Corporate Safety Risk Management (SRM). As used in this document is a process to identify hazards and associated risks, analyze risks, and develop new risk controls affecting multiple process owner areas/departments within the organization. Final risk acceptance for Corporate SRM may be accomplished at a management level above the process owner/department level, or by a committee.

E.    Corrective Action. A corrective action addresses a nonconformity that has occurred.

F.    Design Demonstration. An activity that demonstrates, for purposes of validation, that a certificate holder’s design of safety management processes function in an operational environment.

G.    Design Review. Determines if a certificate holder’s safety management processes conform to the Safety Management System Voluntary Program (SMSVP) Standard.

H.    Gap Analysis. Compares existing processes, procedures, programs, and activities to the SMSVP Standard.

I.    Hazard. Means a condition that can foreseeably cause or contribute to an aircraft accident as defined in Title 49 of the Code of Federal Regulations (49 CFR) part 830, § 830.2.

J.    Preventive Action. A preventive action addresses the potential for a nonconformity to occur.

K.    Risk. Means the composite of predicted severity and likelihood of the potential effect of a hazard.

L.    Risk Control. A means to reduce or eliminate the effects of hazards.

M.    Root Cause. The root cause of a nonconformity or undesirable event is that factor that would with certainty result in the event not occurring were it not present.

N.    Root Cause Analysis (RCA). A method for identifying the underlying causal factor of a nonconformity or undesirable event. A causal factor is considered the root cause if its removal from the event sequence prevents the undesirable event from recurring.

O.    Root Cause Analysis Corrective Action Plan. A formalized plan to eliminate the causal factor that resulted in a nonconformity or undesirable event by addressing the factor determined to be the root cause.

P.    Safety Assurance. Means processes within the SMS that function systematically to ensure the performance and effectiveness of safety risk controls and that the organization meets or exceeds its safety objectives through the collection, analysis, and assessment of information.

Q.    Safety Management System (SMS). Means the formal, top-down, organization-wide approach to managing safety risk and assuring the effectiveness of safety risk controls. It includes systematic procedures, practices, and policies for the management of safety risk.

R.    Safety Objective. Means a measurable goal or desirable outcome related to safety.

S.    Safety Performance. Means realized or actual safety accomplishment relative to the organization’s safety objectives.

T.    Safety Policy. Means the certificate holder’s documented commitment to safety, which defines its safety objectives and the accountabilities and responsibilities of its employees in regards to safety.

U.    Safety Promotion. Means a combination of training and communication of safety information to support the implementation and operation of an SMS in an organization.

V.    Safety Risk Management (SRM). Means a process within the SMS composed of describing the system, identifying the hazards, and analyzing, assessing and controlling safety risk.

W.    System. Means a group of interacting, interrelated, or interdependent elements forming a complete whole.

X.    Validation. CMT activities involving observations, audits, and certificate management functions that provide sufficient information for the CMT to assess whether a certificate holder’s system design achieves stated objectives and meets published SMS standards.

Y.    Validation Plan. Means a forecast of resources needed to perform applicable assessments to confirm a certificate holder’s safety management activities and processes.

17-4-3-9 through 17-4-3-23 RESERVED.