The role of Key Performance Indicators (KPI’s) in the context of and an EASA Compliant Aviation Safety Management System

Sofema Online (SOL) www.sofemaonline.com considers the primary purpose of KPIs within an Aviation SMS.

Introduction

Introduction – Risk Assessment & Management KPI’s

KPIs typically measure how well your operation is doing at achieving its goals, for example, by reducing downtime or costs. KPIs can be used as a benchmark to understand your organisation’s current position as well as to facilitate the development of Key Performance goals.

Key Performance Indicators (KPIs) vary according to the company, its goals, strategies, and action plans. However, there is a set of indicators that is well-regarded and used more often. Consider the following Features:

• The selection of KPIs can relate to many different elements and aspects including financial – efficiency of the process – health and safety (SMS) considerations – defects and pilot reports (PIREPS).

In the context of an EASA-compliant aviation Safety Management System (SMS), Key Performance Indicators (KPIs) are crucial for measuring and monitoring the safety performance of the organization.

• Alignment with Safety Objectives - KPIs should be directly aligned with the organization's safety objectives and goals. This ensures that the indicators provide meaningful insights into how well the organization is achieving its desired safety outcomes.

• Proactive and Reactive Measures - KPIs should encompass both proactive and reactive measures

o Proactive KPIs focus on identifying potential safety issues before they result in incidents or accidents. Examples include safety audits, risk assessments, and safety training completion rates.
o Reactive KPIs measure the outcomes of past events to understand trends and areas of concern. Examples include incident rates, accident rates, and findings from safety investigations. 

• KPIs should include both quantitative and qualitative data:

o Quantitative KPIs provide measurable data, such as the number of safety reports filed, the frequency of safety meetings, or the rate of safety training completion.
o Qualitative KPIs capture more subjective data, such as feedback from safety surveys, employee perceptions of safety culture, and qualitative assessments of safety performance.

• Risk-Based Approach - KPIs should be developed based on a risk-based approach, focusing on areas with the highest potential for significant safety impacts.
• Transparency and Communication

o Effective KPIs should be communicated transparently within the organization. This involves reporting KPI results to all relevant stakeholders, including management, staff, and regulatory authorities, to foster a shared understanding of safety performance and encourage a culture of safety.

Introduction – Safety Performance Indicator

A safety performance indicator (SPI) is defined in the ICAO Safety Management Manual as a measure (or metric) used to express the level of safety performance achieved in a system.

• Generally expressed in terms of the frequency of occurrences of some event causing harm, e.g., A number of events/year or / 000’s of Hrs of operation.
• A set of indicators should also include leading indicators.

Note: These indicators do not require a safety event to take place and are metrics that provide information on the current situation that may affect future performance.

What is the main difference between Safety Performance Indicators (SPI) and Safety Performance Targets (SPT)?

ICAO defines:

• Safety Performance Indicators are “A data-based parameter used for monitoring and assessing safety performance”.
• Safety Performance Targets as “The planned or intended objective for safety performance indicator(s) over a given period”.

SPI/SPT Development Considerations

• Specific understanding regarding the selection of the relevant safety concern to be treated to ensure both validity & relevance.
• Avoid complexity wherever possible – simple is better and can clearly reflect relevant safety concerns.
• Use clear definitions to avoid ambiguity.
• Phased approach to implementation – achieves the ultimate objective in stages.
• Avoid high workload analysis.
• Ensure a broad coverage is important do not only pursue extremes (to capture systemic exposure that can agglomerate into more significant events.
• Make sure time frames are clear and understood (data capture rate).
• Avoid quantity over quality; measure what is important and not easy.

Leading & Lagging SPIs and SPTs

Lagging indicators are reactive in nature and measure an organization’s performance, such as the number and types of incidents that occurred based on information from past incidents and accidents, such as reactionary analysis. Reactionary analysis is analyzing past data to find loopholes in processes and policies.

Leading indicators are proactive and preventive measures that can shed light on the effectiveness of safety and health activities and reveal potential problems in a safety and health program.

Many people are familiar with lagging indicators:

• Incorrect application of the MEL / 1000 Departures
• Missed Airworthiness Directive (AD) / 10,000 FH
• Leading SPIs/SPTs – sometimes known as “Process SPIs/SPTs” – measure situations which have the potential to become or contribute to high severity/low probability negative outcomes:
• Leading SPIs support proactive development of the organisation’s safety management System – Capability to improve safety exposure.

o SPI – 0.75 events of foreign object damage FOD) to aircraft per 10,000 Maintenance Activities (Ramp Transit & Minor Check).
o SPT: To reduce to 0.5 events of foreign object damage to aircraft per 10 000 Maintenance Activities (Ramp Transit & Minor Check) within 12 months from the introduction.

Implementation of Safety Performance Indicators

Safety performance indicators are important because they measure the proper functioning of the organisational barriers that exist as well as consider also the interfaces between our organisation and customers as well as other related (contracted & subcontracted) organizations.

Once a detailed assessment has been made, it is possible to consider the most relevant indicators applicable to the organisational objectives.

• How do these indicators relate to each other, and do they meet the requirements?
• Is the measurement process of the chosen indicators effective?

Notes

• KPIs are not Safety Performance Indicators (SPI) nor are they directly related to the achievement of safety objectives (although they can be a feature within the overall safety management system).
• Whilst KPIs are closely related to the performance level you want to achieve, don’t take them for goals themselves. (This means an indicator is only a metric that we use quantitatively to demonstrate the performance of a given maintenance activity, asset, or department.)
• When developing KPIs, it is important that they relate to the entire process, including the various inputs and outputs, so that they are able to benchmark the performance of the maintenance genuinely.

Next Steps

Sofema Aviation Services (SAS) www.sassofia.com and Sofema Online (SOL) www.sofemaonline.com provide Safety and Risk Management Training as classroom, webinar or online. Please see the websites or email [email protected]