ANSP Key Performance Indicators (KPIs)

  • Home 2023 ANSP Key Performance Indicator....

ANSP Key Performance Indicators (KPIs)

62ND ANNUAL CONFERENCE, Montego Bay, Jamaica, 8-12 May 2023

WP No. 154

ANSP Key Performance Indicators (KPIs)

Presented by PLC

 

Summary

Many Air Navigation Service Providers (ANSPs) use Key Performance Indicators (KPIs) to evaluate the success of their organisation towards achieving their goals of providing a safe and efficient air traffic control service. It has been suggested that IFATCA should similarly develop a framework of KPIs that could be used to monitor the performance of ANSPs towards goals that are considered important to IFATCA. Such a framework, once identified, could be further used as an incentive to improve performance.

This working paper aims to derive from existing IFATCA policy a number of performance goals that IFATCA believes important and develop a framework of KPIs by which to assess, and benchmark, the effectiveness of ANSPs in achieving these goals.

Introduction

1.1 The IFATCA Constitution and Administrative Manual identifies clearly one of its primary objects as being “To protect and safeguard the interests of the Air Traffic Control Profession” and outlines that, in order to achieve this objective, IFATCA will “Collect and distribute information on professional problems and developments”. These two primary goals form the basis of the discussion presented by this working paper.

1.2 In May 1979 the International Labour Organisation held a Meeting of Experts in Geneva to address a single agenda item “Problems concerning air traffic controllers: identification and possible solutions”. The meeting, attended by IFATCA, set out to discuss professional issues facing Air Traffic Controllers and to suggest solutions through the setting of global standards that should be adopted by ANSPs and Associations in order to safeguard the profession.

1.3 These standards, presented as the conclusions of the report of the ILO meeting, form the basis of IFATCA policy material today and represent the goals of IFATCA in the interest of protecting the Air Traffic Control profession.

1.4 Often at odds to the goals of IFATCA are the goals of the ANSPs whose aims primarily focus on the operation of a safe and efficient, and in the case of privatisation, profitable, Air Traffic Control service. It is important to note, though, that the goals considered most important often differ between various ANSPs: safety, working hours, “Just culture” concept implemented or not, etc.

1.5 In order to address such differences and to better monitor their performance towards these goals on a global scale, many ANSPs and regulators have developed combined frameworks of Key  Performance Indicators to facilitate global benchmarking (e.g. EUROCONTROL, FAA, CANSO and IATA). These frameworks allow member ANSPs to highlight areas of possible improvements, provide incentives to poor performers, and allow the sharing of performance information and initiatives between members.

1.6 At present, however, there is no single framework of Key Performance Indicators which allows the benchmarking of the outcomes considered important by IFATCA, particularly in the protection and safe-guarding of the Air Traffic Control profession.

1.7 PLC was tasked by directors to investigate that the IFATCA PLC should discuss and address this issue.

1.8 This working paper therefore proposes a framework of Key Performance Indicators by identifying performance goals from existing IFATCA policy and to produce guidance material to allow benchmarking of performance between the ANSPs with the interest of protection and safe-guarding of the Air Traffic Control profession.

Discussion

2.1 ICAO Vision

2.1.1 ICAO sets out eleven Key Performance Areas (KPAs):

  • Access and equity;
  • Capacity;
  • Cost effectiveness;
  • Flight efficiency;
  • Environmental impact;
  • Flexibility;
  • Global interoperability;
  • Participation by the Air Traffic Management (ATM) community;
  • Predictability;
  • Safety; and
  • Security.

2.1.2 Appendix D of the Global Air Traffic Management Operational Concept (GATMOC) (ICAO Doc 9854) describes each of them:

  • Access and equity. A global air navigation system should provide an operating environment that ensures that all airspace users have the right of access to ATM resources needed to meet their specific operational requirements; and ensures that the shared use of the airspace for different equity for all airspace users that have access to a given airspace or service. Generally, the first aircraft ready to use the ATM resources will receive priority, except where significant overall safety or system operational efficiency would accrue or national defence considerations or interests dictate by providing priority on a different basis.
  • Capacity. The global air navigation system should exploit the inherent capacity to meet airspace user demand at peak times and locations while minimizing restrictions on traffic flow. To respond to future growth, capacity must increase, along with corresponding increases in efficiency, flexibility, and predictability while ensuring that there are no adverse impacts to safety giving due consideration to the environment. The air navigation system must be resilient to service disruption and the resulting temporary loss of capacity.
  • Cost effectiveness. The air navigation system should be cost effective, while balancing the varied interests of the ATM community. The cost of service to airspace users should always be considered when evaluating any proposal to improve ATM service quality or performance. ICAO guidelines regarding user charge policies and principles should be followed.
  • Efficiency. Efficiency addresses the operational and economic cost effectiveness of gate-to-gate flight operations from a single-flight perspective. Airspace users want to depart and arrive at the times they select and fly the trajectory they determine to be optimum in all phases of flight.
  • Environment. The air navigation system should contribute to the protection of the environment by considering noise, gaseous emissions, and other environmental issues in the implementation and operation of the global air navigation system.
  • Flexibility. Flexibility addresses the ability of all airspace users to modify flight trajectories dynamically and adjust departure and arrival times thereby permitting them to exploit operational opportunities as they occur.
  • Global interoperability. The air navigation system should be based on global standards and uniform principles to ensure the technical and operational interoperability of air navigation systems and facilitate homogeneous and non-discriminatory global and regional traffic flows.
  • Participation by the ATM community. The ATM community should continuously be involved in the planning, implementation, and operation of the system to ensure that the evolution of the global air navigation system meets the expectations of the community.
  • Predictability. Predictability refers to the ability of the airspace users and air navigation service providers to provide consistent and dependable levels of performance. Predictability is essential to airspace users as they develop and operate their schedules.
  • Safety. Safety is the highest priority in aviation, and ATM plays an important part in ensuring overall aviation safety. Uniform safety standards and risk and safety management practices should be applied systematically to the air navigation system. In implementing elements of the global aviation system, safety needs to be assessed against appropriate criteria, and in accordance with appropriate and globally standardised safety management processes and practices.
  • Security. Security refers to the protection against threats, which stem from intentional (e.g. terrorism) or unintentional (e.g. human error, natural disaster) acts affecting aircraft, people or installations on the ground. Adequate security is a major expectation of the ATM community and of citizens. The air navigation system should therefore contribute to security and should be protected against security threats. Security risk management should balance the needs of the members of the ATM community who require access to the system with the need to protect the air navigation system. In the event of threats to aircraft or threats using aircraft, ATM shall provide responsible authorities with appropriate assistance and information.

2.1.3 Achievable performance in the areas listed above is made possible by the following enabler levels:

  • services and procedures;
  • human resources;
  • physical infrastructure;
  • systems and technology; and
  • regulation and standardization.

2.1.4 In the Fifth Edition of Global Air Navigation Plan (GANP), ICAO proposes to work on a set of KPIs, according to “needs and capabilities”. States are encouraged to start with a simple set of indicators (Core KPIs) matching their needs, and to complete them later with more complex ones (Additional KPIs). However, neither of them, not even the Core KPIs, takes specifically into consideration Human Factors:


2.2 Existing IFATCA Policy

2.2.1 The main goals for IFATCA are, and always will be, Safety and Professionalism. The purpose of defining KPAs and KPIs is to help achieve these goals. Nevertheless, current frameworks proposed by CANSO, IATA or ICAO completely overlook Human Factors and the related aspects: working environment, Joint Human-Machine Systems, system design, ergonomics.

2.2.2 The core of the entire aviation system, the most valuable resource, is the human resource. This is the reason why any evaluation should focus on human resources specifically. From this point of view, research of existing IFATCA policy identified several KPAs from which a framework of Key Performance Indicators could be developed. They are not new: there were there for quite a long time, but there were not seen as KPIs until now.

2.2.3 Industrial Relations (WC 10.1)

WC 10.1.8 Performance indicators

Global metrics for the performance of the Air Traffic Management System shall be developed through ICAO processes as soon as possible.

Controller expertise shall be used in the establishment and settings of metrics that measure the performance of the Air Traffic Management System.

Controller expertise shall be used in establishing and reviewing models used for determining performance of the Air Traffic Management System to ensure that the models accurately reflect how the ATM system functions.

Controller expertise shall be used in the interpretation of data used to assess the performance of the Air Traffic Management System to ensure that data is not misleading because it is incomplete or incorrectly applied.

The measurement of performance of the ATM system shall reflect the impact of any external-to-ATM constraints, including external environmental constraints.

IFATCA urges MAs to be involved in the creation and application of an ATM Performance Measurement System.

 

2.2.4 Social and Labour Aspects (WC 10.2)

WC 10.2.2 Management of ATS providers
ATC management staff directly concerned with executive air traffic control matters should have a thorough knowledge of air traffic control and be holders of an air traffic controller’s license and, to remain fully conversant with current air traffic control problems, their knowledge should be continually updated. Where ATS providers are controlled by senior management who do not have experience as senior Air Traffic Controllers then the position of Chief Air Traffic Controller shall be created. The Chief Air Traffic Controller shall be an experienced senior air traffic controller and shall be answerable for, amongst other things, the safety of the air traffic control system. The Chief Air Traffic Controller shall report directly to the Chief Executive Officer of the ATS provider and to the head of the regulatory organization.

 

WC. 10.2.2.1 Monitoring privatisation / commercialisation in ATC

The safety and quality levels of the Air Traffic Services system shall not be compromised by privatization/commercialization.

IFATCA should monitor the effects of privatization/commercialization on ATCOs working conditions in co-operation with the ILO.

 

WC 10.2.3 Working environments

Great attention should be paid to the lighting conditions of working positions. Indirect light, adjustable by zones, similar to daylight provoking neither reflections nor dazzling is very important. The luminosity should be adjustable at each working place and light sources be cleaned regularly and replaced if necessary.

The physical working environment regarding control room temperature, lighting, relative humidity, adapted rest areas and facilities for eating and drinking shall be designed so as to facilitate night shift demands.

 

WC 10.2.4 ATC systems

Research should be carried out in each country to determine the capacity of the ATC system and the workload to be carried by each air traffic controller.

Air traffic controllers should be provided with ATC equipment commensurate with their operational requirements so as to promote an optimum level of safety.

 

WC 10.2.5 Automation / Human Factors

Automation shall assist and support ATCOs in the execution of their duties.

The controller shall remain the key element of the ATC system.

Total workload should not be increased without proof that the combined automated/human systems can operate safely at the levels of workload predicted, and to be able to satisfactorily manage normal and abnormal occurrences. Automated tools or systems that support the control function shall enable the controller to retain complete control of the control task in such a way so as to enable the controller to support timely interventions when situations occur that are outside the normal compass of the system design, or when abnormal situations occur which require non-compliance or variation to normal procedures.

 

WC 10.2.7 Working with unserviceable or inadequate equipment

Controllers should be given initial and recurrent training in the degraded mode operations of their equipment.

 

WC 10.2.12 ATM safety monitoring tool

The system should not be used as a performance monitor for individual controllers. Analysis of any derived data should be undertaken by appropriately experienced and trained ATM safety experts.

 

2.2.5 Hours of Work (WC 10.3)

WC 10.3.1 Duty rosters

The duty roster should be based on at least 2 consecutive days off in every 7 days. Duty rosters should be agreed with the air traffic controllers involved. Single Person Operation (SPO) shall be avoided.

 

WC 10.3.2 Work and rest scheme

The average time of operational duty and breaks should not exceed 32 hours per week.

 

WC 10.3.3 Vacation scheme

The annual leave for a controller should be not less than 30 working days (this is the equivalent of 6 weeks), excluding public holidays, of which 3 weeks shall be consecutive.

 

WC 10.3.5 Extra Duty

Extra duty should be voluntary and used only in exceptional situations.

In the interest of aviation safety and the well being of the controller population, extra duty control should be considered as an undesirable method of staffing Air Traffic Control positions and should be avoided.

 

WC 10.3.7 Ageing ATCOs

ANSPs should offer career development plans as medium to long term alternatives to the operational job.

Training courses for ATCOs regarding the issue of ageing should be made available.

ATCOs with an age of 50 years and older shall be entitled to abstain from nightshifts on their request.

Ageing ATCOs should be entitled to specific break plans, in particular additional short breaks, to assist in their performance with short term memory.

Ageing ATCOs should be entitled to reduce the number of their ratings and / or endorsements to a reasonable minimum. Such a reduction shall have no detrimental impact on the individual ATCO.

 

2.2.6 Retirement and pension (WC 10.5)

WC 10.5.1 Retirement

IFATCA recommends that for active air traffic controllers the age of retirement should be closer to 50 than 55.

 

2.2.7 Employment Security (WC 10.6)

WC 10.6.1 Loss of license

Opportunities for re-training and re-deployment, with compensation for loss of income, should be available to Air Traffic Controllers who lose their license.

 

2.2.8 Miscellaneous (WC 10.7)

WC 10.7.3 Safety management systems

Human Factor issues shall be accounted for in each phase of the definition, development, and deployment of new and existing ATM systems and into operational training. Controllers and human factors experts shall be involved from the beginning of any new project.

 

WC 10.7.6 Cognitive processes in ATC

Capabilities and limitations of cognitive processes shall be considered when addressing Human Performance and Human Factors.

 

2.2.9 Specific matters (MED 8.2)

MED 8.2.3 Critical Incident Stress Management

The Federation recognises the importance of stress management for air traffic controllers and recommends that, at regular intervals, air traffic controllers be provided with up-to- date information on stress management techniques.

The Federation urges MAs to bring to their administration’s attention the stress-inducing  potential of their work environment in order that particular consideration is given to ensure that the work environment is suitable and as stress-free as possible.

The Federation endorses the use of professionally trained peers in the provision of critical incident stress management (CISM) to colleagues experiencing critical incident stress (CIS).

Comprehensive and confidential support services should be available at all times for air traffic controllers, support staff and their families.

Professional critical incident stress support services should be made available to air traffic controllers involved in ATC incidents / accidents and any other occurrences that have potential to create critical stress reactions influencing the ATCO’s performance. It is the controllers’ choice whether or not to take advantage of these support services.

 

MED 8.2.4 Fatigue in Air Traffic Control

Management has the prime role for providing fatigue management and prevention of fatigue related catastrophes. Any situation where increased fatigue, decreased sleep, or performance loss can be demonstrated, is a situation where the margin for error is reduced, albeit by some unknown amount, and should be avoided in ATC.

The provision of a satisfactory working environment appropriate rostering, rest periods, facilities, use of overtime, relief controllers and education in human factors shall be agreed with the air traffic controllers involved. Attention must be given to individual differences, age and gender.

In exercising the responsibilities of designing of duty rosters (see WC 8.3.1), management shall be responsible for providing physical arrangements (relief controllers and adapted rest area) and sufficient break periods for controllers to try to maintain their daily eating habits regardless of which shift they are working. Such physical arrangements and sufficient break periods shall be provided to allow for strategic naps during night shifts.

Management shall approve the implementation of strategic naps as an effective way of improving alertness and anchoring the circadian rhythms of controllers during night shift.

Management shall in close coordination with the air traffic controllers involved, carefully consider staffing levels during night shifts. For those controllers who have very heavy traffic loads during the night shifts, additional relief should be considered as an appropriate countermeasure to sleepiness and fatigue in order to increase the safety margins, and to reduce subsequent daytime sleepiness.

Use of overtime shall generally be kept to a minimum, and a system for allocation of overtime which takes the limitations in human performance as a factor shall be established. The combination of overtime shortly before or just after night shifts shall be avoided.

 

2.2.10 Training after licensing (TRNG 9.5)

TRNG 9.5.1 Refresher courses

As well as programmed refresher courses, adequate courses of instruction should be provided prior to the introduction into the ATC system of new or modified equipment and changes to standards or procedures which may require additional skills or changes in operating techniques.

 

TRNG 9.5.2 English language training

Sufficient training shall be available for current ATCOs of all English language abilities so as to be able to meet the required ICAO level and subsequently to retain (or improve) that competency.

Staff who are unable to achieve and maintain the English language requirements shall have their positions protected and given opportunities to reach the required ICAO level.

 

2.2.11 Accident and incident investigation (LM 7.2)

LM 7.2.1 Just culture, trust and mutual respect

Just Culture shall be in the service of safety and by no ways a means of social control or disciplinary mechanism.

Any incident reporting system shall be based on the following principles:

a. Cooperation: with all those having a legitimate and appropriate interest;

b. Dissemination: distribution of safety-related data to all those with appropriate interest;

c. Confidentiality: for the whole procedure, guaranteed by law;

d. Protection: for those involved or mentioned in the report, the provision of which be within the remit of an independent body; and

e. Trust and mutual respect.

 

LM 7.2.4 Protection of identity

Protection of the identity(is) of ATM staff involved in incidents or accidents shall be guaranteed.

 

LM 7.2.5 Reference card

Member Associations should provide their members with information containing the basic rights and the rules that will be applied in case of incident/ accident investigations. This should include guidance on the reporting process, Just Culture, the investigation process, and the specific parts of the legal system that applies in their State(s). Special consideration should be given to any areas where the State has filed difference(s) from provisions contained in Annex 13.


2.3 Draft IFATCA KPAs and KPIs proposal

2.3.1 For IFATCA, ANSPs primary goals are Safety and Quality of their services. These two KPAs could be evaluated throughout KPIs such as:

  • Safety – Safety Management System, CISM, Just Culture, Fatigue Policy;
  • Quality of service provided – English language proficiency, Training after licensing, Social and Labour aspects.

2.3.2 Safety Management System (SMS) is mandatory and it would be nice to know that the reality fulfils the expectation.

2.3.3 Critical Incident Stress Management (CISM), Just Culture and Fatigue Policy are are very important programmes for aviation safety and implementation should be taken into serious consideration.

2.3.4 English language proficiency, Training after licensing and Social and Labour aspects could give an image of the Quality of services provided.

2.3.5 Not all Human Factors which are highlighted in chapter 2.2 are reflected in the proposed KPAs and KPIs, because it is not easy to quantify working environment, Joint Human-Machine Systems, system design or ergonomics. However, the proposed system of KPAs/KPIs should be updated in near future, in order to insure the measurement of Human Factors left outside.

2.3.6 The following expands on these KPIs and provides some context as to what the KPI will measure:

2.3.6.1 SAFETY

  • SMS

KPI01 – SMS implemented – Yes/No

  • CISM

KPI02 – CISM Programme implemented – Yes/No

KPI03 – Percentage of endorsement by management

Fully endorsed by management/ 50% endorsed / No endorsement

  • JUST CULTURE

KPI04 – Principles of Just Culture implemented:

– Cooperation;

– Dissemination;

– Confidentiality;

– Protection of identity; and

– Trust and mutual respect.

  • FATIGUE POLICY

KPI05 – Countermeasure to sleepiness and fatigue – Yes/No

2.3.6.2 QUALITY OF SERVICE PROVIDED

  • ENGLISH LANGUAGE PROFICIENCY

KPI06 – Program to achieve and maintain English proficiency implemented

– Yes/No

KPI07 – Hours of English language training/ATCO

  • TRAINING AFTER LICENSING

KPI08 – Hours of theoretical refresher/ATCO/year

KPI09 – Hours of practical exercises (SIM)/ATCO/year

  • SOCIAL AND LABOR ASPECTS

KPI10 – Hours of work principles implemented

– Duty roster should be based on at least two consecutive days off in every seven days;

– Average time of operational duty and breaks should not exceed 32 hours per week; and

– ATCO’s annual leave should not be less than 30 working days, excluding public holidays, of which 3 weeks must be consecutive.

KPI11 – Ageing ATCOs policy – Yes/No

– ATCOs with an age of 50 years and older are entitled to abstain from nightshifts on their request.

– ATCOs with an age of 50 years and older are entitled to specific break plans to assist in their performance with short term memory.

– ATCOs with an age of 50 years and older are entitled to reduce the number of ratings and/or endorsements to a reasonable minimum.

KPI12 – Remuneration principles implemented

– Equal remuneration for equal work.

– ATCOs remuneration reflects levels of skills, experience and the amount/type of ratings.

– Additional task reflected by a higher remuneration level.

KPI13 – Employment Security – Yes/No

2.3.7 Annex 1 of this document presents a questionnaire for the MAs voluntarily to complete.


IFATCA INFORMATION HANDBOOK (IHB)

IFATCA maintains a handbook to record “current information on all aspects relating to environmental and human factors in ATC in the countries represented in the Federation”. It serves as a repository of information provided by MAs and allows them to compare information on working conditions and professional matters. This information can be used when preparing to negotiate with employers.

KPAs and KPIs are another tool MAs can use. As such, the IHB is the logical place to record this information. The IHB contains a ‘questionnaire’ that MAs can voluntarily complete and which is the source of the information in the IHB. This questionnaire would need to be reviewed to ensure that it  can provide the information required to determine how employers are performing with respect to the agreed KPIs.

Should MAs broadly agree that applying IFATCA defined KPAs and KPIs to employers is worth pursuing, a review of the IHB questionnaire will be undertaken along further development of the KPA/KPI framework.

Conclusions

3.1 In order to better monitor ANSPs performance, IFATCA has developed a number of Key Performance Indicators, to facilitate global benchmarking. Using the table at para. 2.3.5, Member Association have the opportunity to assess ANSPs against key IFATCA policy areas.

3.2 This will permit member associations to highlight areas of possible improvements, and allow the sharing of performance information and initiatives between members.

Recommendations

4.1 It is recommended that this paper be accepted as information.

References

IFATCA IAM (Version 66.1 – August 2022), Part I, Article I, 3.1.

IFATCA TPM (Version 65.0 – July 2022).

ILO Meeting on Problems Concerning Air Traffic Controllers (Geneva, 8-16 May 1979), Report.

ICAO Doc 9854, First Edition, 2005 – Global Air Traffic Management Operational Concept.

Global Air Navigation Plan 2016-2030 (Doc 9750-AN963, Fifth Edition – 2016).

IFATCA TPM, Version 65.0, July 2022 – Industrial Relations (WC 10.1).

IFATCA TPM, Version 65.0, July 2022 – Social and Labour Aspects (WC 10.2).

IFATCA TPM, Version 65.0, July 2022 – Hours of Work (WC 10.3).

IFATCA TPM, Version 65.0, July 2022 – Retirement and pension (WC 10.5).

IFATCA TPM, Version 65.0, July 2022 – Employment Security (WC 10.6).

IFATCA TPM, Version 65.0, July 2022 – Miscellaneous (WC 10.7).

IFATCA TPM, Version 65.0, July 2022 – Specific matters (MED 8.2).

IFATCA TPM, Version 65.0, July 2022 – Training after licensing (TRNG 9.5).

IFATCA TPM, Version 65.0, July 2022 – Legal Matters (LM 7.2.1, LM 7.2.4, LM 7.2.5).

“ATCO Performance” – WP No. 162 at the 53RD Annual Conference, Gran Canaria, Spain, 5 – 9 May 2014, Presented by PLC.

“ATCOs and Professionalism” – WP No. 160 at the 53RD Annual Conference, Gran Canaria, Spain, 5 – 9 May 2014, Presented by PLC.

https://www.eurocontrol.int/publication/air-traffic-management-cost-effectiveness-ace-benchmarking-report-2020

https://www.faa.gov/about/office_org/headquarters_offices/ato/service_units/mission_support/ato_intl/benchmarking

Last Update: September 18, 2023  

September 17, 2023   101   Jean-Francois Lepage    2023    

Comments are closed.


  • Search Knowledgebase