Summary

Introduction

Discussion

2.1 The whole document is to promote the introduction of a Performance Based Management (PBM). It is not prescriptive but it shall serve as ICAO general guidance material. Beside ICAO Panels, PIRGs (Planning and Implementation Regional Group) and State aviation regulators, operators and service providers it aims to address all levels of personnel in these organizations from senior management to front-line workers being responsible for designing, implementing and carrying out effective performance management activities. Furthermore “getting started” assistance shall be provided to those ATM community members who are (relatively) inexperienced and the manual shall assist experienced ATM community members in converging towards a globally harmonised and agreed approach.

2.2 Therefore the manual delivers in chapter 1 – introduction – an explanation of the principles and advantages of a performance based approach without a special relationship to ATM. Because other industries have been using PBM for decades for improving the effectiveness of their day-to-day economic management, for channelling their efforts towards better meeting stakeholder expectations and improving customer satisfaction, and for managing change in a dynamic environment it was decided that the future ATM system shall be “performance based”. A strong focus on desired/required results, an informed decision making process, driven by the desired/required results and the necessary reliance on facts and data for decision making PBM promise to result in a more efficient ATM system. Cost savings are to be expected, even if at the beginning investments will be necessary to establish such management system. More transparency in policymaking due to publicly stated performance targets, the promotion of accountability and seeking solutions orientated on the desired performance instead of a “technology driven approach” are other possible positive effects of a performance based approach. Regarding interdependencies and the need for collaboration the introduction mentions that these have not always been recognised in the past, resulting in non-optimum performance of the Air Navigation System as a whole. To improve that in the future there will be a need of cooperation between all ATM community members in a performance based manner across geographic boundaries, across operational (e.g. flight phase) boundaries, across subject areas and across planning and management activity boundaries.

2.3 While the whole document has been written from a high level management point of view chapter 1 delivers a detailed technical consideration about what is PBM/PBA and which effects may be expected. Regarding commitment the manual stated that decision makers (e.g. senior management) need to be convinced that the performance based approach (PBA) will bring real value. Nothing is said about front-line operator involvement or the commitment of other personnel responsible for the conversion of PBA. Furthermore it says little about mistakes possible to occur during introduction or application of PBM and about human factors in a performance based management environment. But even if it is obvious that the application of Performance Based Management may be a helpful tool for solving a lot of the problems of the current ATM system, it will be difficult to reach the agreed targets without understanding and assistance of concerned employees.

2.4 Content of chapter 2 is “The Performance Based Approach Step-by-Step”. It provides a description according to the circle, as shown below, partially relating to ATM and underlined by a worked-out ATM example:

2.5 Even if this part of the manual is mainly a guideline for managers there are also a few aspects being of interest from a point of view of IFATCA or ATCOs.

2.6 The aim of step 1 (Define/Review Scope, Context & General Ambitions / Expectations) is to reach a common agreement on scope and a common view on the general nature of the expected performance improvements. Performance improvements in ATM shall be related to the eleven Key Performance Areas (KPAs)(safety, security, environmental impact, cost effectiveness, capacity, flight efficiency, flexibility, predictability, access and equity, participation and collaboration, interoperability), covering all expectations (external perspective) of the ATM community and the ambitions (internal initiative) in regard to the performance of flight operations, airspace/airport usage and air navigation services.

Because of simultaneous – and often interrelated – applications of performance management at more specialised and localised levels scope definition is important to avoid misinterpretations regarding expected performance improvement. Likewise, limits of responsibility and accountability will be set. Additionally, it is important to identify external drivers and constraints and necessary cooperation and coordination.

2.7 In step 2 (Identify Opportunities, Issues & Set (new) Objectives) strength and weaknesses (both internal) and opportunities and threats (all external) have to be analysed. A good understanding of the opportunities and issues (threats and weaknesses) is essential to select fit performance objectives to define the performance measurement accordingly and to agree how/where to change the system. Because it is not needed that everything has to be performance managed, focus is necessary to translate general expectations (KPAs) into preferred specific performance objectives.

That happens in a two-stage process. At first one identifies focus areas within each KPA. Secondly performance objectives as a desired trend must be defined within a focus area, but without saying when, where, who or how much has to be achieved.

2.8 The following step 3 (Quantify Objectives) is dealing with the principle “Reliance on facts and data for decision making”. This principle requires that objectives should be specific, measurable, achievable, relevant and time-bound (SMART). Hence definitions of performance indicators, underpinning metrics and common definitions for data aggregation and event classification are needed. Defining indicators should have a specific performance objective in mind. According to the ground rule “If you can’t measure it, you can’t manage it” a performance objective constrained by data availability has to be chosen.

Performance targets being certain values of performance indicators may be set with different intentions for example as a mandatory performance requirement which is necessary for safety reasons like the establishment of a safety management system or as a legal requirement. To understand the challenge for reaching the target the baseline performance should be known. Usually this baseline performance values have been calculated during the previous iteration of the process.

The specific of the Air Navigation System as a world wide system requires appropriate decision-making / policy-making processes to be in place to agree collaboratively on performance objectives, performance indicators and the values of performance targets at local, regional and — where required —global level.

2.9 Step 4 (Select Solutions to Exploit Opportunities and Resolve Issues) follows the principle of “Informed decision making, driven by the desired/required results”. Based upon knowledge about baseline performance, opportunities and issues with the performance objectives and targets an optimised decision making process in terms of priorities, tradeoffs, selection of solutions and resource allocation shall be reached. Prior to the selection of solutions it shall be explained what will be improved and how much. Then the dominant constraining factors need to be tackled first. It is interesting to know that possible solutions depend on time horizon. If the performance objective is part of the day-to-day economic and operational management, one will use improvements, which are readily available, like off-the-shelf solutions and best practices. If there is a longer time horizon (5 years or more) decision makers will have to work with a “living” list of options, because potential options for improvements may be still in Research, Development and Trials phase, enclosing some degree of uncertainty. Nevertheless decision makers should take into consideration the strategic fit, the benefits, cost and feasibility of each option for operational improvement with a focus on (beside other) safety and human factors assessment. The assessment shall be guaranteeing feasibility of the operational improvement from a human factors and safety perspective and shall be announcing a first list of issues that need to be tackled during the development life cycle.

2.10 In the execution phase of the performance management process – step 5 (Implement Solutions) – the changes and improvements are actually implemented, and start delivering benefits.

2.11 Chapter 3 “Getting Started” delivers some practical tips of organisational nature for those ATM community members who are (relatively) inexperienced in applying the Performance Based Approach. For the development of a good understanding of PBA it is recommended, to assess currently used approach, to define implementation, to start with a limited s scope and to establish commitment and collaboration. To get the necessary internal support for information/data collection, processing, storage and reporting is a matter of commitment and should involve the exclusion of any data use for performance assessment of individual employees.

2.12 The Appendices (chapter 4) provide more specific or technical guidance for a selected set of subjects.

2.12.1 Appendix A includes abbreviations and terminology definitions, used in the manual.

2.12.2 Appendix B – Performance Framework recognises that PBA can be applied at different levels, ranging from high level socio-political issues down to technology and that these levels have performance impact on each other. Furthermore the need for a harmonised, structured view of the Air Navigation System is emphasized to enable the harmonisation of performance indicators and supporting metrics.

2.12.3 Risk management, being part of Appendix C – Priorities, Trade-Offs and Risks, as a procedure, is mainly used in the safety and security areas but it is also employed in the other KPAs. Regarding priorities is stated that safety is always a priority in aviation. The importance of other KPAs will be imposed by local circumstances. The spectrum of Air Navigation System performance management, expressed by the 11 KPAs, is very broad. Mutual interdependencies of many of these areas stand for that performance improvement in one area may only be achieved by reduced performance in another area. Therefore a “balanced approach” is necessary as the result of trade-off decision making between the various performance objectives and targets.

2.12.4 In Appendix D – Data and Information Management it is emphasised that each of the participants must be convinced that it is worthwhile to spend the extra effort for data collecting. Additionally, certain data, for example safety or financial related data, may be quite sensitive and an appropriate confidentiality has to be granted. Remarkable is the advice that “a good climate should be established, and a no-blame reporting culture should grow.” for guaranteeing that supplied data will not be used in an inappropriate way.

2.12.5 Appendix E – Performance Analysis and Modelling explains the role of them in PBA, connected with some practical guidance for performance modelling and a suggested checklist for developing and using a performance model.

2.12.6 Appendix F – Metrics and Indicators shows the result of the attempt to define globally harmonised ATM performance indicators. Based on a comparison of ATM performance indicators used in two organizations for the eleven KPAs no identical indicator was found. No commonality in indicators could be identified in the following KPAs:

• Access and Equity
• Flexibility
• Environment
• Participation by the ATM Community
• Predictability
• Security

2.13 Within the safety key performance area it was only tried to find common metrics regarding accidents. But already with such a small focus differences emerged in the definition of terms and the filtering criteria. Keeping in mind that accidents are merely a small part of the safety area one wonders how more than a local performance based approach to this objective shall be reached.

2.14 Capacity area falls into three types: system-wide, airspace and airport capacity. While for system-wide capacity common indicators could not be found, the measures at airspace level are conceptually similar. Presently airspace capacity is measured either as the number of IFR flights able to enter an airspace volume, or as the agreed-upon airspace capacity rates which is the number of IFR flights able to be present in sectors at any one time. Problems arise because the agreed- upon rates is a subjective evaluation used for traffic-flow management and have not been objectively determined. Indicators used for determining airport capacity are based upon the number of movements per unit of time that can be accepted during different meteorological conditions. To reach common airport capacity indicators further harmonisation is needed in regard to definitions of specific conditions (fleet mix, configuration) and methods for its calculation and regarding averaging methods (annual, hourly or daily) and aggregation into a single airport or groups of airports. Common cost effectiveness indicators will require an agreement on whether normalization should be conducted per flight hour or operation is necessary. In addition, types of contained operations (e.g., IFR only), components of cost to be included in the cost calculation and a minimum level of reporting time (monthly, annual) have to be standardized. Cost of ATM today are normalized by average cost per flight at a system-wide annual level, total operating cost plus cost of capital divided by IFR flights and total labour obligations. The KPA Efficiency is strongly associated with delay performance. Difficulties result from different definitions and filtering criteria, used by different organizations. For example, delays may be related to percent of flights departing on-time or may be related to percent of flights with normal flight duration. Moreover, while for one organization a delay is already given with 3 minutes or more behind schedule for another organisation this means 15 minutes or more. Global Interoperability shall be assessed against level of compliance with international standards. For a common indicator the specific definition of a global interoperability requirement and more specificity in the measurement technique for the level of compliance have to be developed.

Conclusions

3.1 The Global Performance Manual has been written primarily from a high level management point of view. It delivers an extensive description of how to introduce a performance based approach and how to apply it even if the relation to ATM remains relatively weak. Consequently the different expectations of the ATM community regarding a performance based ATM and how to deal with them were not discussed. It says little about mistakes possible to occur during introduction or application of PBM and about human factors in a performance based management environment. Also effects on employees, the required acceptance and possible involvements of them were mentioned only peripherally. Even if according to step 4 of the Manual for each option of operational improvement an assessment shall be made to guarantee feasibility from a human factors and safety perspective it will be difficult to reach something without the understanding and assistance of concerned employees.

3.2 Completely insufficient is the situation regarding performance indicators for the Key Performance Areas. Without one globally agreed indicator one of the most important preconditions for a globally harmonized and agreed approach of Regions and States towards a performance based ATM is presently not given. Based on this it has to be expected that also the measurement methods will differ between States and Regions so that a comparison or a common database will not be available in near future. Perhaps the manual is of little or no interest for the single ATCO. But MAs being representatives of legal and professional issues of ATCOs should try to understand the whole process with a special focus on data collecting analyzing reviewing and reporting. Performance measurement will play the key role in a performance based ATM and will much more take place in our working environment than today. Because ATCOs contribute to ATM performance in each of the KPAs via ATC performance, they are to be involved into all phases of creating an ATC performance measurement system. The performance metric will influence the behaviour of controllers so it would be prudent if senior managers give them a say-so in the development of a system that will drive their performance.

3.3 IFATCA supports the introduction of Performance Based Management in ATM. IFATCA accepts that therefore a performance management system in ATM will be needed.

Recommendations

4.1 To insert on page 4 1 1 8:

“IFATCA urges MAs to be involved in the creation of and application of an ATM Performance Measurement System.”

4.2 That the EB communicates to ICAO that front-line operators ought to be involved in the creation of, and application of an ATM Performance Measurement System.

References

ICAO “Manual on Global Performance of the Air Navigation System”.