Modern ATM systems are more and more complex. Safety critical tasks as Radar Data Processing or Flight Data Processing rely on complex software packages while the increasing volume of air traffic makes the ATM system more and more dependent on such software. As a consequence, a software failure may lead to a catastrophic situation. Therefore, new ATM systems must be carefully designed, tested and validated before being considered ready for implementation. This paper discusses the concept of readiness, the development process, and the need of controller involvement in the development of automated ATM systems.

This paper proposes new Policy on this subject.

1.1  Air Traffic Control systems have evolved to be more and more technology dependent. Procedural control gives way to radar control and manual flight data processing is replaced by computerized systems.

1.2  An extension in the automation level increasing the complexity of the ATM system is expected in the future. SESAR acknowledges this stating:

“to accommodate the expected traffic increase and complexity, an advanced level of automation support for the humans will be required”.

In a similar way we can read about NEXTGEN:

“Air traffic controllers will become more effective guardians of safety through automation (…)”

1.3  Experience shows that the implementation of safety-critical software is a complex task. There is evidence showing that new systems have been implemented before being fully ready and problems occurred when issues with the software were subsequently discovered.

1.4  This paper studies the way complex systems are developed and validated until they are ready for its use, analyzes some practical examples and proposes new Policy.

2.1 Different approaches to developing complex systems – Development life cycles

2.1.1 Development life cycles are characterized by the way they organize the interactions between technical development processes. Three basic types of life cycle models may be distinguished:

• Waterfall Model
• V-Model
• Spiral Model

2.1.2 A waterfall model defines a sequential development approach. The number of steps may differ, but in general the sequence contains the following processes: analysis, design, implementation, and test. In some cases it is preceded by a process such as feasibility analysis or requirements capture, and in some cases it is followed by a process such as maintenance or deployment.

2.1.3  There are several weaknesses in this approach. First, because the waterfall model only considers integration and test issues at the end of the life cycle, there is a risk that problems will be discovered too late in the development. Secondly, the design has to be finalized before specific implementation problems are discovered. Coping with these problems late in the development is likely to incur delays and increase cost (e.g. redesign, requirements change). Therefore, the waterfall model can only be used for projects with limited risk in terms of development and final product acceptance, e.g. routine development of conventional solutions.

2.1.4  A V-model defines a development approach that connects design processes with verification processes. At the start of the project, the stakeholder needs are identified to ensure that the final product is valid for its intended environment and that the product will meet the stakeholder expectations. Early in the project it is verified that requirements are testable, and test requirements are written to prepare for verification. The product is designed taking integration aspects into account and is integrated in accordance with its design.

2.1.5 The V-model can be viewed as a modified waterfall model, because it contains the same basic processes; adding only relationships. In this way it tries to discover problems early in the project when it is less costly to solve them. A weakness of this model is that it does not provide guidance on how to deal with unforeseen properties of the product.

2.2 Verification, validation and certification

2.2.1  In the development of a complex system, specifically software, it is important to be able to determine if it meets specifications and if its outputs are correct. This is the process of verification and validation (V & V) and its planning must start early in the development life cycle.

2.2.2  Software development has evolved from small tasks involving a few people to enormously large tasks involving many people. Because of this change, V & V has similarly also undergone a change. It became necessary to look at V & V as a separate activity in the overall software development life cycle. The V & V of today is significantly different from the past as it is practiced over the entire software life cycle. It is also highly formalized and sometimes activities are performed by organizations independent of the software developer.

2.2.3  According to the definitions by the IEEE Standard Glossary of Software Engineering Terminology, verification simply demonstrates whether the output of a phase conforms to its input as opposed to showing that the output is actually correct. Verification will not detect errors resulting from incorrect input specification and these errors may propagate without detection through later stages in the development cycle. It is not enough to only depend on verification; Validation is the process to check for problems with the specification and to demonstrate that the system operates correctly. Certification is defined as a written guarantee that a system or component complies with its specified requirements and is acceptable for operational use. The certification process also starts from the beginning of the life cycle and requires cooperation between the developer and regulatory agency.

2.2.4  An important aspect of certification is that it does not prove that the system is correct. Certification only proves that a system has met certain standards set by the certifying agency. The standards show that a product has met certain guidelines, but it does not mean that the system is correct. Any problem with the system is ultimately the responsibility of the designer and manufacturer, not the certification agency.

In the matrix, the likelihood of an event, defined in terms of probability of occurrence, is considered against its potential severity, which goes from an accident (Catastrophic) to a slight increase in controller workload.

• High Risk (Red): Tracking in a Hazard Tracking Risk Resolution System is required until the risk is reduced or accepted at the appropriate management level.
• Medium Risk (Yellow): Acceptable with review by the appropriate management level. Tracking in a Hazard Tracking Risk Resolution System is required.
• Low Risk (Green): Acceptable without review.
• A special case is considered when the Likelihood is Extremely Improbable and the Hazard Severity is Catastrophic: it is considered Medium Risk but unacceptable with single point and common cause failures.

2.3 Human Factor aspects

2.3.1 Experience gained from the design of large power plants, chemical processes, aircraft, and other large and complex systems has shown the importance of integrating the system inhabitants, the humans, from the beginning and throughout.

2.3.5  While it may appear to be costly to solve the human performance problems early in the development, it is actually cheaper in the long run as can be seen from Figure B. If a problem is left unresolved until the system has been brought into operation, it will not only pose a cost in terms of lower performance, but the cost of mending it is much higher. If the problems are left to the operation of the system, the cost of fixing them will increase throughout the lifetime of the system.

2.4 Integrating human factors in the system life cycle

2.4.1 Integrating human factors in the system design will result in benefits as shown in the figure:

2.4.2  A well-known problem connected with the introduction of a new system (or even changes to an existing system) is that people in the workplace may feel threatened, alienated or otherwise uncomfortable with the change. It is therefore crucial to manage the change and to make sure that its introduction is successful. A number of key activities are related to such implementation:

• Provide help to face up to change:
  It is important to help people to face up to the change and take them towards an acceptance of the need to change. Experience shows, however, that approximately 10% of the people in an organization will have serious doubts concerning the benefits of the change process and will resist it.
• Communicate as you have never communicated before:
  Communication becomes a key activity in the change process. It is important to inform, to the maximum extent possible, in order to avoid uncertainty among employees. Rumours can be detrimental to the change process. It is equally important to receive feedback during the change process to obtain an accurate feeling of how things are being received and to improve implementation.
• Gaining energetic commitment to the change:
  In the short-term, commitment can be gained from a focus on the survival of the organization. In the long-term the commitment has to come from system reward structures, employment policies and management practices.
• Early involvement:
  Early involvement of the people affected by the change will go a long way towards reducing behavioural resistance during implementation. It will considerably reduce the amount of energy that has to be expended on coercion and power strategies to enforce compliance.
• Turn perception of ‘threat’ into opportunity:
  A change process is likely to be seen as a threat by people. Experience shows that an organization, which can change this perception so that the change is seen as an opportunity, is more likely to succeed.
• Avoid over-organizing:
  The change process should be well thought-out before it takes place, but it should be recognized that the process is very organic in nature. One should not stick to the plan just because it is there, but rather learn from the situations as they develop, a change process is a learning process.
• Train for the change:
  People’s perception of the change may transform significantly if they have an opportunity to understand better what will be required of them. Training sessions can significantly reduce the resistance to the change.

2.4.3  To integrate human factors the development team should include people with the following profiles:

• system developers, typically software engineers;
• end-users, usually the operational controllers, supervisors and maintenance engineers;
• managers responsible for directing the development process;
• human factors specialists, who participate to ensure the integration of human factors.

2.4.4  It is important that a person responsible for human factors is nominated for the system development and participates in the planning and management throughout the life cycle of the system. If the human factors specialist does not have a background in ATM, it is important than he or she receives proper training and exposure to an operational environment.

2.4.5 David Hopkin describes the objective of the specialist as follows:

2.8 Examples of ATCOs Implementing already developed systems

2.8.1 VAFORIT

3.1  Development of complex systems, such as ATM systems, is a complicated task. The best way to address this is through an iterative process described as spiral model to integrate the view of every stakeholder.

3.2  Verification is the process to determine if a system or a part of a system delivers the expected outputs according to the inputs. Validation checks the specifications and the proper operation of the system. Certification is a written guarantee that a system or component complies with its specified requirements and has been deemed acceptable for operational use by the appropriate authority. A safety case is part of the certification process.

3.3  Human factors elements must be integrated during the development of the system. To achieve this, the development team should include system developers, operational controllers as end- users, managers and human factors specialists.

3.4  Controllers and developers think differently and speak different languages. Common ground shall be established in order to ensure that cooperation runs smoothly.

3.5  Early involvement of the end-users maximizes the acceptability of the system and improves its usability. There is a clear difference in the attitude of controllers being involved in the development of a system from the beginning of the project and the attitude of their colleagues having to deal with a nearly finished system. The first ones feel as responsible for the system as the code developers and they emphasize the good relationship between controllers and engineers when interviewed, while the second ones usually talk about the difficulties dealing with a system developed without the final user in mind.

It is recommended that;

4.1 IFATCA Policy is:

In the design, development and implementation of new ATM systems, operational controllers should cooperate with software developers, system engineers and human factors specialists. Their role should include:

• Establish user requirements.
• To participate in the risk assessment process.
• To validate the system.
• To provide feedback in the further development of the system.

and is included in the IFATCA Technical and Professional Manual.

4.2 IFATCA Policy on page 4.3.2.4 of the IFATCA Technical and Professional Manual:

Controllers should be involved in the specification, evaluation and implementation of an automated ATC system.

Is deleted.

European ATM Master Plan, page 141.

FAA’s NEXTGEN implementation plan, page 20.

NLR National Airspace Laboratory: Role of requirements in ATM operational concept validation 2007.

Verification/Validation/Certification 18-849b Dependable Embedded Systems Carnegie Mellon University.

ICAO Doc4444 / ATM501 The need for a safety assessment is in section 2.6.1.

ESARR 4: Risk assessment and mitigation in ATM. See section 5.

ESARR 6: Software in ATM systems. See section 2.

FAA: NAS Modernization System Safety Management Program.

EUROCONTROL: Human Factors in the Development of Air Traffic Management Systems, 1998.

EUROCONTROL: The Human Factors Case: Guidance for Human Factors Integration, 2007.

V. David Hopkin: Human Factors in Air Traffic Control; Taylor & Francis Ltd, 1995.

Press release from DFS: https://www.dfs.de/dfs/internet_2008/module/presse/englisch/press_service/press_information/2011/new_ats_system_p1_vaforit_at_uac_karlsruhe_31_january/index.html

FAA Report Number AV-2005-066; June 29, 2005.

MITRE: Independent Assessment of the ERAM Program. The report was not initially intended for public release but is currently released and can be downloaded from: https://assets.fiercemarkets.com/public/sites/govit/mitreindependentassessment_eram.pdf