At the 38th Conference in Santiago, Member Association Ghana requested SC4 to research this subject and develop policy.

Auditing is commonly associated with the financial world however the Oxford English Dictionary defines it as a “searching examination.” It will be quite evident to anyone in our profession that having an independent oversight or “searching examination” of a complicated safety system such as ATC, provided it is properly established, can have great benefits for the management, employees and customers.

ICAO Doc 9734-AN/959 Safety Oversight Manual describes the establishment and management of a State’s safety oversight system. Paragraph 2.1.1 states:

The document goes on to describe how a State should enact basic aviation law, create a balanced oversight system that includes the State, Industry, and recognises the public interest. The State is also required to undertake continued surveillance.

It follows from this that some sort of surveillance organisation needs to exist and in most countries this is the “Civil Aviation Authority” (CAA). The degree of independence this sort of organisation enjoys varies widely, as does it’s integration with what is often another government agency, the air traffic service provider. In the past they have often been one and the same organisation. Given that fact and the small size and uncomplicated organisation inherent in many CAAs the “surveillance organisation” is often the management, supervisors and checking staff, probably overseen by a director-general whose expertise could be from any of the aviation professions.

Auditing has been introduced as the management of the CAAs has become less bureaucratic and better trained in management techniques such as Total Quality Management and the essentials of corporate governance. Many governments have also separated the service provider from the regulating agency and in many cases commercialised the service provider. The service provider then becomes a similar entity to an airline in that it must operate under a licence from the State. It must then be subject to surveillance by the regulator. One of the best ways to undertake surveillance is by way of an audit. It then follows that if an organisation is to be subject to external audits it should run similar internal audits. This should ensure that no surprises are uncovered by the external auditor. It also serves a management purpose in that an organisation reporting to the highest level can independently review and evaluate the effectiveness of ATS operations ensuring that they comply with the established rules and procedures.

Documented audit procedures need to be developed so the auditors and auditees are equally well aware of their responsibilities and the methodology that is to be used.

For instance the objectives of the audit process could be:

1. To determine the compliance of the system elements with specified requirements;
2. To determine the effectiveness of the existing system in meeting specified safety and quality objectives;
3. To provide the auditee with opportunity to improve the safety system by providing feedback on the system’s performance;
4. To highlight commendable findings when appropriate;
5. To meet safety regulatory requirements and legal obligations.

Directions to auditors during an audit could state:

1. Be professional in their approach and performance;
2. Be objective and impartial;
3. Be attentive to the concerns of the auditee’s staff and management;

4. Focus on facts not opinions;
5. Focus on improving the system and not on criticising individuals;
6. Be frank, open and concentrate on factual discussions;
7. Respect the role of ATS staff and management as part of the ATS team.

Areas covered during an audit might be:

1. Personnel;
2. Organisational structure;
3. Administrative procedures;
4. Operational procedures;
5. Material resources;
6. Work area;
7. Conformance with standards and specifications;
8. Documentation management;
9. Report and record keeping;
10. System operation and performance.

Some typical questions that could be addressed through audit are:

1. Is the organisation’s overall safety and quality system capable of achieving the required standards of performance;
2. Is the organisation (or the part being audited) fulfilling all of its obligations under the relevant aviation law and associated regulations and documentation;
3. What are the strengths and weaknesses of the system;
4. Is the organisation (or the part being audited) actually carrying out and achieving what it claims to do?

Audit principles might be:

1. Pro-activity in that it searches for system faults before they result in an accident;
2. Seeking to be positive rather than simply being occupied with fault finding;
3. Focussed on the system rather than the individual;
4. Identifying non compliance with standards;
5. Analysing processes and situations which could lead to non-conformance;
6. Highlighting good practise;
7. Professional, impartial and objective conduct;
8. Indicating strengths and weaknesses in key areas;
9. Providing clear guidelines for improvements.

During an audit auditors should:

1. Remain within the audit scope;
2. Exercise objectivity;
3. Collect and analyse evidence that is relevant and sufficient to permit the drawing of conclusions regarding the audited safety system;

4. Remain alert to any indications of evidence that can influence the audit results and possibly require more extensive auditing;
5. Be able to answer such questions as:

a) Are the procedures, documents and other information describing or supporting the required elements of the safety system known, available, understood and used by the auditee’s personnel;

b) Are all the documents and other information used to describe the system adequate to achieve the required safety objectives?

6. Act in an ethical manner at all times.

A system needs to be put in place to enable an audit team to formally notify the auditee of any deficiencies found. These deficiencies might be in the form of a corrective action request and should be classified as major or minor. The auditee should be required to formally answer the request within a given period of time. These requests should be in addition to the formal reports required at the close of an audit. The purpose of the formal report should be to inform the auditee, senior management and governing bodies such as a Board or Ministry about the safety health of the location subject to the audit and of the air traffic system in general.

The importance of selecting the best-qualified personnel and then ensuring that they can work in an independent role cannot be overstated. Independence permits auditors to render the impartial and unbiased judgements essential to the proper conduct of audits. The independence of the auditors must be supported at the highest level so that no employee is left in any doubt as to the role of the people involved. Preferably there should be a published Safety Management Policy, publicly setting out the aims of the organisation, coupled with specific manuals covering the role of the safety management organisation and it’s auditing role.

ICAO sums up the qualifications and experience required of inspectors thus:

“In addition to the vital importance of technical competency in performing certification, inspection and surveillance functions, it is critical that inspectors possess a high degree of integrity, are impartial in carrying out their tasks, are tactful, have a good understanding of human nature and possess good communication skills”

This description could equally be applied to ATS auditors. Because there is a need to intimately understand the ATS system being audited, candidates should be carefully selected from the ranks of experienced controllers. Emphasis should be placed on the skills needed to accompany that system experience. Adequate training should also be provided to fill in the knowledge gaps, specifically teaching auditing and reporting techniques and providing a clear understanding of the aims and policies of both ICAO and the employer. Audit teams may include auditors with other competencies such as technical skills or administration.

The ICAO requirement for “continued surveillance” creates a need for a mechanism to audit the performance of the ATS system.

This is best performed by ATS professionals selected for their additional skills and integrity.

Audit teams should enjoy support from the highest levels of their organisation and should report directly to the highest level.

Audits should be carried out in accordance with objective, impartial and professional standards.

Audits should concentrate on the system not the individual.

Audit teams should provide both requests for corrective action and full and complete reports.

Auditees should be given every opportunity to respond to reports.

This paper be accepted as Guidance Material.

ICAO Doc 9734-AN/959. Safety Oversight Manual – Part A The Establishment and Management of a State’s Safety Oversight System.

Airservices Australia Safety & Quality Management System Vol.4 Audit Manual.