18TH ANNUAL CONFERENCE, Brussels, Belgium, 23-27 April 1979WP No. 22Examination and Validation Procedures |
Introduction
SC5 distributed a questionnaire to MAs to determine if current examination and validation procedures were satisfactory and if necessary to propose any changes. Only ten replies were received: – Greece, Belgium, Costa Rica, New Zealand, East Africa, South Africa, Switzerland, Denmark, EGATS and Netherlands Antilles.
School training finishes with written, oral and practical examinations. During OJT the student is assessed and then either takes a practical examination (check out) or is considered validated by the supervisor or senior controllers. Most MAs were satisfied with current procedures but still felt that there was need for improvement particularly where assessments could count toward the final examinations.
Of those MAs who were not satisfied with their procedures it was noticed that nor reasons for student failure were sought. The lack of such feedback hinders any real improvement in the training system.
Discussion
Students, somewhat naturally, tend to favour a system in which assessments are made and no formal examinations are taken. Administrations, on the other hand, desire formal examinations. A pass mark is stipulated and the student either passes or fails. ATC is a very responsible profession demanding extremely high standards of knowledge, understanding and skill. These have to be effectively assessed, objectively not subjectively. No perfect system of examination has yet been devised. There are arguments for and against a system of either examination or assessment. That students have to reach an acceptable standard (generally 70% in all subjects, sometimes 80% practical) is indisputable. But what is the most effective way to do this?
A system combining assessment with examinations
During school training regular progress tests are given on all theoretical subjects. Results are analysed and discussed with the students. Final written examinations should be set (70% pass mark). The students should be regularly assessed and debriefed throughout the period of simulation training. A written report should be made by the instructor at least twice during a 6-8 week course. Final assessments (made if possible by at least three instructors) are made about one week from the end of the course and reflect the level attained by the student plus his overall course performance. A final practical examination is held with the assessment contributing 50% of the total marks. Pass mark 70%.
Oral examinations are able to assess that knowledge and understanding which cannot be determined from the written examination. They also examine the ability of the student to express himself in a clear and unambiguous manner. Pass mark 70%.
During OJT a written examination on local procedures, local area, letters of agreement etc. (At least 70%) Regular assessments by OJT coaches. Reports on student progress to training section. At all times the student is kept informed and permitted to see his report. When the training section is satisfied that a student has attained the required standard for validation the student sits an oral examination. His practical ability has been determined by assessments and training reports and the oral examination can discover the depth of his understanding and appreciation of local procedures.
Looking at the reasons for failure
Whatever the examination procedures selected there are always some failures. There must never be any compromise on the high standard required to be a controller. By looking at the reasons for failure we are able to improve on selection and training thereby reducing the costs. An item that is close to the hearts of all administrators.
At each stage of training where failure has occurred it is necessary to look not only at that particular phase but at all previous phases. It must be determined whether the failure is an isolated case (the student – selection?) or a recurring problem (the training system?). Actual reasons for failure are many varied but time spent on analysis and then on improvement of training is not time wasted. High standards in ATC can only be maintained by a competent system of training which is itself under constant review striving to drive out imperfections.
Should a student be allowed to re-sit an examination? Provided the student has shown some signs of success and it can be determined that he has controller potential then after a suitable period of further training a re-examination should be permitted. Progress tests and assessments are useful pointers to potential. Many experienced controllers develop a “feel” about students of their ability to do the job and although a very subjective method it can be a useful guide. No student should be given unlimited training time but neither do all students reach the required standard in the same training time. Some of those students who appear to take longer to qualify than their colleagues become much better controllers in the long run.
Conclusions
Not everyone is satisfied with the examination procedures used by their administrations for the following reasons:
- school training is considered ineffective;
- insufficient personnel for supervision of OJT;
- prefer final examination after OJT so that assessment counts for 75% of total marks;
- unit procedures are inadequate and school courses need revising;
- subjective validations (by supervisors) are inadequate.
Written final examinations cannot be replaced by a series of progress tests. It is possible to replace the final practical examination (validation) by a series of assessments but the practical examination at the end of school training should be retained. Not all administrations seek to discover the reasons for failure during training. Average pass mark is 70%. In some cases 80% for validation.
Recommendations
It is recommended that:
Those MAs not satisfied with current examination procedures consider the proposals put forward above with a view to presenting them to their administrations for implementation.
Training organisations be encouraged to research the reasons for failure in training.
Standing Committee 5 request Conference to direct MAs to accept this working paper and its recommendations as guidance material.
Reference Documentation
SC5 Questionnaire 1978.
Selection and Training of Air Traffic Controllers (Rhodesia) G.T. Thomas.
Training syllabuses: New Zealand, Sweden, Switzerland, ICAO, Eurocontrol and FAA. Information collected by Danish ATCA.
Last Update: September 19, 2020