21ST ANNUAL CONFERENCE, Amsterdam, Netherlands, 3-7 May 1982WP No. 25Aptitude Testing for Air Traffic Controllers; Development of Aptitude Tests for ATCOs |
|
Editor’s Note: The full text of this Working Paper is not available. Only the following parts were retrieved and should be used as an indication of the evaluation and consideration of available information that lead to IFATCA Policy being developed. |
Summary
Aptitude Testing for Air Traffic Controllers
Standing Committee V is indebted to Dr. Niels Busch-Jensen, a Danish psychologist, who has prepared the following comments and observations on the work of the psychologist in ATC selection and the types of tests which are currently used in various administrations including his own. The test programme now in use was brought into existence in 1969 and has only been slightly changed since then. As a starting point, the psychologist made a study on the psychological demands of the ATC profession. Available research material (rather sparse at the time) was studied, and the psychologists were given an introduction to ATC work, including some days of guided observations of air traffic controllers at work, followed by in-depth interviews with a number of active controllers. These activities provided some rough guidelines for selecting the tests to be included in the battery as well as for estimating the requisite standards.
The battery was made up mainly of a number of tests usually employed in pilot selection. Of course none of the psycho—motor tests specific for flying skills were included, so all of the tests in the ATC battery are paper and pencil tests (concerning the need for dynamic tests, see later).
The tests are:
1) Raven’s Advanced Progressive Matrices Set II
This test is claimed to be a non-verbal test of general intelligence. It is rather difficult and therefore appropriate for ATC applicants who are mainly well above average in this respect.
2) Inductive/Verbal Intelligence Test
This is also a very hard test. It is included in the test battery for two reasons. The first is to cover general mental capacity with more than one test. The second is the fact that the test yields two scores: one for inductive intelligence (logical analysis) and one for verbal intelligence. From the outset we had reason to believe that general intelligence is an essential factor in acquiring the knowledge and skills of an air traffic controller. We do still believe that, but with the modification that even if trainees significantly below average intelligence won’t stand much of a chance in ATC training, the difference between a high and a very high score on conventional intelligence tests will not necessarily correspond to differential probabilities for success in training. In the upper part of the scale, other factors are probably of greater importance.
Of course the intelligence test scores are taken into consideration when a batch of applicants are rank ordered with respect to their suitability for ATC training, but their most obvious function is to screen out the few applicants with unquestionably sub-standard intelligence. It must be added that knowing the general intelligence level of the applicant as helpful when his (or her) former educational achievements are evaluated. If these achievements are out of line with what would ordinarily correspond to the mental powers as shown in the test results, this is often an indication of the degree to which the applicant in question tends to utilise his personal resources – a personality facet that is further explored through the psychological interview.
3) Simultaneous Capacity Test
This test is originally intended to measure the testee’s ability to work on more than one task simultaneously, an ability most vital in the work of an air traffic controller. It can be questioned how well the test fulfils that intention, but it has anyhow been shown to have a certain amount of predictive validity in relation to pilot training. The test is scored on work speed and work quality, and note is taken of the balance between these two scores, because this will often give an important indication of the applicant’s preferred working mode. A very fast but sloppy work mode is judged to be incompatible with ATC work, and so is a very slow even if a very cautious working mode.
4) Technical-mechanical Comprehension Test
ATC is a profession that utilises many different kinds of technical aids, and the trainee should preferably not have too many problems in understanding the functions of the tools he uses. The test has a clear gender bias, Female applicants score consistently low and their scores have therefore always been judged comparatively mildly. We have no indication that the low scores on this test has meant a lower success rate for female trainees, but the forthcoming statistical analysis will probably clarify the matter.
5) Air Traffic Problem Test
This test was constructed because FAA experimentation had pointed to the usefulness of tests requiring the candidate to work on simplified ATC problems. Apart from testing what grasp of speed, time and direction the candidates have, an important aspect of this test is that the main thing is not specific arithmetic skills, but rather the ability to understand the nature of the problems as a prerequisite for an adequate application of simple arithmetic skills. To get a more detailed insight into the way the applicant approaches the problems, it has often proved useful as a part of the interview situation to let him work – and “think aloud” -on some of the problems he didn’t solve in the first place.
6) Arithmetics and Mathematics Test
A minimum level of knowledge and skills in this field is judged to be necessary especially in parts of ATC training.
7) English Vocabulary Test
8) English Grammar Test
English being the language of the air, the reason for including these tests in the battery is fairly obvious. The following written tasks are not quantitatively scored, but only used as working material for the psychological interview.
9) Autobiography
10) Biodata sheet
11) Personal Inventory
It may be said that the test and selection procedure for air traffic controllers is rather comprehensive and that consequently it is peculiar in that, relatively speaking, many of the finally selected candidates subsequently fail in the training programme. A number of tentative answers to this Question can be given.
It is possible that a careful statistical analysis of the data will help us to make a more precise and adequate weighting of the test scores than we have hitherto done, thereby perhaps improving the predictive effectiveness of the procedure. But it would certainly be an unrealistically high ambition to think it possible to prevent any training failures at all from occurring. We must envisage the possibility that the ATC training programme in its contemporary design is in fact very demanding. This option can hardly be objectively assessed. Finally, the test battery as here described is by and large pretty representative of the “state of the art” at the time it was established, and it must therefore be subject to those limitations of conventional psychological tests that will be mentioned in the following together with some proposals for possible future improvements.
Up till now, very little systematic statistical follow-up and development work has been done. The reasons, or excuses, must partly be sought in the fact that the psychologists have been employed as consultants on an ad- hoc basis within relatively low budget frames. Another reason is that in some cases the number of trainees is only a dozen or so per year on the average, so that it has to take a long time to build up a sufficiently large sample to make reliable statistical analyses. Such an analysis is now on its way, but unfortunately, due to the above- mentioned fact, the sample to be included in the analysis will span over a whole decade. Within that decade, substantial changes have taken place In ATC training and working conditions, so that the criteria against which the test data must be validated may not have been stable. In the years following the implementation of the psychological test procedure, some kind of qualitative follow-up work was however done.
Three psychologists involved in the selection procedure arranged it so that they had a re- interview with each of the trainees who failed in the training and that with each class of trainees they performed two group interviews, the first of these taking place halfway through the training programme, the other at the end. By these means, the psychologists gained a deeper, but not quantitatively expressed insight into the dynamics of the training programme.
Further knowledge of the general characteristics of the training environment was incidentally obtained by two of the psychologists through their discussions with a large number of air traffic controllers – discussions that took place within a series of courses run by the two psychologists in question. (These courses were concerned with the fundamental psychological and instructional factors in on-the-job-training) One very general statement has repeatedly been warranted by the psychologist’s experience: a success or failure is primarily the success or failure of a particular trainee (and hence, of the selection system), but it is at the same time the success or failure of the training system. The training outcome is the result of a chain of interactions between the trainee and the training, and it is therefore always very difficult to ascertain with precision if a case of failure is due to the trainee’s shortcomings or if equally significant causes should be sought in inadequate training conditions.
A very sensible policy would of course make effort to increase tie quality of both the selection and the training, still bearing in mind that not all training problems can be solved through an increased selection effort, and that not all selection deficiencies can be alleviated by improved training methods. This general comment made, the following problem areas do seem to deserve special mentioning:
- Dynamic aptitude testing.
- The nature of time-sharing abilities.
- The limitations of intelligence tests.
- Training entry motivation, job satisfaction expectancies, socialisation and the development of professional identity.
1. Dynamic aptitude testing
The majority of contemporary aptitude tests are paper and pencil tests. One of the few exceptions can be found in the pilot aptitude test program in a number of countries, where psychomotor tests for pilot applicants were developed during WW2, but this part of the U.S. pilot test programme was for a number of reasons abolished in the fifties. Recently, there has been a renewed interest in these kind of tests, due to the possibilities embodied in modern electronic technology. This recent trend is also found in development work within the ATC area (see Note 1).
The basic difference between paper and pencil tests and psychomotor tests is that the former kind is fundamentally static in nature, whereas the latter can be made dynamic, In an ordinary paper and pencil test, the problem is still while the testee works on it. Even though he may be under time pressure, he is working in a situation where all the relevant information is available simultaneously, so that he is more or less free to determine for himself the order in which he will deal with the different parts of the information. No external, events do, within the given time limits of the test, demand that particular decisions have to be made at particular moments. Obviously, the demand to adjust your own activities to a series of external events is a very essential feature of ATC work and that same demand should therefore be put by at least some of the tests used for ATC selection purposes. Different attempts have been made to measure the ability for dynamic handling of information (apart from the psychomotor tests for pilots). The attempts seem not to have been very successful up till now, partly because a number of the experimental tests constructed have turned out to measure nothing that could not nearly as well be measured by conventional static tests. One attempt that has deliberately sought to avoid this pitfall is the “Information Scheduling Test” (Ridgeway and Fuller) which has not yet been validated (validation studies will begin shortly), but the basic ideas of the test seem sound.
2. The nature of time-sharing abilities
A very often stated aim of the development of dynamic tests for ATC and pilot selection is to make it possible to measure the ability, or the group of abilities, denoted by terms such as “time- sharing”, “simultaneous capacity” or “division of attention”. With even a superficial inspection it is abundantly apparent that a key facet of ATC as well as pilot work is the demand to perform multiple tasks simultaneously or in rapid succession, together with the need to process and integrate information from a number of different sources. Much of the contemporary research on time-sharing abilities has focused on the very basic information-processing capacities of the human brain. A major point of disagreement among researchers is the question whether any kind of general time-sharing ability really exists or if an individual’s ability to perform two different tasks at once is mainly determined by his ability to perform each of them alone. The research in this field is open to some criticism with respect to its relevance to real life problems like selecting and training for the ATC and the pilot professions. (See Note 5).
It is a paradoxical but very simple fact that an air traffic controller exercising his complex skill handles masses of information at a rate that grossly exceeds the “basic information processing capacity” of the human brain. The paradox is resolved by the observation that the very nature of learning this complex skill is to organise these myriads of information. What is learned is to pattern the single bits of information and the many subskills in larger wholes, so that information processing within these larger wholes becomes “automated” and hence put lesser demands on the central information processing capacity, thereby leaving attention free to turn elsewhere. It might very well be for that reason a good idea not only to ascertain whether a trainee is able to learn the basic subskills in ATC work, but also to study how he is able to learn them, i.e. how well and how fast he can organise his skill knowledge into larger wholes, so that we can use his redundant information processing capacity to the purpose of integrating subskills into coherent complex skills.
There is hardly any doubt that this kind of question points to a lacuna in the test repertoire of contemporary occupational psychology. However, some work has been done in quite a different context that might eventually prove useful in this respect. The test methodology described there will in the near future be submitted for evaluation as a selection device.
3. The limitation of intelligence tests
Among the most proven techniques of psychological testing are the different kinds of intelligence tests. The test battery used for selection of certain air traffic controllers bears this out. It must however be noticed that the original development ground for intelligence testing was the field scholastic achievement prediction. Within this field, intelligence tests have proven very useful, but outside this field they have rightly been criticised as weak and doubtful. (See Note 2).
In ATC training, certain amounts of scholastic achievement are needed, but it is, generally, not in this domain that most of the training failures occur. Especially ‘when a rather high educational level is required to enter ATC training, intelligence testing is mostly redundant, or will not serve much purpose beyond that of screening out those few candidates whose mental capabilities in this respect are unequivocally sub-standard. The real proving ground for the abilities of the ATC trainee is the learning of practical skills, and the wisdom in relying very much on conventional intelligence testing in ATC selection must therefore be questioned. It must be noted, however, that the validity of the considerations above depends on the characteristics of the applicant population in question. If you have to deal with a broader range of intelligence from applicants for air traffic control, then intelligence testing is still indispensable.
4. Training entry motivation, job satisfaction expectancies, socialisation and the development of professional identity
It is a “sine qua non” that the ATC trainee is able ‘to assimilate a prescribed amount of knowledge and to develop the required skills within the time limits of the training programme. But the trainee is no robot-like learning machine or a mere conglomeration of skills. He is also a human being with feelings, attitudes, motivations, hopes and desires that are omni-important for his attraction to the profession of ATC, and all of which he brings with him into the training situation. The training process is also a social learning process. During that process, the trainee learns to adjust to ATC as his future working place, how to get along with his fellow trainees, instructors, future colleagues, etc. If this social aspect of the training fails, if he does not gradually acquire norms, attitudes and standards that are consonant with working within ATC he will not stay very long in training. He will become a maladjusted and marginally effective air traffic controller and a nuisance to the people who will have to work next to him.
It is important to realise that the trainee is not only developing ATC skills and competencies, but that he is also in the process of becoming an air traffic controller. He is, in other words, developing his sense of that professional identity that may, for many years, (maybe for the rest of his life) be a most vital part of his experience of personal identity. It is readily observable how quickly trainees in the initial stages of training become eager to be looked on as air traffic controllers-to-be, how quickly they imitate any kind of professional slang, identifying with their future colleagues and developing an “ATC-view” on the world. The identification process is, psychologically speaking, a cornerstone in the development of a professional air traffic controller and any ATC programme must therefore allow sufficient opportunities for this identification process to operate.
When recruiting, the theme of personal identity is contained in the applicant’s ideas of the kind of person he would like to become. By applying, he has, among other things, indicated that he would like to become the kind of person that he thinks an air traffic controller is and to have the kind of satisfactions that he thinks ATC work will give him.
This fundamental aspect of motivation has to apply to a number of facets. How realistic is his perception of the ATC work, working environment and employment conditions? Is it well formed or is it superficial ? Is his wish to become an air traffic controller kind of person based on a realistic self-concept, or is it merely immature, compensatory wishful thinking ?
These are questions of the kind that we, among other things, try to cover through the psychological interviews as well as we can, but it is explicitly mentioned here because it seems to be an area that may profit from a more systematic and quantitative mapping, e.g. by the use of job preview questionnaires or other kinds of job motivation expectancy assessment.
References
Chiles, W .D et al. Time-sharing in complex performance: an expanded replication. Report No. FAA-AN.78-33 Oklahoma City. FAA Civil Aeromedical Institute, 1978.
McClelland, D.C. Testing for Competence rather than for Intelligence. American Psychologist Magazine, January 1973, pages 1-14.
Hannan, R.L. Measures of Expected Work Motivation as an Alternative to Written Ability Tests for Federal Personnel Selection. Report No.PRR-80-24. Washington, D.C. US Office of Personnel Management, 1980.
Isaac, D.J. and O’Connor, B.M. Separation of Two Adult Populations Identified with Two Levels of Psychological Development (1977).
Busch-Jensen, N. Time-Sharing, Automacity and Strategies. Report to the 16th International Symposium of Applied Military Psychology. Amsterdam, May 1980.
R.G. Green & R.M. Taylor. The Psychologist in Aircraft Accident Investigation AGARD CP-254 (Nov 1978).
J.O .Boone. Air Traffic Control Automations – Its Impact and Use in the Selection and Screening of Air Traffic Controllers. AGARD CP-273 (Oct 1979).
C.F. Van Parreran. The Training of Air Traffic Controllers: An Analysis from the Standpoint of Psychology of Learning. Eurocontrol (Automation & Training Seminar, Oct 1974).
H.J. Von Villiez. Selection, Training and Motivation of Staff for Interdisciplinary Functions in an Automated ATC System. Eurocontrol (Automation & Training Seminar, Oct 1974).
J. Ridgeway & S. Fuller. Measuring the Ability to Handle Information Overload.
A. Hugenmatter & L. Wehli. Complex Selection Methods in the Recruitment of Air Traffic Controllers. 14th WEEAP Conference, October 1981.
Eurocontrol Institute. Pedagogical Analysis on the Possible Use of a “Dialogue” System Based upon an ATC Simulator (Conversational ATC Simulator).
C.J. Adcock. Fundamentals of Psychology. Penguin Books 14020664 7.
M.F. Allnut. The Psychologist’s Role in Aircraft Accident Investigation. AGARD.
A.J. Feggetter. Investigative Methodology of Human Factor in Aircraft Accidents. 14th WEAAP Conference, October 1981.
N. Busch-Jensen. Time-Sharing, Automacity and Strategies.
Isaac, D.J. and O’Connor, B.M. Separation of Two Adult Populations Identified with Two Levels of Psychological Development Human Relations Volume 30 No. 10 (1977).
J.D. Thomas. The Selection and Training of Air Traffic controllers, with Particular reference to Zimbabwe (Rhodesia). Feb 1975.
J.H. Henry et al. Training of US Air Traffic Controllers. FAA, January 1975.
Last Update: September 20, 2020