Integrating laboratory and field study for improving selection: Development of a battery for predicting air traffic controller success.

Abstract

An example of combining laboratory-and field-based study to develop a selection battery for field implementation is described. The procedure provides advantages in comparison with sole use of field-based experimentation: namely, greater control over examinee samples, collection of construct validity data, and fewer field demands for cross-validation. Two experiments were conducted that converge on development of a test battery for selection of air traffic controllers (ATCs). The laboratory study (N = 112) used an ATC simulator (terminal radar approach control, or TRACON) for initial development and evaluation of the selection battery. The field study of 206 Federal Aviation Administration ATC trainees provided cross-validation data as a precursor to implementation of the battery. Implications for developing ability-based and self-efficacy-based selection measures for complex job performance are discussed, as are general issues for new selection research and application.

Cognitive Modeling and Task Analysis: Basic Processes and Individual Differences

“…The practice phase of TRACON proceeded in a fashion now generally well documented in our previous studies (e.g., Ackerman, 1992;Ackerman & Kanfer, 1993;Ackerman, Kanfer, & Goff, 1995). Mean performance and between-subjects standard deviations are shown for overall performance (number of planes handled in 30-minute simulations) in Figure 1.…”

“…Extensive data on the TRACON task and the embedded task components have been collected and reported in the literature (e.g., Ackerman, 1992;Ackerman & Cianciolo, under review;Ackerman & Kanfer, 1993;Ackerman, Kanfer, & Goff, 1995). Overflights have been found to be the easiest task component.…”

“…Previous research in our laboratory has demonstrated that perceptual speed abilities may provide a critically important supplement to measures of general and broad content abilities, in the prediction of individual differences in performance during skill acquisition (e.g., Ackerman, 1988;1990;1992;Ackerman & Kanfer, 1993;Ackerman, Kanfer & Goff, 1995). According to Ackerman (1988), Perceptual Speed (PS) is defined as: "In the language of skill acquisition, individual differences found on [perceptual speed] tests are directly attributable to the speed with which these productions can be implemented and compiled (e.g., see Werdelin & Stjernberg, 1969).…”

New Approaches to Assessment and Evaluation of Perceptual Speed Abilities

“…Our research (e.g., Ackerman & Kanfer, 1993a;Ackerman, Kanfer, & Goff, 1995) has documented that the Dial Reading Test is both highly predictive of individual differences in performance during complex skill acquisition, both in the laboratory and in job selectionlargely supportive of the research from the AAF and FAA programs. (It may be useful to note that Ackerman & Kanfer [1993b] developed a parallel version of the Dial Reading Test, and it remains one of the most effective tests in the battery we developed for the Minnesota Air Traffic Control Training Center [a regional facility sponsored by the FAA].)…”

“…Other research in this field has demonstrated that prediction of individual differences in performance during skill acquisition and at asymptotic levels of performance can be significantly and substantially improved (e.g., Ackerman & Kanfer, 1993a, 1993bAckerman & Cianciolo, 1999. By building on the foundation of general and broad content cognitive abilities (e.g., verbal, spatial, numerical), properly selected measures of perceptual speed and psychomotor ability measures may be used to enhance selection procedures and to identify individuals who are more likely to succeed or fail during training or skill acquisition practice.…”