Attention Control Measures Improve the Prediction of Performance in Navy Trainees

Abstract

We present a theoretical framework and empirical data from a sample of 490 U.S. Navy trainees (air traffic controllers, student naval aviators, and student naval flight officers) supporting the use of attention control tests in personnel selection. Current military selection tests leave room for improvement when predicting individual differences in work-relevant outcomes and reducing adverse impact. We tested whether measures of attention control, working memory capacity, and fluid intelligence improved the prediction of training success above and beyond the current composite scores used by the U.S. Military. For air traffic controllers in training, attention control explained 9.1% of the unique variance in academic performance, whereas the Armed Forces Qualification Test explained 5.2% of the unique variance. For student naval aviators, incremental validity estimates were generally small and non-significant. For student naval flight officers, attention control measures explained 11.8% of the unique variance in aviation preflight indoctrination training performance and 4.3% of the unique variance in flight performance. In general, attention control measures improved training outcome classification accuracy, and the antisaccade and selective visual arrays tests of attention control demonstrated relatively small differences between majority and minority subgroups.

Modern Assessments of Intelligence Must Be Fair and Equitable

“…Therefore, the hierarchical structure as often suggested in the commonly used CHC model is not necessary when just as much of the variance can be explained without it ( McGrew et al 2023 ). Thus, as a non- g model that is well supported by basic scientific evidence ( Burgoyne et al 2021 , 2022 ; Shipstead et al 2016 ; Mashburn et al 2021 ; see also Martin et al 2020 ; Peng and Swanson 2022 ), process overlap theory is a more parsimonious approach to thinking about assessment. Further, considering the utility of examining subprocesses of g , as outlined above, non- g POT is also better for working toward equity and fairness goals.…”

“…Moreover, additional work by Bosco et al ( 2015 ) shows that focusing on the domain-general executive processes of attentional control and working memory was better for reducing adverse impact between racial/ethnic groups compared to using more traditional assessments that emphasize psychometric g . Other work by Burgoyne et al ( 2022 ) provides more support for domain-general subprocesses for reducing the adverse impact and improving the variance accounted for in academic performance above and beyond traditional cognitive assessments in a sample of Navy trainees. This work also found that attentional control measures improved training classification accuracy in trainees and reduced subgroup differences between minority and majority members.…”

Modern Assessments of Intelligence Must Be Fair and Equitable

“…Specifically, they show that tests that focus on attentional control and domain general executive processes involved in maintaining focus on task relevant information (e.g., working memory tasks) provide a more equitable approach to cognitive assessments. Moreover, additional work by Bosco et al (2015) shows that focusing on domain general executive processes of attentional control and working memory was better for reducing adverse impact between racial/ethnic groups compared to using more traditional assessments that emphasize psychometric g. Other work by Burgoyne et al (2022) provides more support for domain general subprocesses for reducing adverse impact and improving the variance accounted for in academic performance above and beyond traditional cognitive assessments in a sample of Navy trainees. This work also found that attentional control measures improved training classification accuracy in trainees and reduced subgroup differences between minority and majority members.…”

“…Therefore, the hierarchical structure as often suggested in the commonly used CHC model is not necessary when just as much of the variance can be explained without it (McGrew et al, 2023). Thus, as a non-g model that is well supported by basic scientific evidence (Burgoyne et al, 2021(Burgoyne et al, , 2022Shipstead et al, 2016;Mashburn et al, 2021; also see Martin et al, 2020;Peng & Swanson, 2022), Process Overlap Theory is a more parsimonious approach to thinking about assessment. Further, considering the utility of examining subprocesses of g, as outlined above, non-g POT is also better for working toward equity and fairness goals.…”

Toward inclusive and identity safe learning for supporting racialized student achievement

“…Based on what Black students tend to experience in their learning environments and the degree of mental demands often placed on them, these findings are especially relevant for supporting their learning and achievement. Moreover, even though a great deal of the research on topics of retrieval practice and working memory capacity has been conducted by cognitive scientists with mostly White and less diverse samples, recent research has shown that these constructs are meaningful and have relevance in supporting and predicting performance among more racially and culturally diverse samples, as well as in mostly Black samples (see Burgoyne et al, 2022;Cockcroft, 2022;Holden et al, 2020; also see Bosco et al, 2015;Burgoyne et al, 2021). Furthermore, both retrieval practice and working memory have been shown to be factors that can help boost student learning and performance across a variety of contexts (see Gathercole & Alloway, 2007;Roediger & Butler, 2011).…”