A Meta-Analytic Evaluation of Naglieri Nonverbal Ability Test: Exploring Its Validity Evidence and Effectiveness in Equitably Identifying Gifted Students

Abstract

The Naglieri Nonverbal Ability Test (NNAT) was developed to more equitably identify students of color, as it advertises itself as a culture-fair measure. In this meta-analytic evaluation, we aimed to investigate (a) the generalizability of validity evidence of NNAT by checking its construct and criterion validity with other measures (Part I) and (b) whether NNAT truly meets its goal to identify more culturally diverse students (Part II). After reviewing 1,714 studies, a total of 29 studies met our criteria (59 effect sizes from 22 studies for Part I and 7 effect sizes from 7 studies for Part II). In Part I, we investigated empirical evidence of validity of NNAT in relationship with different types of measures (overall effect size of r was .44); The results revealed that the correlation between NNAT and the achievement test results was 0.68, followed by the intelligence measures similar to NNAT (e.g., Cognitive Abilities Test, Otis–Lennon School Ability Tests; r = .31) and other alternative measures often used to identify gifted students (e.g., teacher-rating scale; r = .20). The moderator analysis results showed high correlations between NNAT and other measures when Naglieri is an author of the study. In Part II, although NNAT identified more students of color compared with other nonverbal tests (overall effect size of risk ratio was 0.42), findings revealed that students of color remain underrepresented in gifted programs and services.

National Non-verbal Cognitive Ability Test (BNV) Development Study

“…It was reported in the present study that these tests applied for criterion-related validity have yielded valid results in contemporary studies as well. As an example, Lee et al (2021) carried out a meta-analysis study examining 29 NNAT related studies as a result of which it was identified that NNAT displays high correlations between NNAT and other measurements. A correlation was identified in the study by Marchesini (2020) between NNAT and The Cognitive Abilities Test which measures reasoning in three areas (Verbal, Quantitative, and Nonverbal).…”

“…It is expected to contribute to the identification of nonverbal cognitive ability, rapid clinical assessment as well as individual/group test applications that require student selection. Even though it is emphasized that nonverbal tests are more successful in socio-cultural factors (Hodges et al, 2018), it should be taken into consideration that the results should be revised continuously in order to preserve the validity of these tests (Lee et al, 2021). Regardless of the selected instrument, the use of nonverbal ability test by itself may not be sufficient to fill the gap in identification (Carman et al, 2020).…”

Challenges and Efforts of Achieving Equity Beyond Gifted Education: Implications From Mathematics Education

“…As described in Peters' (2021) article, scholars proposed diverse methods to solve this issue, such as deleting a verbal subtest from an intelligence measure (Naglieri & Ford, 2003) and investigating the measurement invariance across the groups when developing and checking validity evidence of the HOPE Scale (Peters & Gentry, 2010). However, nonverbal ability tests do not seem to be more effective at identifying underrepresented gifted students (Lee et al, 2021). Similarly, although HOPE Scale scores were invariant across different subgroups indicating no assessment item bias to specific groups, actual score differences were still found among students from diverse backgrounds (Peters & Gentry, 2010).…”

“…Similarly, in terms of the intelligence tests, the subtest of mathematical reasoning has never been questioned. In contrast, the concept of eliminating a verbal subtest to identify underserved populations equitably has been controversial among scholars (Lee et al, 2021). As mathematics curriculum is sequential with its hierarchical nature of concepts and learning trajectories, mathematics has been widely centered in gifted programs to develop students' higher order thinking skills.…”

Integrating Diverse Points of View on Intelligence: A 6P Framework and Its Implications

“…Attempts at cultural attenuation have been made with the Army Beta —a test used during World War I to screen applicants who needed a nonverbal test ( Yerkes 1921 )—the Raven Progressive Matrices ( Raven et al 1992 ), and the so-called Cattell Culture-Fair Test of g ( Cattell and Cattell 1973 ). However, studies have shown that none of these tests has been particularly culture attenuated and certainly not culture free (e.g., Lee et al 2020 ; Lewis et al 2007 ; Lohman and Gambrell 2012 ). This would make sense in the 6P framework, again, because environmental press differs from one place to another, so that the problems that best measure intelligence also may differ.…”