Combining Scores in Multiple-Criteria Assessment Systems

Abstract

The Fall 2009 issue of Gifted Child Quarterly focused on dispelling a wide range of myths related to the identification of gifted students (Borland, 2009;Callahan, 2009;Friedman-Nimz, 2009; Worrell) and touched on the ever-increasing prevalence of using multiple measures as a best practice in gifted and talented identification. In addition, the 2008-2009 State of the States Report (National Association for Gifted Children [NAGC], 2009) noted that multiple measures have become the most common form of identification for admission into gifted education programs used across the responding states. Multiple measures are used to improve aspects of measurement and evaluation, such as reliability, validity, fairness, and security. However, simply using multiple measures does not guarantee improvement in any of these areas (Chester, 2003;Cronbach, Linn, Brennan, & Haertel, 1997). In the context of student achievement, the use of multiple measures to improve these areas has been explored (Douglas & Mislevy, 2010; Erpenbach, Carlson, LaMarca, Winter, & Council of Chief State School Officers, 2002;Ryan, 2002); however, little research has considered the use of multiple measures in the context of identification of gifted students. Despite the increasing level of prevalence in state regulations and best practices, the degree to which schools and school districts correctly apply multiple measures for identification purposes remains unknown. Lohman (2006) and Lohman and Renzulli ( 2007) presented examples for combinations, but no research has documented the effectiveness of such methods in identifying gifted and talented students. Despite commonplace reference to the use of multiple measures in almost all aspects of education, including gifted education identification, no studies have evaluated the variability in the identified gifted populations that occurs when different combination methods are implemented. Literature Review Historical PracticesThe question regarding "Who are the gifted?" has raged for nearly 100 years, and yet only the weakest consensus exists on this topic. In fact, a recent study by Dai, Swanson, and Cheng (2011) noted that identification remains one of the most studied topics in the scholarly literature related to gifted education. Most states have formalized the definition regarding how to identify gifted and talented students. As of 2009, 41 of the 47 responding states reported having a formal definition (NAGC, 2009). Within these 41 states, the most common areas included (in order of prevalence) were intellectual, creative, performing/visual arts, specific academic, and leadership. However, of these 41states, only 29 states require its use by all local education agencies. Despite this variability, those states that have a specific definition and require its use

Creative attitude in a group of youths gifted in the domain of science subjects

“…Analysis of the literature on the topic, as well as this current research, allows one to estimate that about 2-3% of the population are characterised by exceptional talent (Renzulli, 2005), and about 13% of the population qualify as very talented (Foryś, 2014;Freeman, 2010;Limont, 2012). In order to decrease the chance of failing to identify and include in additional programmes students who are gifted, but who have difficulties (Dyrda, 2000;Reis & McCoach, 2000;Rimm, 2000), as well as to avoid excessive inclusion of students in the gifted group due to only analysing test results (Callahan et al, 2013) and school achievements (McBee, Peters, & Waterman, 2014), it is worthwhile suggesting to teachers a procedure of identification of talents that involves not only psychometric criteria (results of standardised tests) but also teachers' nominations, analyses of students' creations and their interest in a given topic. One should not underestimate the importance of raising teachers' sensitivity to creativity, which is grounded in this study's results.…”

Exploring Alignment in Gifted Education Program Policies and Practices

“…Some aspects of identification processes can be interpreted in different ways depending upon context, as multiple criteria for identification can serve as multiple barriers or hurdles a student must jump through in order to be identified, while multiple criteria for identification can also serve to provide a coherent profile of a student to better inform identification processes. One of these practices is a recommended practice (using multiple pieces of information in identification) and one is not (creating multiple barriers to identification); yet it would be difficult to ascertain what specifically is occurring in a particular district with only a "present" or "not-present" criteria (McBee et al, 2014). Though the tool does not provide enough information on its own, noting the presence (or lack of presence) of the NCRGE theory of change criteria using the tool provided an additional layer of information to analyze the complexities of gifted education policies and practices.…”

“…In general, researchers recommend the use of multiple measures for identification, the creation of student profiles, observations of students in the classroom, and some form of universal screening procedure (e.g., Callahan, 2005;Olzsewski-Kubilius & Clarenbach, 2012;Renzulli, 1984;Renzulli & Delcourt, 1986;Stambaugh, 2007). Additionally, results indicate multiple measures should be utilized in ways that expand identification pools (McBee et al, 2014). It is important to note whether and to what extent state and district policies in place are aligned with these recommendations.…”

“…Recent research provides further insight into the nuances of a multiple indicator approach to identification (McBee, Peters, & Waterman, 2014). In their simulation study investigating the ways multiple indicator methods can be applied, the researchers found that using an "and" (multiple cut-off scores), "or" (cut-off score on at least one measure), or "mean" (average of standardized scores) application resulted in the identification of very different populations of students.…”

“…rule results in the most broad, or most heterogeneous group of gifted students based on proposed scores on hypothetical indicators for identification (McBee et al, 2014). Table 3 …”