Divergent productions of metaphors: Combining many-facet Rasch measurement and cognitive psychology in the assessment of creativity.

Abstract

This article presents a new method for the assessment of creativity in tasks such as "The camel is ________ of the desert." More specifically, the study uses Tourangeau and Sternberg's (1981) domain interaction model to produce an objective system for scoring metaphors produced by raters and the many-facet Rasch measurement to model the rating scale structure of the scoring points, item difficulty, and rater severity analysis, thus making it possible to have equated latent scores for subjects, regardless of rater severity. This study also investigates 4 aspects of the method: reliability, correlation between quality and quantity, criterion validity, and correlation with fluid intelligence. The database analyzed in this study consists of 12,418 responses to 9 items that were given by 975 persons. Two to 10 raters scored the quality and flexibility of each metaphor on a 4-point scale. Raters were counterbalanced in a judge-linking network to permit the equating of different "test forms" implied in combinations of raters. The reliability of subjects' latent quality scores was .88, and the correlation between quality and quantity was low (r = -.14), thus showing the desired separation between the 2 parameters established for the task scores. The latent score on the test was significantly associated with the profession that requires idea production (r = .19), and the latent scores for the correlation between creativity and fluid intelligence were high, |3 = .51, even after controlling for crystalized intelligence (r = .47). Mechanisms of fluid intelligence, executive function, and creativity are discussed.

Evaluación Multidimensional de la Superdotación: Criterios de validez de la Batería de Inteligencia y Creatividad para predecirlos talentos artísticos y académicos

“…For example, Beaty and Silvia shows that crystallized knowledge could only predict individuals' ability to generate conventional metaphors (r = .30), but fluid intelligence predicts creative metaphor production (r = .45) and is not associated with conventional metaphors production. We replicated this finding observing a strong association of divergent production of metaphors with fluid reasoning (r =.60, similar to what was found in Primi, 2014). David, Morais, Primi and Miguel (2014) found that scores on Metaphor Creation Test has a strong association with grades in high school Portuguese students.…”

“…Metaphors task is defined as "an instrument for evaluation of cognitive components of creativity" (Primi, Miguel, Couto, & Muniz, 2007, p.198). Recent studies shows a stronger role of intelligence in creative thinking than previously thought especially implicating executive functions, working memory and fluid intelligence in the production of creative metaphors (Benedeck et al, 2013;Chiappe & Chiappe, 2007;Kazmerski, Blasko, & Dessalegn, 2003;Primi, 2014;Silvia & Beaty, 2012). For example, Beaty and Silvia shows that crystallized knowledge could only predict individuals' ability to generate conventional metaphors (r = .30), but fluid intelligence predicts creative metaphor production (r = .45) and is not associated with conventional metaphors production.…”

“…Many scholars have questioned if the divergent tests show predictive validity (Baer, 1994;Feist, 2004;Gardner, 1993;Han, 2003;Jarosewich et al, 2002;Kogan & Pankove, 1974;Schraw, 2005). But a lot of progress have been occurring recently such as new assessment methods (Silvia, Winterstein, Willse, et al, 2008), new methods of data analysis Primi, 2014;Silvia, 2007Silvia, , 2011, predictive validity of creativity assessment tools (Kim, 2006;Plucker & Runco, 1998;Runco, Millar, Acar, & Cramond, 2011;Zeng, Proctor, & Salvendy, 2011), the structure and independence of evaluated traits (Chase, 1985;Clapham, 1998;Heausler & Thomson, 1988;Primi, Nakano, Morais, Almeida, & David, 2013;Runco & Mraz, 1992) and the validity of subjective rating (Benedek, Mühlmann, Jauk, & Neubauer, 2013;Chen, Kasof, Himsel, Greenberger, Dong, & Xue, 2002;Kaufman, Baer, Agars, & Loomis, 2010;Kaufman, Lee, Baer, & Lee, 2007;Silvia, 2011;Silvia, Martin, & Nusbaum, 2009;Silvia, Winterstein, Willse, et al, 2008). Also, in order the gifted children identification (children from 9 to 12 years old) an international research project is validating the Aurora Battery (Chart, Grigorenko, & Sternberg, 2008) in different countries, a specific battery considering three kinds of intelligences: analytical, synthetic (creative) and practical.…”

Approaches to Measuring Creativity: A Systematic Literature Review

“…Creative people might do these activities more creatively, but not necessarily more repeatedly (Baer, 2016). To complicate matters further, recent studies (Primi, 2014;Silvia, 2015) have stated that divergent thinking and intelligence are more closely related than initially thought.If this is generally the case, then the variance of creative achievement may largely be attributed to differences in intelligence, not to differences in divergent thinking ability (Hocevar & Bachelor, 1989;Plucker, 1999a;Plucker & Renzulli, 1999). For instance, most statistical analyses employed in Torrance's (1972a) study were simple correlation coefficients that do not control for the effect of other external variables.…”

Methodological Issues in Measuring Creativity: A Systematic Literature Review

“…Perhaps this is not surprising given that many of the creativity instruments were developed and standardized before the advent of modern psychometric analyses (Fishkin in creativity research, and although limited, their results are promising. These efforts addressed some critical issues in creativity measurement, such as evaluating reliability and validity (Chermahini, Hickendorff, & Hommel, 2012;Karwowski, 2014;Lee, Lee, & Youn, 2005;Primi, 2014;Silvia, 2011;Silvia et al, 2008;Wang, Ho, Cheng, & Cheng, 2014;Zampetakis, 2010), testing domain-specificity of creativity (Barbot, Tan, Randi, SantaDonato, & Grigorenko, 2012;Chen et al, 2006;Silvia, Kaufman, & Pretz, 2009), evaluating the rater's effect on performance assessment (Hung, Chen, & Chen, 2012) and modelling relationships between creativity and other constructs (Nusbaum & Silvia, 2011;Silvia, 2008). Expanding the use of these modern analyses could provide a better understanding of conflicting results in creativity research.…”