Exploration of discriminant validity in divergent thinking tasks: A meta-analysis.

Abstract

Discriminant validity has been a major concern regarding the indices of divergent thinking (DT). The present study used a meta-analytic approach to understand the nature of the problem by synthesizing the research between 2009 to 2019. Three data sets obtained from 242 different studies were analyzed by adopting a multilevel modeling approach, each consisting of one of the paired correlations between fluency, flexibility, and originality. Overall, the correlation of fluency is stronger with flexibility (r = .79, k = 247, m = 178, N = 46,933) than originality (r = .62, k = 465, m = 250, N = 58,656), showing lower discriminant validity with the former compared with the latter. This is the case even when the traditional scoring methods of flexibility (r = .81, k = 228, m = 171, N = 45,991) and originality (rs = .66 and .71, ks = 60 and 102, ms = 49 and 66, Ns = 10,514 and 10,535, respectively) were used. Among the three pairs of correlations, the flexibility-originality pair had the lowest correlation (r = .56, k = 250, m = 161, N = 41,251). Moderator analyses indicated that the averaging approach to aggregation and mixed task modality (i.e., the use of both verbal and figural DT tasks) are useful strategies to improve the discriminant validity of fluency with both flexibility and originality. Specific scoring techniques, such as semantic distance and inverse proportional scoring, improved discriminant validity on originality, and the use of multiple items undermined discriminant validity for flexibility.

Socio‐Economic Status and Creativity: A Meta‐Analysis

“…The concerns around the use of the flexibility index can be viewed as negligible unless high‐stakes decisions (e.g., gifted identification) are made based on them. On the other hand, classic flexibility scoring is highly confounded by fluency (Acar, Ogurlu, & Zorychta, 2022), and the differences in mean effects across various DT indices may be different from what we report here when the fluency confound is controlled (see Kaczykowski, Forthmann, Benedek, & Holling, 2021). Second, the overall positive relationship between SES and creativity should not mask the sizable amount of opposing evidence, as prediction intervals indicate.…”

Measuring flexibility: A text-mining approach

“…There was a strong correlation between OCS-Flex mean and OCS-Orig mean latent factors. This strong relationship between originality and flexibility has been observed previously and could be accounted for by the theoretical relationship between the two dimensions ( Runco, 1986b ; Acar and Runco, 2014 ; Acar et al, 2022 ). The correlation observed between maximum OCS-Flex and OCS-Flu factors suggests that some of the fluency confound still remained using this scoring method.…”

“…Due to its extensively studied validity, the TTCT publisher-generated scores were used to validate OCS-Flex scores in our first Study ( Chase, 1985 ; Treffinger, 1985 ; Torrance, 1998 ; Cramond et al, 2005 ; Kim, 2006 ; Acar et al, 2022 ). All proposed OCS-Flex scores (mean, maximum, sum) correlated significantly with Pub-Flex scores providing proof of the validity of our scoring method.…”

“…Flexibility suffers from the same confound as fluency in which a greater number of ideas generated increases the likelihood of those ideas belonging to different conceptual or semantic categories. Although research on the phenomenon is sparse in the context of flexibility, Acar et al (2022) , in a recent meta-analytic review, found a strong correlation between fluency and flexibility ( r = 0.79) and concluded that fluency bias in the context of flexibility is even more profound than in the case of originality. Several summarization methods may function as solutions to the quantity-quality confound, such as using average quality (i.e., mean originality or mean flexibility), or only counting the most creative idea a participant produces (i.e., maximum originality or maximum flexibility), have been suggested ( Runco and Acar, 2012 ; Acar and Runco, 2019 ; Reiter-Palmon et al, 2019 ).…”

The involvement of conflict monitoring and rewards processing in the appropriateness evaluation of creativity: An event-related potential-based analysis.

“…It is clear that the role of appropriateness in creative solutions has generally been overlooked despite being implicitly assumed as a prerequisite for creativity. Similarly, in the traditional assessment of creativity, most existing studies have measured only originality or novelty (alongside fluency or flexibility; see also reviews in Acar et al, 2022), but not appropriateness (Abraham, 2016;Khalil & Moustafa, 2022). Meanwhile, standard objective measurements of novelty (as well as fluency or flexibility) often use only adequately appropriate responses when scoring, with inappropriate responses excluded (Benedek et al, 2013;Forthmann et al, 2016Forthmann et al, , 2017Forthmann et al, , 2023Reiter-Palmon et al, 2019;Silvia et al, 2008).…”