Measuring divergent thinking originality with human raters and text-mining models: A psychometric comparison of methods.

Dumas, Denis; Organisciak, Peter; Doherty, Michael

doi:10.1037/aca0000319

Cited by 104 publications

(198 citation statements)

References 71 publications

Supporting

Mentioning

163

Contrasting

Order By: Relevance

“…When listing uses of a brick in the Alternative Uses Task, for example, a brick layer may provide different responses than a lawyer, influencing their scores within the sample and leading to more uncontrollable variation. Similarly, different words in the Alternative Uses Task can prompt different responses with varying reliability between computational scoring and manually scored responses (26). The DAT avoids these issues by giving an open-ended prompt and using an international database.…”

Section: Strengthsmentioning

confidence: 99%

“…Like all computational scoring methods, the DAT depends on the training model and corpus used. We chose the GloVe algorithm and the Common Crawl corpus; this combination correlates best with human judgements on the Alternative Uses Task (26). For simplicity, we chose a pre-trained model that is freely available and widely used.…”

Section: Future Researchmentioning

confidence: 99%

“…To address these limitations, recent efforts have moved towards using computational algorithms to score responses (10,16,25,26). Computational methods may also improve the theoretical grounding of the measures, as the assumptions required to score the responses must be made explicit in the program code (10,12).…”

Section: Main Text 31 Introductionmentioning

confidence: 99%

“…Computational methods may also improve the theoretical grounding of the measures, as the assumptions required to score the responses must be made explicit in the program code (10,12). Researchers have successfully scored the Alternative Uses Task using semantic models, achieving scores similar to human ratings (10,16,25,26).…”

Section: Main Text 31 Introductionmentioning

confidence: 99%

“…It requires analysing complex multi-word answers, usually by computing the semantic distance between each of the words provided in the response, independent of their order. Common methods also remove stop words (e.g., the, which, of ) in order to increase the consistency of the scoring (26,27). These common practices can obscure meaning, which is a well-known issue in the field of artificial intelligence.…”

Section: Main Text 31 Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Naming unrelated words reliably predicts creativity

Olson¹,

Nahas²,

Chmoulevitch³

et al. 2020

Preprint

View full text Add to dashboard Cite

Several theories posit that creative people are able to generate more divergent ideas. If this is correct, the simple act of naming unrelated words and then measuring the semantic distance between them could serve as an objective measure of creativity. To test this hypothesis, we asked 8,892 participants to name 10 words that are as different from each other as possible. A computational algorithm then estimated the average semantic distance between the words; related words (e.g., “cat” and “dog”) have shorter distances than unrelated ones (e.g., “cat” and “thimble”). We predicted that people producing greater semantic distances would also score higher on traditional creativity measures. In Study 1, there were moderate to strong correlations between semantic distance and two other creativity measures (the Alternative Uses Task and the Bridge-the-Associative-Gap Task). In Study 2, with participants from 98 countries, semantic distances varied only slightly by demographic variables which suggests that the measure can be used without modification across diverse populations. There was also a positive correlation between semantic distance and performance on problem solving tasks known to predict creativity. Overall, semantic distance correlated at least as strongly with established creativity measures as those measures did with each other. Naming unrelated words in what we call the Divergent Association Task can thus serve as a brief, reliable, and objective measure of creativity.

show abstract

Section: Strengthsmentioning

confidence: 99%

Section: Future Researchmentioning

confidence: 99%

Section: Main Text 31 Introductionmentioning

confidence: 99%

Section: Main Text 31 Introductionmentioning

confidence: 99%

Section: Main Text 31 Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Naming unrelated words reliably predicts creativity

Olson¹,

Nahas²,

Chmoulevitch³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

The role of alpha oscillations in free‐ and goal‐directed semantic associations

Zioga,

Kenett,

Giannopoulos

et al. 2024

Human Brain Mapping

View full text Add to dashboard Cite

Alpha oscillations are known to play a central role in several higher‐order cognitive functions, especially selective attention, working memory, semantic memory, and creative thinking. Nonetheless, we still know very little about the role of alpha in the generation of more remote semantic associations, which is key to creative and semantic cognition. Furthermore, it remains unclear how these oscillations are shaped by the intention to “be creative,” which is the case in most creativity tasks. We aimed to address these gaps in two experiments. In Experiment 1, we compared alpha oscillatory activity (using a method which distinguishes genuine oscillatory activity from transient events) during the generation of free associations which were more vs. less distant from a given concept. In Experiment 2, we replicated these findings and also compared alpha oscillatory activity when people were generating free associations versus associations with the instruction to be creative (i.e. goal‐directed). We found that alpha was consistently higher during the generation of more distant semantic associations, in both experiments. This effect was widespread, involving areas in both left and right hemispheres. Importantly, the instruction to be creative seems to increase alpha phase synchronisation from left to right temporal brain areas, suggesting that intention to be creative changed the flux of information in the brain, likely reflecting an increase in top‐down control of semantic search processes. We conclude that goal‐directed generation of remote associations relies on top‐down mechanisms compared to when associations are freely generated.

show abstract

Four Text‐Mining Methods for Measuring Elaboration

Dumas

Organisciak

Maio

et al. 2020

Journal of Creative Behavior

Self Cite

View full text Add to dashboard Cite

When individuals engage in divergent thinking, they vary on their Elaboration, or the degree to which they explain and embellish their responses. Although Elaboration has been considered relevant to creativity research for decades, its measurement has remained under‐developed. Here, we leverage technical and methodological perspectives from the text‐mining literature to posit four methods for quantifying elaboration: Unweighted Word Count, Stoplisted Inclusion, Part of Speech Inclusion, and Inverse Frequency Weighting. Although the Unweighted Word Count method is becoming typical in the field, more complex weighting methods appear to better fit the conceptualization of Elaboration. We explain the benefits of each of the included methods and demonstrate their application to responses from the Alternate Uses Task: showing that all four of these text‐mining methods produced Elaboration scores with high levels of reliability, but the Stoplisted Inclusion method appeared to maximize the score validity both in terms of criteria correlations and power to discriminate among creative experts and non‐experts. We offer an open‐access module for creativity researchers to apply these methods to their own data via our laboratory website [https://openscoring.du.edu/].

show abstract

Measuring divergent thinking originality with human raters and text-mining models: A psychometric comparison of methods.

Cited by 104 publications

References 71 publications

Naming unrelated words reliably predicts creativity

Naming unrelated words reliably predicts creativity

The role of alpha oscillations in free‐ and goal‐directed semantic associations

Four Text‐Mining Methods for Measuring Elaboration

Contact Info

Product

Resources

About