Enhancing the Wisdom of the Crowd With Cognitive-Process Diversity: The Benefits of Aggregating Intuitive and Analytical Judgments

Abstract

Drawing on dual-process theory, we suggest that the benefits that arise from combining several quantitative individual judgments will be heightened when these judgments are based on different cognitive processes. We tested this hypothesis in three experimental studies in which participants provided estimates for the dates of different historical events (Study 1, N = 152), made probabilistic forecasts for the outcomes of soccer games (Study 2, N = 98), and estimated the weight of individuals on the basis of a photograph (Study 3, N = 3,695). For each of these tasks, participants were prompted to make judgments relying on an analytical process, on their intuition, or (in a control condition) on no specific instructions. Across all three studies, our results show that an aggregation of intuitive and analytical judgments provides more accurate estimates than any other aggregation procedure and that this advantage increases with the number of aggregated judgments.

Scaling up fact-checking using the wisdom of crowds

“…Together, this body of work demonstrates the broad power of crowdsourcing despite systematic polarization in attitudes among members of the crowd. Our results also show the limits of a recent finding that cognitively diverse groups made of a mix of intuitive and analytical thinkers perform better than crowds of only more analytical thinkers ( 57 ). We do not observe this effect in our data, where mixed groups were no more effective—and if anything, slightly less effective—than groups of only more analytic thinkers.…”

Intuition as Emergence: Bridging Psychology, Philosophy and Organizational Science

“…So the article has formed the basis of a number of subsequent influential literature reviews, conceptual contributions, and methodological and empirical works; drawing from it, scholars have differentiated intuition from related constructs (Hodgkinson et al, 2008(Hodgkinson et al, , 2009aSadler-Smith, 2010, 2015Salas et al, 2010;Blume and Covin, 2011;Akinci and Sadler-Smith, 2012, 2013, 2020Baldacchino, 2013Baldacchino, , 2019Baldacchino et al, 2015;Healey et al, 2015;Pratt and Crosina, 2016;Calabretta et al, 2017;Okoli et al, 2021;Sadler-Smith et al, 2021). In addition, the article has spread over psychology and neuroscience (Hodgkinson et al, 2008;Hogarth, 2010;Gore and Sadler-Smith, 2011;Hodgkinson and Healey, 2011;Ben-Soussan et al, 2020;Keck and Tang, 2020;Korteling and Toet, 2020;Li et al, 2020;Stephens et al, 2020;West et al, 2020;Muñoz-Cobos and Postigo-Zegarra, 2021;Reynolds et al, 2021;Yu et al, 2021;Zhang et al, 2021), as well as distant disciplinary domains, such as medicine and health sciences (Glatzer et al, 2020;Cameron and Singh, 2021;Chlupsa et al, 2021) or engineering and design (Cash and Maier, 2021;de Rooij et al, 2021;Paige et al, 2021;…”

A hypothesis test algorithm for determining when weighting individual judgments reliably improves collective accuracy or just adds noise.

“…Corresponding to sample sizes ( n = 16, 20, 24, 28, 32), the total rejection proportions for OW are (0.000, 0.001, 0.003, 0.002, 0.008) and the total rejection proportions for CWM are (0.000, 0.002, 0.004, 0.005, 0.008). This finding indicates that we cannot trust these weighted averages to beat the simple average in Keck and Tang’s (2020) data, and many more samples are necessary before decision makers should consider using a weighted average instead of the simple average to combine these judgments.…”