The Composition of Optimally Wise Crowds

Davis‐Stober, Clintin P.; Budescu, David V.; Broomell, Stephen B.; Dana, Jason

doi:10.1287/deca.2015.0315

Cited by 53 publications

(51 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“… 2. In addition, the extent of bracketing and hence the benefits of aggregation might also be affected by each judge’s tendency to systematically under- or overestimate the true values across a number of judgments ( Davis-Stober et al, 2015 ; Davis-Stober, Budescu, Dana, & Broomell, 2014 ). See the Supplemental Material available online for further information.…”

mentioning

confidence: 99%

Enhancing the Wisdom of the Crowd With Cognitive-Process Diversity: The Benefits of Aggregating Intuitive and Analytical Judgments

Keck

Tang

2020

Psychol Sci

View full text Add to dashboard Cite

Drawing on dual-process theory, we suggest that the benefits that arise from combining several quantitative individual judgments will be heightened when these judgments are based on different cognitive processes. We tested this hypothesis in three experimental studies in which participants provided estimates for the dates of different historical events (Study 1, N = 152), made probabilistic forecasts for the outcomes of soccer games (Study 2, N = 98), and estimated the weight of individuals on the basis of a photograph (Study 3, N = 3,695). For each of these tasks, participants were prompted to make judgments relying on an analytical process, on their intuition, or (in a control condition) on no specific instructions. Across all three studies, our results show that an aggregation of intuitive and analytical judgments provides more accurate estimates than any other aggregation procedure and that this advantage increases with the number of aggregated judgments.

show abstract

mentioning

confidence: 99%

Enhancing the Wisdom of the Crowd With Cognitive-Process Diversity: The Benefits of Aggregating Intuitive and Analytical Judgments

Keck

Tang

2020

Psychol Sci

View full text Add to dashboard Cite

show abstract

“…The idea that cognitive diversity improves crowd judgment is well supported (6,7). Cognitive diversity-variation in people's judgments or how they think-is hard to directly assess.…”

mentioning

confidence: 99%

Demographically diverse crowds are typically not much wiser than homogeneous crowds

Oliveira

Nisbett

2018

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Averaging independent numerical judgments can be more accurate than the average individual judgment. This "wisdom of crowds" effect has been shown with large, diverse samples, but the layperson wishing to take advantage of this may only have access to the opinions of a small, more demographically homogeneous "convenience sample." How wise are homogeneous crowds relative to diverse crowds? In simulations and survey studies, we demonstrate three necessary conditions under which small socially diverse crowds can outperform socially homogeneous crowds: Social identity must predict judgment, the effect of social identity on judgment must be at least moderate in size, and the average estimates of the social groups in question must "bracket" the truth being judged. Seven survey studies suggest that these conditions are rarely met in real judgment tasks. Comparisons between the performances of diverse and homogeneous crowds further confirm that social diversity can make crowds wiser but typically by a very small margin.

show abstract

“…The bias and variance in this decomposition is the average of the individual biases and variances. MSE decompositions have been used in machine learning (e.g., Geman, Bienenstock, & Doursat, 1992) and judgment and decision making research (e.g., Davis-Stober, Budescu, Broomell, & Dana, 2015), but have yet to be used to connect the characteristics of individual judgment strategies with the performance of aggregated judgments.…”

Section: The Strategy Aggregation Effect In Judgmentmentioning

confidence: 99%

“…Recently, there has been considerable advancement in understanding when and why aggregation produces more, or less, accurate judgments (Davis-Stober, Budescu, Dana, & Broomell, 2014;Davis-Stober et al, 2015). These advancements rely mainly on the decomposition of mean squared error into components of bias, variance, and covariance.…”

Section: Aggregating Judgments: Wisdom Of Crowdsmentioning

confidence: 99%

The Strategy Aggregation Effect in Judgment

Olsson¹

2020

Preprint

View full text Add to dashboard Cite

Predicting a criterion that is probabilistically related to pieces of information, or cues, is a paradigmatic judgment task that has been investigated both, in research trying to identify the individual judgment and decision making strategies people use, and in the wisdom-of-crowds literature where the focus is on how aggregation can improve accuracy. I combine these two lines of research to investigate how the performance of individual and aggregated linear strategies are affected by different environmental and group aggregation factors and how performance differences between individual and aggregated linear strategies can be understood in a unified framework. I show that constrained linear strategies are more accurate for individual judgments, but when these judgments are averaged, an unconstrained linear strategy is more accurate. This strategy aggregation effect can be understood by analyzing a decomposition of the mean squared error into bias, variance, and covariance. Because of their lower bias but higher variance, unconstrained linear strategies perform worse for individual judgments, but better for averaged judgments where aggregation minimizes variance. In simulations with artificial and real environments, I further show that this aggregation effect does not occur if there are correlations between individual judgments. Here, constrained linear strategies always outperform an unconstrained linear strategy, because the larger covariance component of the unconstrained linear strategy outweighs its lower bias. I end with real-world implications of the results for cognitive strategies and decision environments in group and organizational settings.

show abstract

The Composition of Optimally Wise Crowds

Cited by 53 publications

References 25 publications

Enhancing the Wisdom of the Crowd With Cognitive-Process Diversity: The Benefits of Aggregating Intuitive and Analytical Judgments

Enhancing the Wisdom of the Crowd With Cognitive-Process Diversity: The Benefits of Aggregating Intuitive and Analytical Judgments

Demographically diverse crowds are typically not much wiser than homogeneous crowds

The Strategy Aggregation Effect in Judgment

Contact Info

Product

Resources

About