Does interaction matter? Testing whether a confidence heuristic can replace interaction in collective decision-making

Bang, Dan; Fusaroli, Riccardo; Tylén, Kristian; Olsen, Karsten; Latham, Peter E.; Lau, Jennifer Y. F.; Roepstorff, Andreas; Rees, Geraint; Frith, C. D.; Bahrami, Bahador

doi:10.1016/j.concog.2014.02.002

Cited by 80 publications

(127 citation statements)

References 29 publications

Supporting

Mentioning

120

Contrasting

Unclassified

Order By: Relevance

“…We investigated the performance of virtual groups of diagnosticians using either of two collective intelligence rules: the confidence rule (17,20) and the majority rule (34,35). For any particular group evaluating any particular case, the confidence rule adopts the judgment of the most confident diagnostician, whereas the majority rule adopts the judgment receiving the most support within that group.…”

Section: Resultsmentioning

confidence: 99%

“…Both datasets include the judgments of experts who independently evaluated the same cases and rated their confidence in each diagnosis. We created virtual groups of diagnosticians who evaluated the cases "together" using two collective intelligence rules: the confidence rule (17,20) and the majority rule (34,35). Confidence rule.…”

Section: Methodsmentioning

confidence: 99%

“…Our work builds on recent findings on combining decisions, a research paradigm known as "two heads better than one" (15)(16)(17)(18)(19)(20). In their seminal study, Bahrami et al (15) showed that two individuals permitted to communicate freely while engaging in a visual perception task, achieved better results than the better of the two did alone.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Boosting medical diagnostics by pooling independent judgments

Kurvers

Herzog

Hertwig

et al. 2016

Proc. Natl. Acad. Sci. U.S.A.

143

134

View full text Add to dashboard Cite

Collective intelligence refers to the ability of groups to outperform individual decision makers when solving complex cognitive problems. Despite its potential to revolutionize decision making in a wide range of domains, including medical, economic, and political decision making, at present, little is known about the conditions underlying collective intelligence in real-world contexts. We here focus on two key areas of medical diagnostics, breast and skin cancer detection. Using a simulation study that draws on large real-world datasets, involving more than 140 doctors making more than 20,000 diagnoses, we investigate when combining the independent judgments of multiple doctors outperforms the best doctor in a group. We find that similarity in diagnostic accuracy is a key condition for collective intelligence: Aggregating the independent judgments of doctors outperforms the best doctor in a group whenever the diagnostic accuracy of doctors is relatively similar, but not when doctors' diagnostic accuracy differs too much. This intriguingly simple result is highly robust and holds across different group sizes, performance levels of the best doctor, and collective intelligence rules. The enabling role of similarity, in turn, is explained by its systematic effects on the number of correct and incorrect decisions of the best doctor that are overruled by the collective. By identifying a key factor underlying collective intelligence in two important real-world contexts, our findings pave the way for innovative and more effective approaches to complex real-world decision making, and to the scientific analyses of those approaches.collective intelligence | groups | medical diagnostics | dermatology | mammography C ollective intelligence, that is, the ability of groups to outperform individual decision makers when solving complex cognitive problems, is a powerful approach for boosting decision accuracy (1-7). However, despite its potential to boost accuracy in a wide range of contexts, including lie detection, political forecasting, investment decisions, and medical decision making (8-14), little is known about the conditions that underlie the emergence of collective intelligence in real-world domains. Which features of decision makers and decision contexts favor the emergence of collective intelligence? Which decision-making rules permit this potential to be harnessed? We here provide answers to these important questions in the domain of medical diagnostics.Our work builds on recent findings on combining decisions, a research paradigm known as "two heads better than one" (15)(16)(17)(18)(19)(20). In their seminal study, Bahrami et al. (15) showed that two individuals permitted to communicate freely while engaging in a visual perception task, achieved better results than the better of the two did alone. Koriat (17) subsequently demonstrated that this collective intelligence effect also emerges in the absence of communication when the "maximum-confidence slating algorithm" (hereafter called confidence rule) is used and th...

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Boosting medical diagnostics by pooling independent judgments

Kurvers

Herzog

Hertwig

et al. 2016

Proc. Natl. Acad. Sci. U.S.A.

143

134

View full text Add to dashboard Cite

show abstract

“…In such situations, an equality bias can be damaging for the group. Indeed, previous research has shown that group performance in the task described here depends critically on how similar group members are in terms of their competence (6,8,9). Future research could refine our understanding of this bias by assessing the actual real-life probability of the breakdown of the similarity assumption.…”

Section: Discussionmentioning

confidence: 99%

“…One recent suggestion is to weight each opinion by the confidence with which it is expressed (6,7). However, this strategy may fail dramatically (8,9) when "the fool who thinks he is wise" is paired with "the wise who [thinks] himself to be a fool" (10). In the face of such a competence gap, knowing whose confidence to lean on is critical (11).…”

mentioning

confidence: 99%

Equality bias impairs collective decision-making across cultures

Mahmoodi

Bang

Olsen

et al. 2015

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

123

View full text Add to dashboard Cite

We tend to think that everyone deserves an equal say in a debate. This seemingly innocuous assumption can be damaging when we make decisions together as part of a group. To make optimal decisions, group members should weight their differing opinions according to how competent they are relative to one another; whenever they differ in competence, an equal weighting is suboptimal. Here, we asked how people deal with individual differences in competence in the context of a collective perceptual decision-making task. We developed a metric for estimating how participants weight their partner's opinion relative to their own and compared this weighting to an optimal benchmark. Replicated across three countries (Denmark, Iran, and China), we show that participants assigned nearly equal weights to each other's opinions regardless of true differences in their competence-even when informed by explicit feedback about their competence gap or under monetary incentives to maximize collective accuracy. This equality bias, whereby people behave as if they are as good or as bad as their partner, is particularly costly for a group when a competence gap separates its members.social cognition | joint decision-making | bias | equality

show abstract

Collective decision making reduces metacognitive control and increases error rates, particularly for overconfident individuals

Blanchard

Jackson

Kleitman

2020

Behavioral Decision Making

View full text Add to dashboard Cite

This research aimed to investigate the changes in judgment accuracy, confidence, control thresholds, and decision outcomes when people act in two‐person groups (dyads) compared with acting individually. First, we used interacting dyads to determine the metacognitive and behavioral outcomes of collective decision making and compared them with those of individuals. Second, we examined whether these changes were related to the trait‐confidence and bias of individuals working together. Using a within‐person design, undergraduate psychology students (N = 116) completed a General‐knowledge Test individually, then together as a dyad. Each question was accompanied by a confidence rating and a decision to bet $10 on the answer. Dyads had significantly higher confidence and lower control thresholds than individuals. They were also significantly more decisive (made more bets) and reckless (lost a higher rate of bets) than when working alone. Thus, we observed a higher rate of decision errors for groups than individuals. The results also demonstrated the important role of individual differences: Overconfident individuals became even more confident, decisive, and reckless when working together compared with less confident or underconfident individuals working together. These findings have important theoretical and applied implications for collective decision making; metacognitive bias and potentially control thresholds may be targeted to alleviate the larger error rates and guide the formation of more effective groups.

show abstract

Does interaction matter? Testing whether a confidence heuristic can replace interaction in collective decision-making

Cited by 80 publications

References 29 publications

Boosting medical diagnostics by pooling independent judgments

Boosting medical diagnostics by pooling independent judgments

Equality bias impairs collective decision-making across cultures

Collective decision making reduces metacognitive control and increases error rates, particularly for overconfident individuals

Contact Info

Product

Resources

About