The distribution of initial estimates moderates the effect of social influence on the wisdom of the crowd

Abstract

Whether, and under what conditions, groups exhibit “crowd wisdom” has been a major focus of research across the social and computational sciences. Much of this work has focused on the role of social influence in promoting the wisdom of the crowd versus leading the crowd astray and has resulted in conflicting conclusions about how social network structure determines the impact of social influence. Here, we demonstrate that it is not enough to consider the network structure in isolation. Using theoretical analysis, numerical simulation, and reanalysis of four experimental datasets (totaling 2885 human subjects), we find that the wisdom of crowds critically depends on the interaction between (i) the centralization of the social influence network and (ii) the distribution of the initial individual estimates. By adopting a framework that integrates both the structure of the social influence and the distribution of the initial estimates, we bring previously conflicting results under one theoretical framework and clarify the effects of social influence on the wisdom of crowds.

Beyond Playing 20 Questions with Nature: Integrative Experiment Design in the Social and Behavioral Sciences

“…Similarly, prior work has variously suggested that group performance depends on the composition of the group with respect to individual-level traits as captured by, say, average skill (Bell, 2007; Devine & Philips, 2001; LePine, 2003; Stewart, 2006), skill diversity (Hong & Page, 2004; Page, 2008), gender diversity (Schneid, Isidor, Li, & Kabst, 2015), social perceptiveness (Engel, Woolley, Jing, Chabris, & Malone, 2014; Kim et al, 2017; Woolley, Chabris, Pentland, Hashmi, & Malone, 2010), and cognitive-style diversity (Aggarwal & Woolley, 2018; Ellemers & Rink, 2016), all of which could be represented as dimensions of the design space. Finally, group-process variables might include group size (Mao et al, 2016), properties of the communication network (Almaatouq, Rahimian, Burton, & Alhajri, 2022; Becker et al, 2017; Mason & Watts, 2012), and the ability of groups to reorganize themselves (Almaatouq et al, 2020). Together, these variables might identify upward of 50 dimensions that define a design space of possible experiments for studying group synergy through integrative experiment design, where any given study should, in principle, be assignable to one unique point in the space 5…”

Algorithmically mediating communication to enhance collective decision-making in online social networks

“…It may be worthwhile to experimentally explore how the rewiring algorithms presented here would fare in other prediction contexts (e.g., continuous, numeric predictions rather than probabilistic predictions); however, several past studies already provide data from crowdwisdom experiments in which communication took place over static network structures that are ripe for re-analysis (e.g., Becker et al, 2017Becker et al, , 2019Gürçay et al, 2015;Lorenz et al, 2011). In fact, a re-analysis of the data from those studies demonstrates that the optimal network structure for eliciting the wisdom of the crowd depends on the estimation context-the specific population of individuals faced with a specific estimation task (Almaatouq et al, 2022). Specifically, that work shows that when a group's initial estimates are highly skewed or heavy-tailed, a centralized network structure can promote collective accuracy, whereas decentralized network structures might hinder collective accuracy in such contexts (and vice versa).…”

“…However, instead of having each agent integrate evidence (represented as samples from a Bernoulli distribution) via Bayes' theorem to establish their initial estimate, we assign each agent an initial estimate by sampling from a compilation of empirical data from four previously published experiments (Becker et al, 2017(Becker et al, , 2019Gürçay et al, 2015;Lorenz et al, 2011). This compiled dataset spans a total of 54 estimation tasks on which 2,885 individuals provided independent estimates (Almaatouq et al, 2022). Each task-or "estimation context"-in this dataset is represented by a distribution of independent estimates and a true value.…”

Experiment commensurability does not necessitate research consolidation

“…Developing research cartography will help identify research gaps, established findings, and controversial problems. The approach will also aid the reuse and reanalysis of existing data to answer new research questions (Almaatouq, Rahimian, Burton, & Alhajri, 2022; Rand, Greene, & Nowak, 2012; Tsvetkova, Wagner, & Mao, 2018). In short, whether retrospective or prospective, a comprehensive and systematic research cartography will help consolidate knowledge and stimulate new research.…”