Towards Stable Principles of Collective Intelligence under an Environment-Dependent Framework

Abstract

A large body of work has shown that a group of individuals can often achieve higher levels of intelligence than the group members working alone. Despite these expectations of group advantage, many examples of collective failure have been documented—from market crashes to the spread of false and harmful rumors. To reconcile these results, a major effort in the study of collective decision making has been focused on understanding the role of group composition and communication patterns in promoting the "wisdom of the crowd" or, conversely, leading to "madness of the mob." In the past decades, much of this effort has been devoted to inferring the importance of a particular attribute, in isolation, by its capacity to explain the accuracy of collective judgments. In this thesis, we argue that such a perspective can lead to inconsistent conclusions: an `incoherency problem.' We assert that the importance of an individual-level or structural attribute may change as a function of the environment in which the group is situated. Hence, we propose a research agenda to investigate the relative importance of the group composition and the structure of interaction networks under an environment-dependent framework. We show that under such a framework, we can reconcile previously conflicting claims from the collective intelligence literature and motivate a future research program to identify stable principles of collective performance. Although implementing such a program is logistically challenging, "virtual lab" experiments of the sort discussed in this thesis, in combination with emerging "open science" practices such as pre-registration, data availability, open code, and "many-labs" collaborations, offer a promising route forward.

Public participation in crisis policymaking. How 30,000 Dutch citizens advised their government on relaxing COVID-19 lockdown measures

“…When citizens deliberate, they can expand their knowledge, including both their own self-understanding and their collective understanding of what will best serve other affected groups [ 98 ]. Moreover, empirical studies show that individuals interacting with one another generally outperform groups of unconnected individuals [ 99 ]. Hence, enriching PVE experiments with deliberative elements (e.g., group discussion, consulting expert witnesses or a forum) may contribute to well-formed preferences in the case of unfamiliar and complex government policies and may even increase adherence to subsequent government measures [ 67 ].…”

Adaptive social networks promote the wisdom of crowds

“…Studies have found that these divergent effects are moderated by placing well-informed individuals in prominent positions in the network structure (21,22,29); those individuals' self-confidence (27,(30)(31)(32); the group's ability to identify experts (33); dispersion of skills (34)(35)(36); quality of information (25); diversity of judgments (36,37) or lack thereof (38); social learning strategies (39,40); and the structure of the task (35,40). In other words, whether social interaction is advantageous for the group depends on the environment in which the group is situated (41). Given that people often do not have access to all of the parameters of their environment (or the environment can change), it is advantageous to find an easy-to-implement mechanism that performs well across shifting environments.…”

“…Therefore, neither the emergent ("macroscopic") structure nor individual ("microscopic") properties can be decoupled from the characteristics of the environment ("context"). Hence, we hypothesize that dynamic social-influence networks guided by feedback may be central to context-adaptive collective intelligence (41), acting as core mechanisms by which groups, which may not initially be wise, evolve wisdom, adapting to biased and potentially nonstationary information environments.…”

Beyond Playing 20 Questions with Nature: Integrative Experiment Design in the Social and Behavioral Sciences

“…For any substantive area of the social and behavioral sciences on which we have undertaken a significant amount of reading, we see hundreds of experiments that each test the effects of some independent variables on other dependent variables while suppressing innumerable "aspects of the situation." 1 Setting aside the much-discussed problems of replicability and reproducibility, many of these papers are interesting when read in isolation, but it is no more possible to "put them all together" today than it was in Newell's time (Almaatouq, 2019;Muthukrishna & Henrich, 2019;D. J. Watts, 2017).…”