Using cross-domain expertise to aggregate forecasts when within-domain expertise is unknown.

Abstract

In recent years, a number of crowd aggregation approaches have been proposed to combine the judgments of different individuals in problems where decision-makers do not have records of the individuals’ past performance in that domain. However, it is often possible to obtain a measure of the individuals’ past performance in other domains. The current article explores the extent to which individuals’ relative expertise in one domain can be used to weight their judgments in another domain. Over three experiments comprising a range of decision problems from art, science, sport, and a test of emotional intelligence, we compare the performance of aggregation approaches that do not use individuals’ past performance to those that weight by individuals’ past performance on questions from the same domain (within-domain weighting) or from a different domain (cross-domain weighting). Our results show that although within-domain weighting generally outperforms all other aggregation approaches, cross-domain weighting can be as effective as within-domain weighting in some circumstances. We present a simple model of the relationship between within-domain and cross-domain performance and discuss the conditions under which cross-domain weighting is likely to be effective. Our results demonstrate the potential of cross-domain weighting in problems where records of individuals’ past performance in the domain of interest are unavailable.

Introduction to the special issue on judgment and decision research on the wisdom of the crowds.

“…The articles in this special issue on the wisdom of the crowds contribute to outstanding questions regarding what to aggregate (Feng & Budescu, 2024; Hasan et al, 2024; Summerville et al, 2024), optimal weighting schemes (Huang et al, 2024; Collins et al, 2024; Powell et al, 2024), incentives (Peker, 2024), establishing expertise (Howe et al, 2024), social processes (Beauchamp et al, 2024; Mayer & Heck, 2024), and beliefs about the efficacy of crowd wisdom (Schultze et al, 2024). One way to group the articles relates to an overarching question: To what extent should aggregation be left to the crowd members themselves?…”