Getting to the source of the illusion of consensus

Abstract

Consensus between informants is a valuable cue to a claim's epistemic value, when informants' beliefs are developed independently of each other. Recent work (Yousif et al., 2019) described an illusion of consensus such that people did not generally discriminate between the epistemic warrant of true consensus, where a majority claim is supported by multiple independent sources, and false consensus arising from a single source's repeated claim. Four experiments tested a novel account of the illusion of consensus; that it arises when people are unsure about the independence of the primary sources on which informant claims are based. When this independence relationship was ambiguous we found evidence for the illusion. However, when steps were taken to highlight the independence between data sources in the true consensus conditions, and confidence in a claim was measured against a no consensus baseline (where there was an equal number of reports supporting and opposing a claim), we eliminated the illusion of consensus. Under these conditions, more weight was given to claims based on true consensus than false consensus. These findings show that although the illusion of consensus is prevalent, people do have the capacity to distinguish between true and false consensus.

Trust and Reliance in Consensus-Based Explanations from an Anti-Misinformation Agent

“…Agreement with information when there is true consensus tends to be higher than when there is false or no consensus [4,36]. Still, if the trust model for automation [11,13] applies to XAI systems, then we would expect reliance behavior to be mediated by trust.…”

“…Second, how do we distinguish those who are simply skeptical of AI? And third, if the illusion occurs, can it be decreased by emphasizing the independence of sources via the design of the explanations, as suggested by work outside of social media [4]? To the best of our knowledge, this has not yet been explored.…”

“…A pressing question is what forms of explanations are effective at enabling lay end-users to trust XAI systems [2,30,33,34]. In social media platforms, lay people make judgments and decisions based on evaluating and integrating reports from multiple informants [4,31]. One factor that affects lay people's confidence in such reports is the degree of consensus across related reports [4].…”

“…In social media platforms, lay people make judgments and decisions based on evaluating and integrating reports from multiple informants [4,31]. One factor that affects lay people's confidence in such reports is the degree of consensus across related reports [4]. In social media, people may desire explanations that rely on consensus-based data sources.…”

“…Placing these sources within explanations provided by anti-misinformation XAI may increase users' confidence in the system's fact-checking and possibly reliance on it. Yet, this poses a new problem: an illusion of consensus effect [4,36], where people are unable to distinguish "true" consensus, i.e., different informants relying on different sources but drawing the same conclusion, and "false" consensus, i.e., different informants relying on the same source. This raises three questions at the intersection of user-centred design and research methods.…”