"Fact-checking" videos reduce belief in misinformation and improve the quality of news shared on Twitter

Abstract

"Fake news" are widely acknowledged as an important challenge for Western democracies. Yet, surprisingly little effort has been devoted to measuring the effects of various counter-strategies. We address this void by running a pre-registered field experiment analyzing the causal effects of popular fact-checking videos on both believing and sharing fake news among Twitter users (N = 1,600). We find that the videos improve truth discernment ability as measured by performance in a fake news quiz immediately after exposure. However, the videos have not reduced sharing links from verified "fake news" websites on Twitter in the weeks following the exposure. Indeed, we find no relationship between truth discernment ability and fake news sharing. These results imply that the development of effective interventions should be based on a nuanced view of the distinct psychological motivations of sharing and believing "fake news".

Cash versus Lotteries: COVID-19 Vaccine Incentives Experiment*

“…We use the Qualtrics Web Service feature to run this co-variate adaptive randomization remotely using an API. 6 Balance. Table 1 in the text summarized the covariate distributions within each of the three treatment groups.…”

Review of social science research on the impact of countermeasures against influence operations

“…Nassetta & Gross (2020) found that YouTube's media label became more effective by simply changing the color. Bor et al (2020) show that while exposure to fact-checking videos "improved participants' ability to assess the credibility of news story headlines," they continued to share "false and untrustworthy news sources on Twitter." The importance of the medium also applied to disinformation itself: Hameleers et al (2020) found that "multimodal disinformation was perceived as slightly more credible than textual disinformation" and that "the presence of textual and visual fact checkers resulted in lower levels of credibility."…”

“…Nassetta & Gross (2020) evaluate the impact of placing state-sponsored media labels and disclaimers below RT YouTube videos on participant perceptions. Bor et al (2020) measured how much fake news and disinformation was shared on the Twitter feeds of users after exposure to fact-checking credibility; whether initial priming about evaluating the accuracy of information impacted user ability to better discern disinformation; if legislators were less likely to produce disinformation in their public statements when presented with information about the reputational risks of negative fact-checking ratings; how differences in judgement processes impact whether individuals will update false beliefs after being presented with corrective information, differences in effectiveness of various accuracy prompt intervention approaches; the difference in effectiveness of crowdsourced versus academic corrections to false information; the effectiveness of audience-targeted communication strategies for disinformation disclosure; and the roles of inattention emotionality and deliberation in influencing accurate identification of disinformation. information videos.…”

“…Nyhan et al (2019) and Swire-Thompson et al (2020), for example, both find that while fact-checks of politicians' false claims successfully reduce beliefs in the claims, they do not impact support for the politician. With respect to intended sharing behavior, Bor et al (2020) and Pennycook et al (2020) found that identification of fake news may not prevent sharing on Twitter. Such outcomes are significant in designing effective factchecking interventions.…”

“…Further, literature has done little to study the platforms' major actions. Only one study directly measures the impact of platform interventions on real-time social media behavior (Bor et al, 2020) and few studies (2%) sought to measure the impact of interventions on online behavior broadly. This was achieved by having participants disclose their Twitter usernames and asking that their feeds remain public in order for the research team to engage in data scraping over a period of one year.…”