Countering Misinformation on Social Media Through Educational Interventions: Evidence from a Randomized Experiment in Pakistan

Abstract

Fake news is a growing problem in developing countries with potentially far-reaching consequences. We conduct a randomized experiment in urban Pakistan to evaluate the effectiveness of two educational interventions to counter misinformation among low-digital literacy populations. We do not find a significant effect of video-based general educational messages about misinformation. However, when such messages are augmented with personalized feedback based on individuals' past engagement with fake news, we find an improvement of 0.14 standard deviations in identifying fake news. We also find negative but insignificant effects on identifying true news, driven by female respondents. Our results suggest that educational interventions can enable information discernment but their effectiveness critically depends on how well their features and delivery are customized for the population of interest.

Digital Literacy and Vulnerability to Misinformation

“…We find that less digitally literate users are more likely to express non-neutral emotional reactions to news. This is consistent with a digital media literacy intervention, which reduced non-neutral emotional reactions to news headlines in a randomized experiment in Pakistan (Ali and Qazi, 2021). However, we did not find a statistically significant association between digital literacy and confirmation bias, that is, whether people consider a headline to be true or false due to its alignment with their prior beliefs.…”

“…Badrinathan (2021) found no effect of an in-person media literacy training on truth discernment in a low digital literacy sample in India. Ali and Qazi (2021) found that video-based messages aimed at teaching people how to spot fake news augmented with personalized feedback improved accuracy of beliefs about fake news among low digital literacy users in Pakistan.…”

“…Our partner for field data collection was the survey wing of the Institute of Development Alternatives (IDEAS). Our field survey was designed to include a separate experiment for evaluating whether a digital media literacy intervention affects how people evaluate true and false news headlines(Ali and Qazi, 2021). The results of the experiment are examined in a separate study.…”

Countering Misinformation: Evidence, Knowledge Gaps, and Implications of Current Interventions

“…Alternative data collection methods such as phone surveys are much more difficult to conduct, and can be expensive. Importantly, what works in the West may not work elsewhere: interventions that were shown to be effective in Western countries did not yield the same results in countries such as India (Badrinathan, 2021;Guess et al, 2020) and Pakistan (Ali & Qazi, 2021). As Ali and Qazi (2021) note: the effectiveness of interventions "critically depends on how well their features and delivery are customised for the population of interest".…”

“…Importantly, what works in the West may not work elsewhere: interventions that were shown to be effective in Western countries did not yield the same results in countries such as India (Badrinathan, 2021;Guess et al, 2020) and Pakistan (Ali & Qazi, 2021). As Ali and Qazi (2021) note: the effectiveness of interventions "critically depends on how well their features and delivery are customised for the population of interest". Similarly, automated solutions may work well in English, but are typically not highly advanced in other languages.…”