Effects of inductive learning and gamification on news veracity discernment.

Abstract

This preregistered study tests a novel psychological intervention to improve news veracity discernment. The main intervention involved inductive learning (IL) training (i.e., practice discriminating between multiple true and fake news exemplars with feedback) with or without gamification. Participants (N = 282 Prolific users) were randomly assigned to either a gamified IL intervention, a nongamified version of the same IL intervention, a no-treatment control group, or a Bad News intervention, a notable web-based game designed to tackle online misinformation. Following the intervention (if applicable), all participants rated the veracity of a novel set of news headlines. We hypothesized that the gamified intervention would be the most effective for improving news veracity discernment, followed by its nongamified equivalent, then Bad News, and finally the control group. The results were analyzed with receiver-operating characteristic curve analyses, which have previously never been applied to news veracity discernment. The analyses indicated that there were no significant differences between conditions and the Bayes factor indicated very strong evidence for the null. This finding raises questions about the effectiveness of current psychological interventions and contradicts prior research that has supported the efficacy of Bad News. Age, gender, and political leaning all predicted news veracity discernment.

Gamified inoculation interventions do not improve discrimination between true and fake news: Reanalyzing existing research with receiver operating characteristic analysis.

“…Therefore, we expect our results to generalize to situations in which participants rate similar item sets. Indeed, when participants were tested on a larger set of true and fake news items that had been posted on the internet in the past, Bad News had no effect at all; it did not improve news veracity discernment nor elicit more conservative responding (Modirrousta-Galian et al, in press). Finally, we do not have evidence that our findings will generalize to gamified fake news interventions other than Bad News and Go Viral!.…”

“…If the gamified intervention has more general effects, this will result in an effect on response bias because both the FARs and HRs would decrease. Despite this, to the best of our knowledge, ROC analysis has only been used once before with research on fake news (Modirrousta-Galian et al, in press). Instead, most studies have analyzed mean ratings, which are not ideally suited for separating discrimination and response bias (more on this later).…”

“…ROC analysis is ideally suited for determining the generality of interventions, as it allows for discrimination (ability to distinguish between true and fake news) and response bias (tendency to rate news items as true or fake regardless of their objective veracity) to be measured separately. Despite its usefulness, ROC analysis has scarcely been used in online misinformation research (although see Modirrousta-Galian et al, in press). Consequently, we filled this gap in the literature by using ROC analysis to reanalyze data from published papers on gamified inoculation interventions, which allowed us to determine their generality and thus effectiveness.…”

“…Indeed, Batailler et al recommended using ROC analysis in future research. To the best of our knowledge, ROC analysis has only been used once before with research on fake news (Modirrousta-Galian et al, in press). Instead, most studies have analyzed mean belief ratings in true and fake news, which are not ideally suited for separating discrimination and response bias.…”

“…As noted earlier, despite its usefulness for assessing the specificity of an intervention's effect, we are aware of only one paper that has analyzed data from gamified fake news interventions using ROC analysis (Modirrousta-Galian et al, in press). Indeed, in some prior research, there is seemingly a lack of concern over the generality of the intervention's effect.…”