Influences of study design on the effectiveness of consensus messaging: The case of medicinal cannabis

Abstract

This study examines to what extent study design decisions influence the perceived efficacy of consensus messaging, using medicinal cannabis as the context. We find that researchers’ decisions about study design matter. A modified Solomon Group Design was used in which participants were either assigned to a group that had a pretest (within-subjects design) or a posttest only group (between-subjects design). Furthermore, participants were exposed to one of three messages—one of two consensus messages or a control message—attributed to the National Academies of Sciences, Engineering and Medicine. A consensus message describing a percent (97%) of agreeing scientists was more effective at shifting public attitudes than a consensus message citing substantial evidence, but this was only true in the between-subject comparisons. Participants tested before and after exposure to a message demonstrated pre-sensitization effects that undermined the goals of the messages. Our results identify these nuances to the effectiveness of scientific consensus messaging, while serving to reinforce the importance of study design.

Strong correlational but no causal evidence on the link between the perception of scientific consensus and support for vaccination

“…To examine whether the null effects of consensus messaging are a methodological artifact of repeatedly measuring attitudes, in Study 3, participants were exposed to key DVs before and after consensus messaging, or only after the consensus messaging. In contrast to evidence from Landrum et al 10 , we did not find evidence for sensitization effects: whether people had seen these questions or not beforehand did not seem to affect how they responded. As a result, it is unlikely that sensitization caused the consensus messaging to have null effect on vaccine attitudes and intentions.…”

“…We included the main and interactive effects of messaging (control vs. consensus) and design (pre-post vs. post-only) for each model. Statistically significant effects of the design and design x messaging interaction would suggest sensitization effects 10,42 ; especially problematic would be statistically significant interaction term as it would suggest that the effects of consensus messaging manipulation depends on whether participants responded to DV questions beforehand. However, no statistically significant effect of the design was found, nor of the interaction between study design and consensus messaging.…”

Strong correlational but no causal evidence on the link between the perception of scientific consensus and support for vaccination

“…We included the main and interactive effects of messaging (control vs. consensus) and design (pre-post vs. post-only) for each model. Statistically significant effects of the design and design x messaging interaction would suggest sensitization effects [ 10 , 61 ]; especially problematic would be statistically significant interaction term as it would suggest that the effects of consensus messaging manipulation depends on whether participants responded to DV questions beforehand. However, no statistically significant effect of the design was found, nor of the interaction between study design and consensus messaging for any of the DVs (i.e., perceived consensus, vaccine beliefs, worry, policy support, and vaccination intentions).…”

“…Attitudes toward a wide range of issues are related to how much consensus people think there is among experts. This includes controversial topics such as climate change [ 4 – 7 ], genetically-modified food [ 8 ], nuclear power [ 9 ], or medicinal cannabis [ 10 ]. Importantly, such a correlation has also been demonstrated in the context of infectious diseases [ 11 , 12 ].…”