Persuasion with Precision: Using Natural Language Processing to Improve Instrument Fidelity for Risk Communication Experimental Treatments

Abstract

Instrument fidelity in message testing research hinges upon how precisely messages operationalize treatment conditions. However, numerous message testing studies have unmitigated threats to validity and reliability because no established procedures exist to guide construction of message treatments. Their construction typically occurs in a black box, resulting in suspect inferential conclusions about treatment effects. Because, a mixed methods approach is needed to enhance instrument fidelity in message testing research, this article contributes to the field of mixed methods research by presenting an integrated multistage procedure for constructing precise message treatments using an exploratory sequential mixed methods design. This work harnesses the power of integration through crossover analysis to improve instrument fidelity in message testing research through the use of natural language processing (NLP).

In This Issue: Tribute to Michael Fetters, Queer Approaches to Mixed Methods, Mixed Methods Community-Engaged Evaluation, Instrument Fidelity and Natural Language Processing, Teaching Mixed Methods, and Geographic Information System

“…In the second article, Reinhold et al (2013), with affiliations in land resources, agriculture, political science, computer science, ecosystems, and Earth sciences, contributed to the mixed methods literature in the topic of instrument fidelity by specifying how mixed methods can improve this instrument fidelity of treatment conditions in experimental message testing research. Message testing research examines the persuasive power of different kinds of messages, trying to understand how messages influence attitudes, beliefs, and behaviors.…”