Assessing the Feasibility of Studying Awareness of a Digital Health Campaign on Facebook: Pilot Study Comparing Young Adult Subsamples

Abstract

Background Mass media campaigns for preventive health messaging have been shown to be effective through years of research. However, few studies have assessed the effectiveness of campaigns on digital media, which is currently how youths and young adults are primarily consuming media. In particular, a platform that can accurately assess exposure to digital messaging in a real-life setting has yet to be developed. Objective This study examines the feasibility of a unique survey platform, Virtual Lab, to conduct a study on exposure to a media campaign within Facebook using a chatbot-style survey administration technique. Methods Virtual Lab is a survey platform that was used to recruit and survey participants within Facebook and Facebook Messenger, respectively. We created a Facebook business account with 2 Facebook pages: one for recruitment and disseminating the survey and the other one for serving the target advertisements. Pre- and postexposure surveys were administered via Facebook Messenger using a chatbot-style questionnaire 1 week apart. During this time, the target advertisements were shown to participants who completed the pre-exposure survey. The total time from recruitment to completion of the postexposure survey was 13 days, and incentive costs were US $10 per participant. Survey data were compared between those who completed both pre- and postexposure surveys and those who only completed the pre-exposure survey; that is, those who were lost to follow-up. The demographics of the complete cases were also compared to the US census data. Results A total of 375 Facebook users aged between 18 and 24 years met eligibility requirements and consented to the study, which consisted of complete cases (n=234) and participants lost to follow-up (n=141). A few differences between complete cases and participants lost to follow-up were observed. Regarding gender, complete cases comprised 40.2% males and 59.4% females, and among participants lost to follow-up, 44.0% were male and 50.4% were female (P=.003). Differences were also observed for e-cigarette use status, where a greater number of current users and fewer past and never users were lost to follow-up than complete cases (P=.01). Conclusions The use of Virtual Lab yielded a diverse sample quickly and cost-effectively. Demographic characteristics of participants who completed the study and those who were lost to follow-up were similar, indicating that no biases were caused by the platform during recruitment or testing. This study suggests the feasibility of the Virtual Lab survey platform for studies of media campaign exposure within Facebook. This platform can advance health campaign research by providing more accurate data to inform digital messaging.

Effects of a Social Media Intervention on Vaping Intentions: Randomized Dose-Response Experiment

“…Cell sizes between the study conditions were reduced at the second FU, which may have reduced the statistical power to below the levels needed to detect some dose-response outcomes of interest. Previous studies have shown the dose-response effects of anti-industry messaging on vaping-related content exposure and attitudinal outcomes [ 14 ].…”

“…Thus, there is a great need for more research on the relationship between digital media and health behaviors, social norms, and social networks. Although research is being conducted to determine what digital media as an intervention tool would look like, how it works, and how effective it is [ 13 , 14 ], these studies have only scratched the surface [ 15 ].…”

“…Furthermore, this study builds on recent studies using a social media–based data collection platform for random assignment studies [ 14 ]. The use of social media recruitment, chatbots for survey delivery, and retargeting technology for intervention delivery and follow-up (FU) have been proven feasible and produce short-term effects on content (eg, advertising) exposure.…”

Effects of a Social Media Intervention on Vaping Intentions: Randomized Dose-Response Experiment (Preprint)

“…Thus, there is a great need for more research on the relationship between digital media and health behaviors, social norms, and social networks. While research is being conducted to determine what digital media as an intervention tool would look like, how it works, and how effective it is [13][14], these studies have only scratched the surface [15].…”

Evaluation of response to incentive recruitment strategies in a social media-based survey

“…One important objective of future research using social media platforms is to determine how to optimize study designs and responses to recruitment and intervention methods. 29 …”