Advantages and challenges in using mobile apps for field experiments: A systematic review and a case study

Abstract

As smartphone’s computing power continues to grow and as mobile applications (apps) continue to dominate digital engagement, apps have become a new frontier for advancing field experiment methodology. Using apps may help researchers to scale up the reach, precisely control randomization and experiment materials, collect a variety of objective and self-reported data over time, and more conveniently replicate and adapt an experiment. We performed a systematic review on field experiments involving apps published between 2007 and 2017. Seven databases were scanned using a predefined search strategy. The database search retrieved 4,810 citations; 101 articles met the inclusion criteria. Our review suggests that scholars have only started to employ apps in field experiments in the last 4 years. Most studies only used apps as an experiment treatment instead of an experiment platform; therefore, researchers have yet to fully leverage the advantages. Almost all studies were from the health research domain and 77.2% used randomized controlled trial design. Only 7 studies utilized smartphone sensors for collecting data. Only one study reported cost and ethical concerns regarding using apps for the experiment. Given these findings, we reported a case study that targeted a minority racial group and leveraged the advantages of apps as an experiment platform and as a data collection tool to illustrate practical challenges and lessons learned regarding time, financial cost, and technical support. In conclusion, we suggest apps provide new ways to study causal mechanisms with experiment big data. Limitations of generalizability, retention, and design quality were discussed as well.

Artificial Intelligence Chatbot Behavior Change Model for Designing Artificial Intelligence Chatbots to Promote Physical Activity and a Healthy Diet: Viewpoint

“…Lifestyle modification programs have consistently evolved with emerging digital and communication technologies [ 8 - 13 ]. In the past two decades, there has been a large number of published studies using internet and mobile-based behavior interventions to support the effectiveness of using digital technologies to deliver intervention materials to diverse populations [ 8 , 14 ].…”

The state of wearable health technologies: a transdisciplinary literature review

“…To guide this study effectively, we derived explicit criteria from our previous work (Chib & Lin, 2018;Chib et al, 2015) on mHealth reviews. In order to avoid bias and error, the selected material was checked by a second co-author, similar to methods applied by Zhang, Calabrese et al (2018).…”

Mobile methods: Explorations, innovations, and reflections

“…In “Advantages and Challenges in Using Mobile Apps for Field Experiments: A Systematic Review and a Case Study,” Jingwen Zhang and colleagues (2018) perform a systematic review of 101 articles involving field experiments that use mobile apps. In their analysis they found that most of these experiments only used the apps to administer treatment materials, rather than collect data directly from respondents.…”