Mobile applications (apps) to improve health are proliferating, but before healthcare providers or organizations can recommend an app to the patients they serve, they need to be confident the app will be user-friendly and helpful for the target disease or behavior. This paper summarizes seven strategies for evaluating and selecting health-related apps: (1) Review the scientific literature, (2) Search app clearinghouse websites, (3) Search app stores, (4) Review app descriptions, user ratings, and reviews, (5) Conduct a social media query within professional and, if available, patient networks, (6) Pilot the apps, and (7) Elicit feedback from patients. The paper concludes with an illustrative case example. Because of the enormous range of quality among apps, strategies for evaluating them will be necessary for adoption to occur in a way that aligns with core values in healthcare, such as the Hippocratic principles of nonmaleficence and beneficence.

No abstract

Commercial mobile apps for health behavior change are flourishing in the marketplace, but little evidence exists to support their use. This paper summarizes methods for evaluating the content, usability, and efficacy of commercially available health apps. Content analyses can be used to compare app features with clinical guidelines, evidence-based protocols, and behavior change techniques. Usability testing can establish how well an app functions and serves its intended purpose for a target population. Observational studies can explore the association between use and clinical and behavioral outcomes. Finally, efficacy testing can establish whether a commercial app impacts an outcome of interest via a variety of study designs, including randomized trials, multiphase optimization studies, and N-of-1 studies. Evidence in all these forms would increase adoption of commercial apps in clinical practice, inform the development of the next generation of apps, and ultimately increase the impact of commercial apps.

BackgroundEffective web-assisted tobacco interventions (WATIs) have been underutilized by smokers; moreover, despite practice guideline recommendations, clinical teams do not routinely refer smokers to WATIs. Our goal was to test a clinical practice innovation, an ePortal designed to change practice and patient behavior. Our hypotheses were that the integrated system would result in increased smoker referrals, with an automated follow-up system resulting in more smoker registrations and finally augmentations of the WATI would result in more smokers quitting at 6 months.MethodsPractice ePortal Implementation Trial: Practices (n = 174) were randomized to an online practice ePortal with an “e-referral tool” to the WATI (e-referred smokers received automated email reminders from the practice) and with practice feedback reports with patient tracking and practice-to-patient secure messaging versus comparison (a paper “referral prescription”). Implementation success was measured by the number of smokers referred and smokers registering.Clinical Effectiveness Trial: To estimate the effectiveness of the WATI components on 6-month smoking cessation, registered smokers were randomized into three groups: a state-of-the-art tailored WATI control [control], the WATI enhanced with proactive, pushed tailored email motivational messaging (messaging), and the WATI with messaging further enhanced with personal secure messaging with a tobacco treatment specialist and an online support group (personalized).ResultsPractice ePortal Trial results: A total of 4789 smokers were referred. The mean smokers referred per practice was not statistically different by group (ePortal 24.89 (SD 22.29) versus comparison 30.15 (SD 25.45), p = 0.15). The e-referral portal implementation program resulted in nearly triple the rate of smoker registration (31 % of all smokers referred registered online) versus comparison (11 %, p < 0.001).Clinical Effectiveness Trial results: Active smokers randomized to the personalized group had a 6-month cessation rate of 25.2 %, compared with the messaging group (26.7 %) and the control (17 %). Next, when using an inverse probability weighted selection model to account for attrition, those randomized to the two groups that received motivational messaging (messaging or personalized) were more likely to quit than those in the control (p = 0.04).ConclusionsAmong all smokers referred, the e-referral resulted in nearly threefold greater registrants (31 %) than paper (11 %). The practice ePortal smokers received multiple reminders (increasing registration opportunities), and the practices could track patient progress. The result was more smokers registering and, thus, more cessation opportunities. Combining the proactive referral and the WATI resulted in higher rates of smoking cessation.Trial RegistrationWeb-delivered Provider Intervention for Tobacco Control (QUIT-PRIMO) - a randomized controlled trial: NCT00797628.Electronic supplementary materialThe online version of this article (doi:10.1186/s13012-015-0336-8) contains...

BackgroundOutside health care, content tailoring is driven algorithmically using machine learning compared to the rule-based approach used in current implementations of computer-tailored health communication (CTHC) systems. A special class of machine learning systems (“recommender systems”) are used to select messages by combining the collective intelligence of their users (ie, the observed and inferred preferences of users as they interact with the system) and their user profiles. However, this approach has not been adequately tested for CTHC.ObjectiveOur aim was to compare, in a randomized experiment, a standard, evidence-based, rule-based CTHC (standard CTHC) to a novel machine learning CTHC: Patient Experience Recommender System for Persuasive Communication Tailoring (PERSPeCT). We hypothesized that PERSPeCT will select messages of higher influence than our standard CTHC system. This standard CTHC was proven effective in motivating smoking cessation in a prior randomized trial of 900 smokers (OR 1.70, 95% CI 1.03-2.81).MethodsPERSPeCT is an innovative hybrid machine learning recommender system that selects and sends motivational messages using algorithms that learn from message ratings from 846 previous participants (explicit feedback), and the prior explicit ratings of each individual participant. Current smokers (N=120) aged 18 years or older, English speaking, with Internet access were eligible to participate. These smokers were randomized to receive either PERSPeCT (intervention, n=74) or standard CTHC tailored messages (n=46). The study was conducted between October 2014 and January 2015. By randomization, we compared daily message ratings (mean of smoker ratings each day). At 30 days, we assessed the intervention’s perceived influence, 30-day cessation, and changes in readiness to quit from baseline.ResultsThe proportion of days when smokers agreed/strongly agreed (daily rating ≥4) that the messages influenced them to quit was significantly higher for PERSPeCT (73%, 23/30) than standard CTHC (44%, 14/30, P=.02). Among less educated smokers (n=49), this difference was even more pronounced for days strongly agree (intervention: 77%, 23/30; comparison: 23%, 7/30, P<.001). There was no significant difference in the frequency which PERSPeCT randomized smokers agreed or strongly agreed that the intervention influenced them to quit smoking (P=.07) and use nicotine replacement therapy (P=.09). Among those who completed follow-up, 36% (20/55) of PERSPeCT smokers and 32% (11/34) of the standard CTHC group stopped smoking for one day or longer (P=.70).ConclusionsCompared to standard CTHC with proven effectiveness, PERSPeCT outperformed in terms of influence ratings and resulted in similar cessation rates.ClinicalTrialClinicaltrials.gov NCT02200432; https://clinicaltrials.gov/ct2/show/NCT02200432 (Archived by WebCite at http://www.webcitation.org/6lEJY1KEd)

BackgroundWhat is the next frontier for computer-tailored health communication (CTHC) research? In current CTHC systems, study designers who have expertise in behavioral theory and mapping theory into CTHC systems select the variables and develop the rules that specify how the content should be tailored, based on their knowledge of the targeted population, the literature, and health behavior theories. In collective-intelligence recommender systems (hereafter recommender systems) used by Web 2.0 companies (eg, Netflix and Amazon), machine learning algorithms combine user profiles and continuous feedback ratings of content (from themselves and other users) to empirically tailor content. Augmenting current theory-based CTHC with empirical recommender systems could be evaluated as the next frontier for CTHC.ObjectiveThe objective of our study was to uncover barriers and challenges to using recommender systems in health promotion.MethodsWe conducted a focused literature review, interviewed subject experts (n=8), and synthesized the results.ResultsWe describe (1) limitations of current CTHC systems, (2) advantages of incorporating recommender systems to move CTHC forward, and (3) challenges to incorporating recommender systems into CTHC. Based on the evidence presented, we propose a future research agenda for CTHC systems.ConclusionsWe promote discussion of ways to move CTHC into the 21st century by incorporation of recommender systems.

BackgroundAlthough screening for tobacco use is increasing with electronic health records and standard protocols, other tobacco-control activities, such as referral of patients to cessation resources, is quite low. In the QUIT-PRIMO study, an online referral portal will allow providers to enter smokers' email addresses into the system. Upon returning home, the smokers will receive automated emails providing education about tobacco cessation and encouragement to use the patient smoking cessation website (with interactive tools, educational resources, motivational email messages, secure messaging with a tobacco treatment specialist, and online support group).MethodsThe informatics system will be evaluated in a comparative effectiveness trial of 160 community-based primary care practices, cluster-randomized at the practice level. In the QUIT-PRIMO intervention, patients will be provided a paper information-prescription referral and then "e-referred" to the system. In the comparison group, patients will receive only the paper-based information-prescription referral with the website address. Once patients go to the website, they are subsequently randomized within practices to either a standard patient smoking cessation website or an augmented version with access to a tobacco treatment specialist online, motivational emails, and an online support group. We will compare intervention and control practice participation (referral rates) and patient participation (proportion referred who go to the website). We will then compare the effectiveness of the standard and augmented patient websites.DiscussionOur goal is to evaluate an integrated informatics solution to increase access to web-delivered smoking cessation support. We will analyze the impact of this integrated system in terms of process (provider e-referral and patient login) and patient outcomes (six-month smoking cessation).Trial RegistrationWeb-delivered Provider Intervention for Tobacco Control (QUIT-PRIMO) - a randomized controlled trial: NCT00797628.

Background Tailored, web-assisted interventions can reach many smokers. Content from other smokers (peers) through crowdsourcing could enhance relevance. Purpose To evaluate whether peers can generate tailored messages encouraging other smokers to use a web-assisted tobacco intervention (Decide2Quit.org). Methods Phase 1: In 2009, smokers wrote messages in response to scenarios for peer advice. These smoker-to-smoker (S2S) messages were coded to identify themes. Phase 2: resulting S2S messages, and comparison expert messages, were then emailed to newly registered smokers. In 2012, subsequent Decide2Quit.org visits following S2S or expert-written e-mails were compared. Results Phase 1: a total of 39 smokers produced 2886 messages (message themes: attitudes and expectations, improvements in quality of life, seeking help, and behavioral strategies). For not-ready-to-quit scenarios, S2S messages focused more on expectations around a quit attempt and how quitting would change an individual’s quality of life. In contrast, for ready-to-quit scenarios, S2S messages focused on behavioral strategies for quitting. Phase 2: In multivariable analysis, S2S messages were more likely to generate a return visit (OR=2.03, 95% CI=1.74, 2.35), compared to expert messages. A significant effect modification of this association was found, by time-from-registration and message codes (both interaction terms p<0.01). In stratified analyses, S2S codes that were related more to “social” and “real-life” aspects of smoking were driving the main association of S2S and increased return visits. Conclusions S2S peer messages that increased longitudinal engagement in a web-assisted tobacco intervention were successfully collected and delivered. S2S messages expanded beyond the biomedical model to enhance relevance of messages.