Commencement of and Retention in Web-Based Interventions and Response to Prompts and Reminders: Longitudinal Observational Study Based on Two Randomized Controlled Trials

Abstract

Background Web-based interventions are effective for several psychological problems. However, recruitment, adherence, and missing data are challenges when evaluating these interventions. Objective This study aimed to describe the use patterns during the commencement phase, possible retention patterns (continuation of data provision), and responses to prompts and reminders among participants in 2 randomized controlled trials (RCTs) evaluating web-based interventions. Methods Data on use patterns logged in 2 RCTs aiming to reduce symptoms of anxiety and depression among adult patients recently diagnosed with cancer (AdultCan RCT) and patients with a recent myocardial infarction (Heart RCT) were analyzed. The web-based intervention in the AdultCan trial consisted of unguided self-help and psychoeducation and that in the Heart trial consisted of therapist-supported cognitive behavioral therapy. In total, 2360 participants’ use patterns at first log-in, including data collection at baseline (ie, commencement) and at 2 follow-ups, were analyzed. Both the intervention and comparison groups were analyzed. Results At commencement, 70.85% (909/1283) and 86.82% (935/1077) of the participants in AdultCan and Heart RCTs, respectively, logged in and completed baseline data collection after receiving a welcome email with log-in credentials. The median duration of the first log-in was 44 minutes and 38 minutes in AdultCan and Heart RCTs, respectively. Slightly less than half of the participants’ first log-ins were completed outside standard office hours. More than 80% (92/114 and 103/111) of the participants in both trials explored the intervention within 2 weeks of being randomized to the treatment group, with a median duration of 7 minutes and 47 minutes in AdultCan and Heart RCTs, respectively. There was a significant association between intervention exploration time during the first 2 weeks and retention in the Heart trial but not in the AdultCan trial. However, the control group was most likely to retain and provide complete follow-up data. Across the 3 time points of data collection explored in this study, the proportion of participants responding to all questionnaires within 1 week from the prompt, without a reminder, varied between 35.45% (413/1165) and 66.3% (112/169). After 2 reminders, up to 97.6% (165/169) of the participants responded. Conclusions Most participants in both RCTs completed the baseline questionnaires within 1 week of receiving the welcome email. Approximately half of them answered questions at baseline data collection outside office hours, suggesting that the time flexibility inherent in web-based interventions contributes to commencement and use. In contrast to what was expected, the intervention groups generally had lower completion rates than the comparison groups. About half of the participants completed the questionnaires without a reminder, but thereafter, reminders contributed to both baseline and follow-up retention, suggesting they were effective. Strategies to increase commencement of and retention in eHealth interventions are important for the future development of effective interventions and relevant research.

A Web-Based Sexual Violence, Alcohol Misuse, and Bystander Intervention Program for College Women (RealConsent): Randomized Controlled Trial

“…Although participants agreed to complete the program modules as part of this RCT, completion was not mandatory, and all participants had the option to drop out, withdraw, or, if so inclined, simply not view the content while it was running. In viewing RealConsent completion rates, our results show that 75.5% (335/444) of the participants completed all 4 modules of the program, which can be viewed as a high percentage of completion for a web-based program [ 67 ] and which suggests perceived relevance and quality. Further research regarding the acceptability of RealConsent and its relevance and perceived quality in terms of entertainment value is warranted.…”

“…Second, although it was not extreme, we experienced some loss to follow-up in terms of completion of the respective web-based programs and follow-up survey. It is unclear what the potential reasons were for this loss to follow-up; however, previous research has shown that attrition in web-based trials may be higher than that in in-person trials [ 67 , 69 , 70 ]. Nevertheless, our observed attrition rates (overall 161/881, 18.3%) are lower than (eg, 40% [ 71 ]), or in line with (eg, 18% [ 72 ]), other in-person SV risk reduction trials with similar follow-up periods.…”

Examining outcomes for service users accessing the Breaking Free Online computer-assisted therapy program for substance use disorders via a ‘telehealth’ approach: protocol for a two arm, parallel group randomized controlled trial

“…Although the fortnightly recovery check-in phone calls are intended to enhance retention of participants in the BFO arm of the study, it may be more challenging to retain participants in the study once the treatment period is over and they are no longer receiving the check-in phone calls—a prediction supported by previous systematic reviews into smoking cessation interventions [ 51 ]. Every attempt will be made to obtain follow-up data—as follow-up data is being collected via electronic methods, there is scope to send reminders to participants to complete follow-up assessments via email, a method indicated to be successful in increasing follow-up retention in previously conducted RCTs [ 52 ].…”

Feasibility of Digital Memory Assessments in an Unsupervised and Remote Study Setting

“…Interestingly, subjective task performance in the MDT-OS was rated lower and retention was shorter compared to ORR and CSR. This might suggest that specific feedback strategies to increase subjective task performance levels might help to increase long term retention ( 23 ). Of note, Pratap and colleagues report that retention times are in general higher in studies with personal contact between participants and study personnel.…”