Methodological Challenges in Online Trials

Murray, Elizabeth; Khadjesari, Zarnie; White, Ian R.; Kalaitzaki, Eleftheria; Godfrey, Christine; McCambridge, Jim; Thompson, Simon G.; Wallace, Paul

doi:10.2196/jmir.1052

Cited by 182 publications

(207 citation statements)

References 42 publications

Supporting

Mentioning

202

Contrasting

Unclassified

Order By: Relevance

“…In this review we scope and appraise available evidence on effectiveness and cost-effectiveness of IDIs for sexual health, 97 how best to design, develop and implement interventions 98 and how best to evaluate them. 99 We assess the future potential for IDIs for sexual health to guide strategic investment in research, development and implementation in the UK.…”

Section: Why This Review Now?mentioning

confidence: 99%

Sexual health promotion for young people delivered via digital media: a scoping review

et al. 2015

Self Cite

View full text Add to dashboard Cite

BackgroundYoung people are at risk of poor sexual health and are, therefore, in need of comprehensive, effective sexual health education. Young people are confident and constant users of digital technology, such as the internet and mobile phones, and there are many innovative possibilities for sexual health education involving these technologies.ObjectivesTo summarise evidence on effectiveness, cost-effectiveness and mechanism of action of interactive digital interventions (IDIs) for sexual health; optimal practice for intervention development; contexts for successful implementation; research methods for digital intervention evaluation; and the future potential of sexual health promotion via digital media.DesignLiterature review of evidence on digital interventions for sexual health for young people, integrating the findings with the views of young people, parents and experts in digital media/sexual health. IDIs are defined as digital media programmes that provide health information and tailored decision support, behaviour-change support and/or emotional support. We focus on sexual well-being for young people aged 13–24 years in the UK.ResultsThere are many imaginative IDIs for sexual health promotion, but few interventions address issues that are important to young people, such as sexual pleasure and relationships. It is vital to collaborate with young people and to use Behaviour-Change Theory in designing interventions. We located 19 randomised controlled trials of IDIs for sexual health promotion for young people, finding a moderate effect on sexual health knowledge [standardised mean difference (SMD) 0.54, 95% confidence interval (CI) 0.17 to 0.92], a small effect on confidence (self-efficacy) (SMD 0.11, 95% CI 0.02 to 0.20) and a positive effect on sexual behaviour (odds ratio 1.28, 95% CI 1.01 to 1.61), but no significant effects on safer sex intention or biological outcomes. One study suggests that IDIs may be as good as face-to-face interventions for sexual health knowledge and safer sex intention. There are no existing data on the cost-effectiveness of IDIs for sexual health promotion. The impact of an IDI will be determined by the proportion of the target population reached, intervention efficacy, adoption in a setting, how well it is delivered and maintenance/sustainability. All of these elements must be addressed for IDIs to be successful. More collaboration is needed to capitalise on the knowledge of users and stakeholders, the design and software skills of the commercial sector and the theoretical expertise and evaluation skills of academia.ConclusionsIDIs are effective for knowledge acquisition and sexual behaviour, and could usefully contribute to sexual health education in schools, in clinic settings and online; however, there are obstacles to overcome, such as access to information technology and ensuring the quality and safety of interventions.Future workMore evidence is needed on the best designs for interventions (e.g. choice of behaviour-change mechanisms and interactive features) and the best models of delivery (e.g. setting, modes of delivery, methods of facilitation and support for engagement) to improve sexual behaviour, biological outcomes and sexual well-being in a cost-effective way.FundingThe National Institute for Health Research Public Health Research programme.

show abstract

Section: Why This Review Now?mentioning

confidence: 99%

Sexual health promotion for young people delivered via digital media: a scoping review

et al. 2015

Self Cite

View full text Add to dashboard Cite

show abstract

“…by age, sex or sexuality) and content can be tailored for individuals. 14 IDIs can be expensive to develop, but offer the advantages of intervention fidelity 15 and the potential to reach large audiences at relatively low dissemination costs.…”

Section: Sexual Health Interventionsmentioning

confidence: 99%

The Men’s Safer Sex project: intervention development and feasibility randomised controlled trial of an interactive digital intervention to increase condom use in men

Bailey

Webster

Hunter

et al. 2016

Health Technol Assess

Self Cite

View full text Add to dashboard Cite

BackgroundThis report details the development of the Men’s Safer Sex website and the results of a feasibility randomised controlled trial (RCT), health economic assessment and qualitative evaluation.Objectives(1) Develop the Men’s Safer Sex website to address barriers to condom use; (2) determine the best design for an online RCT; (3) inform the methods for collecting and analysing health economic data; (4) assess the Sexual Quality of Life (SQoL) questionnaire and European Quality of Life-5 Dimensions, three-level version (EQ-5D-3L) to calculate quality-adjusted life-years (QALYs); and (5) explore clinic staff and men’s views of online research methodology.Methods(1) Website development: we combined evidence from research literature and the views of experts (n = 18) and male clinic users (n = 43); (2) feasibility RCT: 159 heterosexually active men were recruited from three sexual health clinics and were randomised by computer to the Men’s Safer Sex website plus usual care (n = 84) or usual clinic care only (n = 75). Men were invited to complete online questionnaires at 3, 6, 9 and 12 months, and sexually transmitted infection (STI) diagnoses were recorded from clinic notes at 12 months; (3) health economic evaluation: we investigated the impact of using different questionnaires to calculate utilities and QALYs (the EQ-5D-3L and SQoL questionnaire), and compared different methods to collect resource use; and (4) qualitative evaluation: thematic analysis of interviews with 11 male trial participants and nine clinic staff, as well as free-text comments from online outcome questionnaires.Results(1) Software errors and clinic Wi-Fi access presented significant challenges. Response rates for online questionnaires were poor but improved with larger vouchers (from 36% with £10 to 50% with £30). Clinical records were located for 94% of participants for STI diagnoses. There were no group differences in condomless sex with female partners [incidence rate ratio (IRR) 1.01, 95% confidence interval (CI) 0.52 to 1.96]. New STI diagnoses were recorded for 8.8% (7/80) of the intervention group and 13.0% (9/69) of the control group (IRR 0.75, 95% CI 0.29 to 1.89). (2) Health-care resource data were more complete using patient files than questionnaires. The probability that the intervention is cost-effective is sensitive to the source of data used and whether or not data on intended pregnancies are included. (3) The pilot RCT fitted well around clinical activities but 37% of the intervention group did not see the Men’s Safer Sex website and technical problems were frustrating. Men’s views of the Men’s Safer Sex website and research procedures were largely positive.ConclusionsIt would be feasible to conduct a large-scale RCT using clinic STI diagnoses as a primary outcome; however, technical errors and a poor response rate limited the collection of online self-reported outcomes. The next steps are (1) to optimise software for online trials, (2) to find the best ways to integrate digital health promotion with clinical services, (3) to develop more precise methods for collecting resource use data and (4) to work out how to overcome barriers to digital intervention testing and implementation in the NHS.Trial registrationCurrent Controlled Trials ISRCTN18649610.FundingThis project was funded by the NIHR Health Technology Assessment programme and will be published in full inHealth Technology Assessment; Vol. 20, No. 91. See the NIHR Journals Library website for further project information.

show abstract

“…However, attrition is a common problem across digital health technologies [35][36][37]. The importance of investigating approaches to improve user engagement has been highlighted in the literature [38], with the use of incentives [39] and prompts [40] identified as methods of increasing engagement.…”

Section: Digital As a Vehicle For Providing Supportmentioning

confidence: 99%

A Web-Based Intervention for Relatives of People Experiencing Psychosis or Bipolar Disorder: Design Study Using a User-Centered Approach (Preprint)

Honary¹,

Fisher²,

McNaney³

et al. 2018

Preprint

View full text Add to dashboard Cite

Background: Relatives of people experiencing bipolar mood episodes or psychosis, face a multitude of challenges (e.g. social isolation, limited coping strategies, issues with maintaining relationships). Despite this, there is limited informational and emotional support for people who find themselves in supporting or caring roles. Digital technologies provide us with an opportunity to offer accessible tools, which can be used flexibly to provide evidence-based information and support, allowing relatives to build their understanding of mental health problems and learn from others who have similar experiences. However, in order to design tools that are useful to relatives, we first need to understand their needs.

show abstract

Methodological Challenges in Online Trials

Cited by 182 publications

References 42 publications

Sexual health promotion for young people delivered via digital media: a scoping review

Sexual health promotion for young people delivered via digital media: a scoping review

The Men’s Safer Sex project: intervention development and feasibility randomised controlled trial of an interactive digital intervention to increase condom use in men

A Web-Based Intervention for Relatives of People Experiencing Psychosis or Bipolar Disorder: Design Study Using a User-Centered Approach (Preprint)

Contact Info

Product

Resources

About