The Safety of Digital Mental Health Interventions: Systematic Review and Recommendations

Abstract

Background Evidence suggests that digital mental health interventions (DMHIs) for common mental health conditions are effective. However, digital interventions, such as face-to-face therapies, pose risks to patients. A safe intervention is considered one in which the measured benefits outweigh the identified and mitigated risks. Objective This study aims to review the literature to assess how DMHIs assess safety, what risks are reported, and how they are mitigated in both the research and postmarket phases and building on existing recommendations for assessing, reporting, and mitigating safety in the DMHI and standardizing practice. Methods PsycINFO, Embase, and MEDLINE databases were searched for studies that addressed the safety of DMHIs. The inclusion criteria were any study that addressed the safety of a clinical DMHI, even if not as a main outcome, in an adult population, and in English. As the outcome data were mainly qualitative in nature, a meta-analysis was not possible, and qualitative analysis was used to collate the results. Quantitative results were synthesized in the form of tables and percentages. To illustrate the use of a single common safety metric across studies, we calculated odds ratios and CIs, wherever possible. Results Overall, 23 studies were included in this review. Although many of the included studies assessed safety by actively collecting adverse event (AE) data, over one-third (8/23, 35%) did not assess or collect any safety data. The methods and frequency of safety data collection varied widely, and very few studies have performed formal statistical analyses. The main treatment-related reported AE was symptom deterioration. The main method used to mitigate risk was exclusion of high-risk groups. A secondary web-based search found that 6 DMHIs were available for users or patients to use (postmarket phase), all of which used indications and contraindications to mitigate risk, although there was no evidence of ongoing safety review. Conclusions The findings of this review show the need for a standardized classification of AEs, a standardized method for assessing AEs to statically analyze AE data, and evidence-based practices for mitigating risk in DMHIs, both in the research and postmarket phases. This review produced 7 specific, measurable, and achievable recommendations with the potential to have an immediate impact on the field, which were implemented across ongoing and future research. Improving the quality of DMHI safety data will allow meaningful assessment of the safety of DMHIs and confidence in whether the benefits of a new DMHI outweigh its risks. Trial Registration PROSPERO CRD42022333181; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=333181

Developing a process for assessing the safety of a digital mental health intervention and gaining regulatory approval: an academic’s guide

“…Customisation included adding introductory scripting, one or more prompt questions under each adverse event category, examples of typical events for researchers' reference and reordering/ grouping categories and questions to optimise e ciency and acceptability of the delivery. According to the ICH-GCP guidelines all AE data need to be categorized based on seriousness, severity, relatedness, and expectedness (7). This was done following standard guidance widely available across clinical trials units (see Appendix D for further details).…”

“…The research team needed to present current safety data relevant to STOP such as previous publications, feasibility or pilot studies from the same or similar devices/ interventions. To achieve this, the team referred to a previously conducted study that had assessed the intervention's feasibility and safety as a desktop intervention (7). The feasibility study included two arms: treatment (CBM-pa which is the therapeutic intervention used in STOP delivered in a web-based format) and an active control (7).…”

“…Thus, the purpose of this paper is to share a worked example of how our clinical trial team successfully applied and received the MHRA's NoNO for STOP (Successful Treatment for Paranoia). STOP is a mobile app DMHI that uses Cognitive Bias Modi cation for paranoia (CBM-pa) to reduce symptoms of paranoia (7). Through our example, we aim to provide valuable insights to other research teams undertaking clinical investigations of DMHIs, particularly those requiring regulatory (e.g., MHRA) approval and guidance in the assessment of safety in DMHI research.…”

“…As part of the NoNO application, the DMHI team needs to demonstrate to the MHRA the preliminary safety of their product (6). This can be done by providing the MHRA with a comprehensive summary of the existing knowledge around the safety of the intervention, a risk management plan (identifying all possible hazards, their potential impact, ways to mitigate them), and a plan of how the safety of the DMHI will be assessed in the proposed clinical investigation, such as collecting and assessing Adverse Events (AEs) (6); where AEs are de ned as an "untoward medical occurrence" (7).…”

“…A recent review found that the identi cation and categorisation of AEs in DMHI trials was inconsistent and often inadequate (3,7). It is essential that consistent processes and guidelines are developed so that DMHI manufacturers and researchers undertake and achieve the necessary regulatory approvals to a common standard.…”