Usability Issues in Evidence-Based Psychosocial Interventions and Implementation Strategies: Cross-project Analysis

Abstract

Background People often prefer evidence-based psychosocial interventions (EBPIs) for mental health care; however, these interventions frequently remain unavailable to people in nonspecialty or integrated settings, such as primary care and schools. Previous research has suggested that usability, a concept from human-centered design, could support an understanding of the barriers to and facilitators of the successful adoption of EBPIs and support the redesign of EBPIs and implementation strategies. Objective This study aimed to identify and categorize usability issues in EBPIs and their implementation strategies. Methods We adapted a usability issue analysis and reporting format from a human-centered design. A total of 13 projects supported by the National Institute of Mental Health—funded Accelerating the Reach and Impact of Treatments for Youth and Adults with Mental Illness Center at the University of Washington used this format to describe usability issues for EBPIs and implementation strategies with which they were working. Center researchers used iterative affinity diagramming and coding processes to identify usability issue categories. On the basis of these categories and the underlying issues, we propose heuristics for the design or redesign of EBPIs and implementation strategies. Results The 13 projects reported a total of 90 usability issues, which we categorized into 12 categories, including complex and/or cognitively overwhelming, required time exceeding available time, incompatibility with interventionist preference or practice, incompatibility with existing workflow, insufficient customization to clients/recipients, intervention buy-in (value), interventionist buy-in (trust), overreliance on technology, requires unavailable infrastructure, inadequate scaffolding for client/recipient, inadequate training and scaffolding for interventionists, and lack of support for necessary communication. These issues range from minor inconveniences that affect a few interventionists or recipients to severe issues that prevent all interventionists or recipients in a setting from completing part or all of the intervention. We propose 12 corresponding heuristics to guide EBPIs and implementation strategy designers in preventing and addressing these usability issues. Conclusions Usability issues were prevalent in the studied EBPIs and implementation strategies. We recommend using the lens of usability evaluation to understand and address barriers to the effective use and reach of EBPIs and implementation strategies. International Registered Report Identifier (IRRID) RR2-10.2196/14990

Helping Educational Leaders Mobilize Evidence (HELM): The iterative redesign of the Leadership and Organizational Change for Implementation (LOCI) intervention for use in schools

“…Ultimately, the collaborative redesign process allowed greater alignment with HELM components and school needs and priorities and should, in theory, increase adoption and implementation. These findings are consistent with recent calls for more tailorable or customizable implementation in complex settings ( Munson et al, 2022 ) and other efforts to collaboratively redesign implementation strategies outside the education sector ( Deatrick et al, 2021 ; Harkness et al, 2022 ). Given there are a number of usability challenges for implementation strategies when applied to novel settings, it is critical to ensure redesign solutions address barriers in the implementation context ( Lyon et al, 2020 ).…”

Development of a method for Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI): A modified Delphi study

“…A unique strength of MODIFI is its inclusion of HCD approaches for engaging stakeholders, understanding perspectives, and redesigning products in ways that elevate the voices of users. Increasingly, intervention and implementation researchers are drawing upon HCD concepts and techniques (e.g., Haines et al, 2021 ; Lyon et al, 2020a ; Lyon et al, 2020b ; Munson et al, 2022 ), and MODIFI provides an HCD-informed method with which to structure the application of these techniques to real-world intervention implementation problems. Thus, MODIFI may offer researchers new ways to approach intervention adaptations by prioritizing time- and resource-efficiency and user perspectives.…”

Formative Evaluation of a Comprehensive Self-Management Intervention for Irritable Bowel Syndrome, Comorbid Anxiety, and Depression: Mixed Methods Study

“…Research has argued for applying human-centered design and usability principles to address the lack of intervention implementation by redesigning interventions to improve usability while retaining the effective components [20][21][22]. Although usability has been most often applied in technology-based applications, usability evaluation principles can also be used to assess other products and services, including interventions and implementation strategies [9,23,24]. Human-centered design approaches focus on developing usable interventions through stakeholder input [21].…”