Usability testing of Avoiding Diabetes Thru Action Plan Targeting (ADAPT) decision support for integrating care-based counseling of pre-diabetes in an electronic health record

Abstract

Purpose Usability testing can be used to evaluate human computer interaction (HCI) and communication in shared decision making (SDM) for patient-provider behavioral change and behavioral contracting. Traditional evaluations of usability using scripted or mock patient scenarios with think-aloud protocol analysis provide a to identify HCI issues. In this paper we describe the application of these methods in the evaluation of the Avoiding Diabetes Thru Action Plan Targeting (ADAPT) tool, and test the usability of the tool to support the ADAPT framework for integrated care counseling of pre-diabetes. The think-aloud protocol analysis typically does not provide an assessment of how patient-provider interactions are effected in “live” clinical workflow or whether a tool is successful. Therefore, “Near-live” clinical simulations involving applied simulation methods were used to compliment the think-aloud results. This complementary usability technique was used to test the end-user HCI and tool performance by more closely mimicking the clinical workflow and capturing interaction sequences along with assessing the functionality of computer module prototypes on clinician workflow. We expected this method to further complement and provide different usability findings as compared to think-aloud analysis. Together, this mixed method evaluation provided comprehensive and realistic feedback for iterative refinement of the ADAPT system prior to implementation. Methods The study employed two phases of testing of a new interactive ADAPT tool that embedded an evidence-based shared goal setting component into primary care workflow for dealing with pre-diabetes counseling within a commercial physician office electronic health record (EHR). Phase I applied usability testing that involved “think-aloud” protocol analysis of 8 primary care providers interacting with several scripted clinical scenarios. Phase II used “near-live” clinical simulations of 5 providers interacting with standardized trained patient actors enacting the clinical scenario of counseling for pre-diabetes, each of whom had a pedometer that recorded the number of steps taken over a week. In both phases, all sessions were audio-taped and motion screen-capture software was activated for onscreen recordings. Transcripts were coded using iterative qualitative content analysis methods. Results In Phase I, the impact of the components and layout of ADAPT on user’s Navigation, Understandability, and Workflow were associated with the largest volume of negative comments (i.e. approximately 80% of end-user commentary), while Usability and Content of ADAPT were representative of more positive than negative user commentary. The heuristic category of Usability had a positive-to-negative comment ratio of 2.1, reflecting positive perception of the usability of the tool, its functionality, and overall co-productive utilization of ADAPT. However, there were mixed perceptions about content (i.e., how the information was displayed, organized and described in the tool). In Phase II...

La Actividad Física en El Currículo Universitario Español De Los Grados De Medicina Y Ciencias De La Actividad Física Y Del Deporte Como Recurso en La Prevención Y Tratamiento De Las Enfermedades Crónicas

“…En España, se han desarrollado programas preventivos educacionales para adecuar y aumentar la práctica de ejercicio físico en pacientes con diabetes tipo II que han tenido resultados satisfactorios (37). También se debe fomentar el empleo de una evaluación de diagnóstico que indique el nivel de actividad y de sedentarismo que ofrece un paciente como un marcador de salud (38).…”

Usability evaluation of the SMART application for youth with mTBI

“…All participants were asked to think aloud while performing task in the study. This type of methodology has been used to study usability because it provides further insight of the participant’s thought processes while attempting to perform a specific task [34,35]. This is particularly useful with small sample sizes and can capture on both usability and content data about SMART as participants are experiencing it.…”

Bridging the Gap Between Academic Research and Pragmatic Needs in Usability: A Hybrid Approach to Usability Evaluation of Health Care Information Systems (Preprint)

“…[35] These studies provide novel examples of a hybrid approach that meets the needs of system developers charged with optimizing systems, as well as academic usability researchers tasked with furthering our knowledge and perspective on the role of usability testing in healthcare technology.…”

“…The ADAPT tool was designed as part of an NIH-funded decision support trial to support integrated care counseling of pre-diabetes, providing templates within an (EHR) to guide physician-patient dialogues. [35][36][37] The ADAPT study used the think aloud method to examine usability and workflow issues for eight primary care physicians as they interacted with the prototype ADAPT EHR module during scripted clinical scenarios. These sessions were observed by members of the research team and summaries of issues and possible solutions were created.…”

“…This is evident in our publication of usability findings and implications from ADAPT EHR in a peer reviewed journal publication. [35][36][37] …”