“Think aloud” and “Near live” usability testing of two complex clinical decision support tools

Abstract

Objectives Low provider adoption continues to be a significant barrier to realizing the potential of clinical decision support. “Think Aloud” and “Near Live” usability testing were conducted on two clinical decision support tools. Each was composed of an alert, a clinical prediction rule which estimated risk of either group A Streptococcus pharyngitis or pneumonia and an automatic order set based on risk. The objective of this study was to further understanding of the facilitators of usability and to evaluate the types of additional information gained from proceeding to “Near Live” testing after completing “Think Aloud”. Methods This was a qualitative observational study conducted at a large academic health care system with 12 primary care providers. During “Think Aloud” testing, participants were provided with written clinical scenarios and asked to verbalize their thought process while interacting with the tool. During “Near Live” testing participants interacted with a mock patient. Morae usability software was used to record full screen capture and audio during every session. Participant comments were placed into coding categories and analyzed for generalizable themes. Themes were compared across usability methods. Results “Think Aloud” and “Near Live” usability testing generated similar themes under the coding categories visibility, workflow, content, understand-ability and navigation. However, they generated significantly different themes under the coding categories usability, practical usefulness and medical usefulness. During both types of testing participants found the tool easier to use when important text was distinct in its appearance, alerts were passive and appropriately timed, content was up to date, language was clear and simple, and each component of the tool included obvious indicators of next steps. Participant comments reflected higher expectations for usability and usefulness during “Near Live” testing. For example, visit aids, such as automatically generated order sets, were felt to be less useful during “Near-Live” testing because they would not be all inclusive for the visit. Conclusions These complementary types of usability testing generated unique and generalizable insights. Feedback during “Think Aloud” testing primarily helped to improve the tools’ ease of use. The additional feedback from “Near Live” testing, which mimics a real clinical encounter, was helpful for eliciting key barriers and facilitators to provider workflow and adoption.

Development and usability testing of an electronic patient-reported outcome measure (ePROM) system for patients with advanced chronic kidney disease

“…5 usability issues may be detected and addressed prior to full-scale implementation [31] in other to reduce attrition rates. [26,32,33]…”

Bridging the gap between evidence and practice in asthma management by developing a mobile-based clinical decision support system for GINA asthma guideline

“…By applying this method, the thoughts, interactions, and cognitions process are verbalized while the tasks are completed [24]. The usability sessions were conducted by representing written scenarios that described patients with different conditions to participants [25,26].…”

Emerging Insights of Health Informatics Research: A Literature Analysis for Outlining New Themes

“…Shin et al (2017) presented a study that validated a model using data collected in a time-series study of consumer use of online HIS in a user oriented context. Richardson et al (2017) Robinson et al (2014) identified the importance of patient portals for patients' interaction with healthcare organizations and to view information. The purposes are categorised as informational, medical, logistical, or social.…”