Usability evaluation of a clinical decision support tool for osteoporosis disease management

Abstract

BackgroundOsteoporosis affects over 200 million people worldwide at a high cost to healthcare systems. Although guidelines are available, patients are not receiving appropriate diagnostic testing or treatment. Findings from a systematic review of osteoporosis interventions and a series of focus groups were used to develop a functional multifaceted tool that can support clinical decision-making in osteoporosis disease management at the point of care. The objective of our study was to assess how well the prototype met functional goals and usability needs.MethodsWe conducted a usability study for each component of the tool--the Best Practice Recommendation Prompt (BestPROMPT), the Risk Assessment Questionnaire (RAQ), and the Customised Osteoporosis Education (COPE) sheet--using the framework described by Kushniruk and Patel. All studies consisted of one-on-one sessions with a moderator using a standardised worksheet. Sessions were audio- and video-taped and transcribed verbatim. Data analysis consisted of a combination of qualitative and quantitative analyses.ResultsIn study 1, physicians liked that the BestPROMPT can provide customised recommendations based on risk factors identified from the RAQ. Barriers included lack of time to use the tool, the need to alter clinic workflow to enable point-of-care use, and that the tool may disrupt the real reason for the visit. In study 2, patients completed the RAQ in a mean of 6 minutes, 35 seconds. Of the 42 critical incidents, 60% were navigational and most occurred when the first nine participants were using the stylus pen; no critical incidents were observed with the last six participants that used the touch screen. Patients thought that the RAQ questions were easy to read and understand, but they found it difficult to initiate the questionnaire. Suggestions for improvement included improving aspects of the interface and navigation. The results of study 3 showed that most patients were able to understand and describe sections of the COPE sheet, and all considered discussing the information with their physicians. Suggestions for improvement included simplifying the language and improving the layout.ConclusionsFindings from the three studies informed changes to the tool and confirmed the importance of usability testing on all end users to reduce errors, and as an important step in the development process of knowledge translation interventions.

Clinical Decision Support Tool for Parental Tobacco Treatment in Primary Care

“…[21][22][23][24] The development team included researchers, clinical informaticians, clinical content experts, and end-users. To further inform tool development, we performed focus groups and 1-on-1 interviews with clinicians and the practice's medical director in April and May 2015.…”

Application of the Knowledge-to-Action and Medical Research Council frameworks in the development of an osteoporosis clinical decision support tool

“…For example, the tool development process considered not only empirical evidence to build the prototype but also relied on input from a multidisciplinary group of experts (e.g., human factors' engineers, health information technologists, knowledge translation experts, and osteoporosis specialists). Secondly, both frameworks were used to guide the selection of knowledge relevant to osteoporosis disease management (i.e., systematic review [32]), and to ensure that all tool development studies (i.e., focus groups, and usability studies) were appropriately and rigorously conducted [33,39]. We tested the usability of all three components of the osteoporosis tool (with relevant end users) with as many cycles of iterations as was needed to eliminate problems and errors [39].…”

“…Secondly, both frameworks were used to guide the selection of knowledge relevant to osteoporosis disease management (i.e., systematic review [32]), and to ensure that all tool development studies (i.e., focus groups, and usability studies) were appropriately and rigorously conducted [33,39]. We tested the usability of all three components of the osteoporosis tool (with relevant end users) with as many cycles of iterations as was needed to eliminate problems and errors [39]. Thirdly, the theoretical frameworks emphasized the need to ensure that all stakeholders were involved in the tool development process/design, and identified which end users of the tool would be the most appropriate targets for specific tool components.…”

“…Our usability studies were guided by Kushniruk and Patel's [34] usability framework, a methodology widely regarded as the most useful method for testing most aspects of usability in the medical context [37,38]. Findings were used to make informed modifications and refinements to the osteoporosis tool prototype (see Appendix B at www.jclinepi.com), and confirmed the importance of testing the usability of the tool on all end users to reduce problems and errors [39]. These evaluations are important to ensure that the needs of end users are considered over the potentially biased perceptions of researchers and designers, and that this iterative analysis is conducted before implementation in clinical practice [40].…”

Clinical Decision Support Tool for Parental Tobacco Treatment in Hospitalized Children

“…This framework involves developing a tool, incorporating input from future users of that tool, followed by inspection of its use [19][20][21]. We included input from a diverse team comprised of research experts, clinical informatics experts, clinical content experts (all members of the study team), and end-users (first year pediatric residents, hereafter referred to as "clinicians") [22].…”