User Involvement in the Design and Development of Patient Decision Aids and Other Personal Health Tools: A Systematic Review

Abstract

Background When designing and developing patient decision aids, guidelines recommend involving patients and stakeholders. There are myriad ways to do this. We aimed to describe how such involvement occurs by synthesizing reports of patient decision aid design and development within a user-centered design framework and to provide context by synthesizing reports of user-centered design applied to other personal health tools. Methods We included articles describing at least one development step of 1) a patient decision aid, 2) user- or human-centered design of another personal health tool, or 3) evaluation of these. We organized data within a user-centered design framework comprising 3 elements in iterative cycles: understanding users, developing/refining prototype, and observing users. Results We included 607 articles describing 325 patient decision aid projects and 65 other personal health tool projects. Fifty percent of patient decision aid projects reported involving users in at least 1 step for understanding users, 35% in at least 1 step for developing/refining the prototype, and 84% in at least 1 step for observing users’ interaction with the prototype. In comparison, other personal health tool projects reported 91%, 49%, and 92%, respectively. A total of 74% of patient decision aid projects and 92% of other personal health tool projects reported iterative processes, both with a median of 3 iterative cycles. Preliminary evaluations such as usability or feasibility testing were reported in 66% of patient decision aid projects and 89% of other personal health tool projects. Conclusions By synthesizing design and development practices, we offer evidence-based portraits of user involvement. Those wishing to further align patient decision aid design and development with user-centered design methods could involve users earlier, design and develop iteratively, and report processes in greater detail.

An 11-Item Measure of User- and Human-Centered Design for Personal Health Tools (UCD-11): Development and Validation (Preprint)

“…This included consultation with 15 external experts and four rounds of pilot data extraction and refinement of response processes across randomly selected sets of five articles each time (total: 20 articles). We had also confirmed the accuracy of the extracted data with the authors of the original articles included in the systematic review and found very low rates of error [13].…”

“…Content validity (point 1 in the validity framework [31]) refers to how well items match the definition of a construct. To ensure content validity of items, in our original systematic review, we had used foundational literature [15,16,18,[43][44][45]; held monthly or bimonthly consultations in person and by teleconference over the course of 2 years within our interdisciplinary group of experts, including patients, caregivers, health professionals, academic researchers, and other stakeholders; and consulted with 15 additional experts outside the research team [13]. Discussions over the years of the project centered on the items themselves as well as prioritization of items according to their relevance within our conceptual framework.…”

“…First, our data came from published reports, not direct capture of design and development processes. Although we have reason to believe the data are of high quality given our rigorous data validation and low rates of error [13], data from a systematic review of this nature may not contain full details of design and development processes. We chose to use these data because we believed they might offer valuable insights across hundreds of projects.…”

“…Accordingly, designers and developers frequently seek to involve users in design and development processes to ensure such usability and utility. In a previous systematic review of the design and development processes of a range of personal health tools, we documented that the extent and type of user involvement varies widely [13]. Structured ways to describe this variation could help capture data across projects and may serve to build an evidence base about the potential effects of design and development processes.…”

“…In our previous systematic review, we used the above framework of user-centered design to extract and organize the extracted data from 623 articles describing the design, development, or evaluation processes of 390 personal health tools, predominantly patient decision aids, which are tools intended to support personal health decisions [13]. We documented a wide range of practices, leading us to question whether it might be possible to use this structured data set to develop a measure to capture aspects of the user-centeredness of design and development processes, similar to how other measures capture complex concepts or processes that are not directly observable; for example, social capital [27], learning processes [28], health-related quality of life [29], and health care quality [30].…”