Design of an interface to communicate artificial intelligence-based prognosis for patients with advanced solid tumors: a user-centered approach

Abstract

Objectives To design an interface to support communication of machine learning (ML)-based prognosis for patients with advanced solid tumors, incorporating oncologists’ needs and feedback throughout design. Materials and Methods Using an interdisciplinary user-centered design approach, we performed 5 rounds of iterative design to refine an interface, involving expert review based on usability heuristics, input from a color-blind adult, and 13 individual semi-structured interviews with oncologists. Individual interviews included patient vignettes and a series of interfaces populated with representative patient data and predicted survival for each treatment decision point when a new line of therapy (LoT) was being considered. Ongoing feedback informed design decisions, and directed qualitative content analysis of interview transcripts was used to evaluate usability and identify enhancement requirements. Results Design processes resulted in an interface with 7 sections, each addressing user-focused questions, supporting oncologists to “tell a story” as they discuss prognosis during a clinical encounter. The iteratively enhanced interface both triggered and reflected design decisions relevant when attempting to communicate ML-based prognosis, and exposed misassumptions. Clinicians requested enhancements that emphasized interpretability over explainability. Qualitative findings confirmed that previously identified issues were resolved and clarified necessary enhancements (eg, use months not days) and concerns about usability and trust (eg, address LoT received elsewhere). Appropriate use should be in the context of a conversation with an oncologist. Conclusion User-centered design, ongoing clinical input, and a visualization to communicate ML-related outcomes are important elements for designing any decision support tool enabled by artificial intelligence, particularly when communicating prognosis risk.

Patient and Caregiver Perceptions of an Interface Design to Communicate Artificial Intelligence–Based Prognosis for Patients With Advanced Solid Tumors

“…39,40 For these reasons, participants underscored the importance of thoughtful implementation. Agreeing with oncologists, 26 participants thought the tool should be used in context of a conversation with their oncologist; however, participants further suggested presenting Recommended Actions separate from prognosis information. In the future, the tool's content could be incorporated into a more comprehensive interface with individualized data about specific treatments and side effects, especially if the tool is integrated into the EHR and patient portal, and population outcomes can be described for alternative treatment options.…”

“…Previous iterative design with oncologists resulted in an interface that displays patient-specific EHR data and conveys 6-month chance of survival, change in prognosis over time, survival of similar patients (“Patients Like Me”), and recommended next steps including supportive care options. 26 For this study, the tool was presented to study participants in a pre-recorded video to (1) ensure standardization of content and presentation, (2) remove dependence on researchers explaining the tool in real time, and (3) given the sensitive nature of prognosis, we wanted the study participant to be an observer and not feel as though the conversation was directed toward them. The video depicted a fictional conversation between an oncologist and patient at a third treatment decision point and focused on the oncologist's explanation of the tool.…”

“…This version of the interface was designed on the basis of usercentered design methods with input from oncologists. 26 negative perceptions from wants, as participants expressed negative sentiment while also suggesting improvements for the tool.…”

“…26 Additionally, clinicians noted the interface may help facilitate SICs and should be shared with patients and their caregivers. 26 Thus, we applied user-centered design processes to solicit input from patients and caregivers.…”

AI and professional liability assessment in healthcare. A revolution in legal medicine?

“…The specific design of interfaces emerges as a key factor in achieving the goal of enhancing human interaction and control. Tailoring user interfaces to facilitate transparent communication between humans and AI systems is crucial for better understanding and interpretation of AI-generated insights ( 30 ). Intuitive and user-friendly interfaces could enhance the understanding of a logarithmic decision with a high level of causal understanding, empowering individuals to effectively engage with and influence the decision-making process ( 31 ).…”