Human-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making

Abstract

Machin e lear nin g (ML) is incr easingly being use d in image retrieval systems for medical decision making. On e app lication of ML is to retrieve visually similar medical images from pas t patients (e.g. tissue from biops ies) to reference whe n making a medical decision with a new pat ient. Howeve r, no algorithm can perfectly captu re an expert ' s ideal notion of similarity for every case: an image th at is algorithmi cally determin ed to be similar may not be medically relevant to a doctor' s specific diagnostic needs. In this pape r, we identified the needs of patho logists whe n searchin g for similar images retrieved usin g a deep lear nin g algorithm , and develope d tools that empower use rs to cope with the search algorithm on-the -fly, communi cating what types of similarity are most import ant at different moment s in time. In two evaluations with path ologists, we found th at th ese refinement tools increased the diagnos tic utility of images found and increased user trus t in the algorithm. Th e tools we re preferred over a traditi onal interface, without a loss in diagnostic accuracy. We also observe d that users adopted new str ategies whe n using refinement tools, re-purpos ing th em to test and understand the underlying algorithm and to disambiguate ML errors from their own errors. Taken togethe r, these findings inform futur e hum an-ML collabo rative systems for expe rt decision-m aking. CCS CONCEPTS• Human-centered computing --> Human computer interaction (HCI); KEYWORDS Human -AI int eraction ; machin e learnin g; clinical healthPermission to mak e digital or har d copies of part or all of this work for personal or classroom use is grant ed without fee provi ded that copies are not made or distributed for profit or commercial advanta ge and that copies bear this notice an d the full citation on the first page. Figure 1: Medical images contain a wide range of clinical features , such as cellular (1) and glandular morphology (2), interaction between components (3), processing artifacts (4), and many more. It can be difficult for a similar -image search algorithm to perfectly capture an expert's notion of similarity ,

explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning

“…Others target multi-model visual comparison for refinement, e.g., [51,18]. In addition to this distinction, various interactive refinement approaches are used in iterative cycles, e.g., Cai et al [10] on medical images or El-Assady et al [19,17] for topic modeling. Such examples highlight the need for interactive and iterative refinement cycles in our XAI pipeline.…”

Using content‐based image retrieval of dermoscopic images for interpretation and education: A pilot study

“…Trust is a challenging factor for medical decision support tools, and some studies propose that a second opinion by a decision support system is not always welcomed by clinicians when it does not match their own initial diagnosis …”

“…Trust is a challenging factor for medical decision support tools, and some studies propose that a second opinion by a decision support system is not always welcomed by clinicians when it does not match their own initial diagnosis. 6 Since retrieving similar cases can provide a diagnostic support environment rather than a single second diagnosis, we investigated "trust in CBIR results" as another critical factor in the post-task questionnaire.…”

“…Despite the proposed benefits of modern CBIR systems, most CBIR‐related research to date has focused on improving the accuracy of AI systems for diagnostic decisions: we know little about the perceived utility and usability of CBIR systems for end users from a human‐computer interaction (HCI) perspective . In this paper, we describe a pilot study on how an interactive dermoscopic CBIR‐based decision support tool can be used to help classify skin lesion images, and determine if the tool has educational value for users without prior dermoscopy training.…”

Amplifying Domain Expertise in Clinical Data Pipelines

“…Examples include natural language interfaces and form-based input [ 27 ]. Finally, domain experts are highly trained individuals, which allows systems to accelerate their input by using domain-specific assumptions and ontologies [ 28 , 29 ]. Keeping these factors in mind, expertise amplification involves summarization, guidance, interaction, and acceleration ( Figure 1 ).…”