Human Factors and Technological Characteristics Influencing the Interaction of Medical Professionals With Artificial Intelligence–Enabled Clinical Decision Support Systems: Literature Review

Abstract

Background The digitization and automation of diagnostics and treatments promise to alter the quality of health care and improve patient outcomes, whereas the undersupply of medical personnel, high workload on medical professionals, and medical case complexity increase. Clinical decision support systems (CDSSs) have been proven to help medical professionals in their everyday work through their ability to process vast amounts of patient information. However, comprehensive adoption is partially disrupted by specific technological and personal characteristics. With the rise of artificial intelligence (AI), CDSSs have become an adaptive technology with human-like capabilities and are able to learn and change their characteristics over time. However, research has not reflected on the characteristics and factors essential for effective collaboration between human actors and AI-enabled CDSSs. Objective Our study aims to summarize the factors influencing effective collaboration between medical professionals and AI-enabled CDSSs. These factors are essential for medical professionals, management, and technology designers to reflect on the adoption, implementation, and development of an AI-enabled CDSS. Methods We conducted a literature review including 3 different meta-databases, screening over 1000 articles and including 101 articles for full-text assessment. Of the 101 articles, 7 (6.9%) met our inclusion criteria and were analyzed for our synthesis. Results We identified the technological characteristics and human factors that appear to have an essential effect on the collaboration of medical professionals and AI-enabled CDSSs in accordance with our research objective, namely, training data quality, performance, explainability, adaptability, medical expertise, technological expertise, personality, cognitive biases, and trust. Comparing our results with those from research on non-AI CDSSs, some characteristics and factors retain their importance, whereas others gain or lose relevance owing to the uniqueness of human-AI interactions. However, only a few (1/7, 14%) studies have mentioned the theoretical foundations and patient outcomes related to AI-enabled CDSSs. Conclusions Our study provides a comprehensive overview of the relevant characteristics and factors that influence the interaction and collaboration between medical professionals and AI-enabled CDSSs. Rather limited theoretical foundations currently hinder the possibility of creating adequate concepts and models to explain and predict the interrelations between these characteristics and factors. For an appropriate evaluation of the human-AI collaboration, patient outcomes and the role of patients in the decision-making process should be considered.

How, for whom, and in what contexts will artificial intelligence be adopted in pathology? A realist interview study

“…At present, scarce empirical evidence exists regarding factors necessary for integrating AI into clinical practice, 29 although trust has been identified as an important challenge. 30–32 While some authors have argued for the importance of system performance and AI being able to explain or justify its conclusions, 29 for our interviewees, system performance alone was considered adequate for generating trust.…”

“…At present, scarce empirical evidence exists regarding factors necessary for integrating AI into clinical practice, 29 although trust has been identified as an important challenge. 30–32 While some authors have argued for the importance of system performance and AI being able to explain or justify its conclusions, 29 for our interviewees, system performance alone was considered adequate for generating trust. Pathology and radiology are often considered similar fields, but a survey of radiologists suggests that while, like our interviewees, they place emphasis on evaluation, including evaluation using their own data, they also require algorithms to be understandable.…”

Predicting the causative pathogen among children with pneumonia using a causal Bayesian network

“…Second, the models may not be transportable from the training context to other clinical contexts, because the predictive values of input variables are often driven by baseline prevalence [ 15 ], infection severity and testing techniques employed in the validated study population. Third, association-driven predictions that lack causal explanation may not be accepted by their intended end-users (i.e., clinicians) [ 23 ]. The clinical implementation of any model for decision support is largely dependent on trust which, in turn, depends on end-users understanding how the model works.…”

Predicting the causative pathogen among children with pneumonia using a causal Bayesian network

“…Second, the models may not be transportable from the training context to other clinical contexts, because the predictive values of input variables are often driven by baseline prevalence [14], infection severity and testing techniques employed in the validated study population. Third, association-driven predictions that lack causal explanation may not be accepted by their intended end-users (i.e., clinicians) [23]. The clinical implementation of any model for decision support is largely dependent on trust which, in turn, depends on end-users understanding how the model works.…”