Assessing Barriers to Implementation of Machine Learning and Artificial Intelligence–Based Tools in Critical Care: Web-Based Survey Study (Preprint)

Abstract

BACKGROUND Although there is considerable interest in machine learning (ML) and artificial intelligence (AI) in critical care, implementation of effective algorithms into practice has been limited. OBJECTIVE We sought to understand physician perspectives of a novel intubation prediction tool as well as provider and patient perspectives on the use of ML in healthcare to elucidate implementation determinants of ML/AI-based algorithms in critical care. METHODS We developed two anonymous surveys in Qualtrics, one single-center survey distributed to 99 critical care physicians via email, and one social media survey distributed via Facebook and Twitter with branching logic to tailor questions for providers and patients. The surveys included a mixture of categorical, Likert scale, and free-text items. Likert scale means with standard deviations were reported from 1-5. We used student t-tests to examine differences between groups. RESULTS Forty-seven critical care physicians completed the initial survey (47/99, 47.5%). Willingness to use the ML-based algorithm was 3.32 (0.95). The social media survey had 770 total responses (provider n=605 (78.6%), patient n=165 (21.4%)). We found no difference in providers’ knowledge based on level of experience in either survey. We found that patients had significantly less knowledge of ML (3.04 (1.53) vs 3.43 (0.941), p<0.001) and comfort with ML (3.28 (1.02) vs 3.53 (0.935), p= 0.0038) than providers. Free text responses revealed multiple shared concerns, including workflow interruptions and data bias. CONCLUSIONS These data suggest that providers and patients have positive perceptions of ML-based tools, and that a tool to predict need for intubation would be of interest to critical care providers. There were shared concerns regarding workflow interruption and data bias. These survey results provide a baseline evaluation of implementation determinants of ML/AI-based tools that will be important in their optimal implementation and adoption in the critical care setting.