Grand rounds in methodology: key considerations for implementing machine learning solutions in quality improvement initiatives

Abstract

Machine learning (ML) solutions are increasingly entering healthcare. They are complex, sociotechnical systems that include data inputs, ML models, technical infrastructure and human interactions. They have promise for improving care across a wide range of clinical applications but if poorly implemented, they may disrupt clinical workflows, exacerbate inequities in care and harm patients. Many aspects of ML solutions are similar to other digital technologies, which have well-established approaches to implementation. However, ML applications present distinct implementation challenges, given that their predictions are often complex and difficult to understand, they can be influenced by biases in the data sets used to develop them, and their impacts on human behaviour are poorly understood. This manuscript summarises the current state of knowledge about implementing ML solutions in clinical care and offers practical guidance for implementation. We propose three overarching questions for potential users to consider when deploying ML solutions in clinical care: (1) Is a clinical or operational problem likely to be addressed by an ML solution? (2) How can an ML solution be evaluated to determine its readiness for deployment? (3) How can an ML solution be deployed and maintained optimally? The Quality Improvement community has an essential role to play in ensuring that ML solutions are translated into clinical practice safely, effectively, and ethically.

Approaches to identify scenarios for data science implementations within healthcare settings: recommendations based on experiences at multiple academic institutions (Preprint)

“…There are several papers that have described frameworks for healthcare institutions to develop and implement data science solutions, including ML. [2][3][4][5][6][7][8] However, most propose solutions that are generated from a theoretical or a single institution perspective. In contrast, we hypothesized that building upon the experiences of several academic health sciences institutions implementing data science solutions would reveal more general patterns associated with successful implementations.…”

Letter to the Editor: Digital health: dashboards, dashboards everywhere

“…Not all healthcare organisations have the capacity to incorporate robust research designs into clinical analytics implementation, but at the very least they should be following a checklist or framework. 4a , 5a We acknowledge more work needs to be done to understand the impact of dashboards on patient care outcomes. 6a , 7a The goal should always be to provide evidence-based information at the point of care.…”