CheXstray: Real-time Multi-Modal Data Concordance for Drift Detection in Medical Imaging AI

Abstract

Background/Motivation: Rapidly expanding Clinical AI applications worldwide have the potential to impact to all areas of medical practice. Medical imaging applications constitute a vast majority of approved clinical AI applications with more than 150 to date. Though healthcare systems are eager to adopt AI solutions after initial successes in model validation and deployment for clinical workflows, fundamental questions remain: Is the model still working as expected?, What is causing the change?, Is it time to intervene? While infrastructure and local validation barriers are addressed during implementation, the larger problem looms regarding how to monitor input data and track model performance in real-time production systems. Real-world healthcare data environments are dynamic which can disrupt AI model performance, referred to as "drift". For example, medical equipment updates, new imaging workflows, changes in patient and disease characteristics, and more, can change over time and cause drift. While there are dozens of tools for monitoring drift, all lack functionality for image-based deep learning model architectures, cannot use healthcare medical imaging data, and require contemporaneous ground truth which is rarely available in healthcare settings. As a result, healthcare systems are left without insight for ongoing medical imaging model performance which poses serious safety concerns for healthcare system leadership, government regulators, and patients.Methods: In this study, we utilize the CheXpert and PadChest open datasets to build, design, and test a medical imaging AI drift monitoring workflow that can track data and model drift in the absence of contemporaneous ground truth for a chest X-ray use case. Ten pathology labels were chosen and reconciled between the two datasets in order to fine-tune a densely connected convolutional neural network on PadChest after being pre-trained on frontal-only CheXpert images. The PadChest dataset and accompanying ground truth labels were chronologically arranged and split such that all finetuning and validation used only data between 2007-2013 while tests were performed on data between 2014-2017. To test the robustness of our approach, we simulate drift in a variety of experiments and observe measured performance and drift within a data stream. We measure performance using ground truth labels and compare our drift metric with actual model performance over time in the test dataset and identify correlative signals. Drift evaluation used DICOM metadata, image appearance, and model output scores as input. Variational autoencoders (VAE) were used to capture an appearance feature encoding by using the underlying latent space representation. For continuous features (e.g.

Developing, Purchasing, Implementing and Monitoring AI Tools in Radiology: Practical Considerations. A Multi-Society Statement From the ACR, CAR, ESR, RANZCR & RSNA

“…To gain earlier insights into AI's accuracy, additional tools for assessing expected AI outcomes based on input data (e.g., determining whether the input data falls within or outside the training data distribution) or comparing the results of one AI model to those of other AI models simultaneously can be employed. 88 Autonomous AI should be designed to initiate actions that are transparent, identifiable, and discoverable. The capacity to disable the AI system swiftly and effectively in the event of failure is crucial for patient safety.…”

Mapping the Landscape of Care Providers’ Quality Assurance Approaches for AI in Diagnostic Imaging

“…Data shift is, of course, present in the data input to the AI model, and one possibility is to detect data shifts at the input stage. A recent radiology example is the data drift monitoring method for chest X-ray data by Soin et al [33], taking multi-modal input data into account.…”

“…Another aspect to consider is the risk that the identified potential issue is a false alarm or has negligible effect. For example, this is an inherent difficulty in data shift detection [33]. In general, the more one deploys automated quality issue indicators, the more you need to deal with in terms of drill-down efforts.…”

CyclOps: Cyclical development towards operationalizing ML models for health

“…PyHealth [25] is a Python library that offers preprocessing across various EHR datasets and modelling for multiple clinical prediction tasks. Moreover, these tools fail to offer capabilities to evaluate model robustness to data shifts and existing tools for monitoring of clinical ML models like CheXstray [18] focus on specific domains and offer limited functionality.…”