Background/Motivation: Rapidly expanding Clinical AI applications worldwide have the potential to impact to all areas of medical practice. Medical imaging applications constitute a vast majority of approved clinical AI applications with more than 150 to date. Though healthcare systems are eager to adopt AI solutions after initial successes in model validation and deployment for clinical workflows, fundamental questions remain: Is the model still working as expected?, What is causing the change?, Is it time to intervene? While infrastructure and local validation barriers are addressed during implementation, the larger problem looms regarding how to monitor input data and track model performance in real-time production systems. Real-world healthcare data environments are dynamic which can disrupt AI model performance, referred to as "drift". For example, medical equipment updates, new imaging workflows, changes in patient and disease characteristics, and more, can change over time and cause drift. While there are dozens of tools for monitoring drift, all lack functionality for image-based deep learning model architectures, cannot use healthcare medical imaging data, and require contemporaneous ground truth which is rarely available in healthcare settings. As a result, healthcare systems are left without insight for ongoing medical imaging model performance which poses serious safety concerns for healthcare system leadership, government regulators, and patients.Methods: In this study, we utilize the CheXpert and PadChest open datasets to build, design, and test a medical imaging AI drift monitoring workflow that can track data and model drift in the absence of contemporaneous ground truth for a chest X-ray use case. Ten pathology labels were chosen and reconciled between the two datasets in order to fine-tune a densely connected convolutional neural network on PadChest after being pre-trained on frontal-only CheXpert images. The PadChest dataset and accompanying ground truth labels were chronologically arranged and split such that all finetuning and validation used only data between 2007-2013 while tests were performed on data between 2014-2017. To test the robustness of our approach, we simulate drift in a variety of experiments and observe measured performance and drift within a data stream. We measure performance using ground truth labels and compare our drift metric with actual model performance over time in the test dataset and identify correlative signals. Drift evaluation used DICOM metadata, image appearance, and model output scores as input. Variational autoencoders (VAE) were used to capture an appearance feature encoding by using the underlying latent space representation. For continuous features (e.g.