Algorithm Change Protocols in the Regulation of Adaptive Machine Learning–Based Medical Devices

Gilbert, Stephen; Fenech, Matthew; Hirsch, Martin C.; Upadhyay, Shubhanan; Biasiucci, Andrea; Starlinger, Johannes

doi:10.2196/30545

Cited by 32 publications

(39 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…To optimize on the benefits associated with SaMD, patient safety and effectiveness need to be aptly assessed for which 2 key steps are necessary. First, international standards and guiding principles addressing the uniqueness of SaMD with a continuous learning algorithm are required [ 14 ], outlining best practice oversight and reporting requirements. Aligned regulatory requirements, tailor-made for SaMD with a continuous learning algorithm, are essential, particularly to verify maintenance measures to keep in check modifications throughout the life cycle of SaMD.…”

Section: Discussionmentioning

confidence: 99%

Technology-Enabled, Evidence-Driven, and Patient-Centered: The Way Forward for Regulating Software as a Medical Device

Carolan¹,

McGonigle²,

Dennis³

et al. 2022

JMIR Med Inform

View full text Add to dashboard Cite

Artificial intelligence (AI) is a broad discipline that aims to understand and design systems that display properties of intelligence. Machine learning (ML) is a subset of AI that describes how algorithms and models can assist computer systems in progressively improving their performance. In health care, an increasingly common application of AI/ML is software as a medical device (SaMD), which has the intention to diagnose, treat, cure, mitigate, or prevent disease. AI/ML includes either “locked” or “continuous learning” algorithms. Locked algorithms consistently provide the same output for a particular input. Conversely, continuous learning algorithms, in their infancy in terms of SaMD, modify in real-time based on incoming real-world data, without controlled software version releases. This continuous learning has the potential to better handle local population characteristics, but with the risk of reinforcing existing structural biases. Continuous learning algorithms pose the greatest regulatory complexity, requiring seemingly continuous oversight in the form of special controls to ensure ongoing safety and effectiveness. We describe the challenges of continuous learning algorithms, then highlight the new evidence standards and frameworks under development, and discuss the need for stakeholder engagement. The paper concludes with 2 key steps that regulators need to address in order to optimize and realize the benefits of SaMD: first, international standards and guiding principles addressing the uniqueness of SaMD with a continuous learning algorithm are required and second, throughout the product life cycle and appropriate to the SaMD risk classification, there needs to be continuous communication between regulators, developers, and SaMD end users to ensure vigilance and an accurate understanding of the technology.

show abstract

Section: Discussionmentioning

confidence: 99%

Technology-Enabled, Evidence-Driven, and Patient-Centered: The Way Forward for Regulating Software as a Medical Device

Carolan¹,

McGonigle²,

Dennis³

et al. 2022

JMIR Med Inform

View full text Add to dashboard Cite

show abstract

“…To capture these shifts, we should incorporate online and incremental learning (frequent model retraining) into the local deployment pipeline. Recent regulatory updates such as the FDA algorithmic predetermined change control plan make the localisation and frequent fine-tuning practically, and regulatorily possible [29].…”

Section: Discussionmentioning

confidence: 99%

RapiD_AI: A framework for Rapidly Deployable AI for novel disease & pandemic preparedness

Youssef

Zhu

Thakur

et al. 2022

Preprint

View full text Add to dashboard Cite

COVID-19 is unlikely to be the last pandemic that we face. According to an analysis of a global dataset of historical pandemics from 1600 to the present, the risk of a COVID-like pandemic has been estimated as 2.63% annually or a 38% lifetime probability. This rate may double over the coming decades. While we may be unable to prevent future pandemics, we can reduce their impact by investing in preparedness. In this study, we propose RapiD_AI: a framework to guide the use of pretrained neural network models as a pandemic preparedness tool to enable healthcare system resilience and effective use of ML during future pandemics. The RapiD_AI framework allows us to build high-performing ML models using data collected in the first weeks of the pandemic and provides an approach to adapt the models to the local populations and healthcare needs. The motivation is to enable healthcare systems to overcome data limitations that prevent the development of effective ML in the context of novel diseases. We digitally recreated the first 20 weeks of the COVID-19 pandemic and experimentally demonstrated the RapiD_AI framework using domain adaptation and inductive transfer. We (i) pretrain two neural network models (Deep Neural Network and TabNet) on a large Electronic Health Records dataset representative of a general in-patient population in Oxford, UK, (ii) fine-tune using data from the first weeks of the pandemic, and (iii) simulate local deployment by testing the performance of the models on a held-out test dataset of COVID-19 patients. Our approach has demonstrated an average relative/absolute gain of 4.92/4.21% AUC compared to an XGBoost benchmark model trained on COVID-19 data only. Moreover, we show our ability to identify the most useful historical pretraining samples through clustering and to expand the task of deployed models through inductive transfer to meet the emerging needs of a healthcare system without access to large historical pretraining datasets.

show abstract

“…Thus, a total product lifecycle approach (TLPC) and multi-stakeholder involvement are essential features of any ethical or legal approach to environments comprising MLMD agents and humans and any technical risk-management framework concerning MLMD. Regulatory outlets supporting continuous learning are an issue in Europe (67).…”

Section: Summary and Reflections On The Future Of Regulationmentioning

confidence: 99%

Real-World and Regulatory Perspectives of Artificial Intelligence in Cardiovascular Imaging

Wellnhofer

2022

Front. Cardiovasc. Med.

View full text Add to dashboard Cite

Recent progress in digital health data recording, advances in computing power, and methodological approaches that extract information from data as artificial intelligence are expected to have a disruptive impact on technology in medicine. One of the potential benefits is the ability to extract new and essential insights from the vast amount of data generated during health care delivery every day. Cardiovascular imaging is boosted by new intelligent automatic methods to manage, process, segment, and analyze petabytes of image data exceeding historical manual capacities. Algorithms that learn from data raise new challenges for regulatory bodies. Partially autonomous behavior and adaptive modifications and a lack of transparency in deriving evidence from complex data pose considerable problems. Controlling new technologies requires new controlling techniques and ongoing regulatory research. All stakeholders must participate in the quest to find a fair balance between innovation and regulation. The regulatory approach to artificial intelligence must be risk-based and resilient. A focus on unknown emerging risks demands continuous surveillance and clinical evaluation during the total product life cycle. Since learning algorithms are data-driven, high-quality data is fundamental for good machine learning practice. Mining, processing, validation, governance, and data control must account for bias, error, inappropriate use, drifts, and shifts, particularly in real-world data. Regulators worldwide are tackling twenty-first century challenges raised by “learning” medical devices. Ethical concerns and regulatory approaches are presented. The paper concludes with a discussion on the future of responsible artificial intelligence.

show abstract

Algorithm Change Protocols in the Regulation of Adaptive Machine Learning–Based Medical Devices

Cited by 32 publications

References 19 publications

Technology-Enabled, Evidence-Driven, and Patient-Centered: The Way Forward for Regulating Software as a Medical Device

Technology-Enabled, Evidence-Driven, and Patient-Centered: The Way Forward for Regulating Software as a Medical Device

RapiD_AI: A framework for Rapidly Deployable AI for novel disease & pandemic preparedness

Real-World and Regulatory Perspectives of Artificial Intelligence in Cardiovascular Imaging

Contact Info

Product

Resources

About