Algorithms on regulatory lockdown in medicine

Babic, Boris; Gerke, Sara; Evgeniou, Theodoros; Cohen, I. Glenn

doi:10.1126/science.aay9547

Cited by 82 publications

(72 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…healthcare and social care sectors, police and armed forces, firefighting, water and electricity supply, critical manufacturing). Other nonessential activities are hence stopped or carried out from home [11,12].…”

Section: Introductionmentioning

confidence: 99%

Health risks and potential remedies during prolonged lockdowns for coronavirus disease 2019 (COVID-19)

et al. 2020

View full text Add to dashboard Cite

As coronavirus disease 2019 (COVID-19) pandemic continues, an increasing number of countries and territories are adopting restrictive measures based on physical ("social") distancing, aimed at preventing human-to-human transmission and thereby limiting virus propagation. Nationwide lockdowns, encompassing mass quarantine under stay-at-home ordinances, have already been proven effective to contain the COVID-19 outbreak in some countries. Nevertheless, a prolonged homestay may also be associated with potential side effects, which may jeopardize people's health and thus must be recognized and mitigated in a way without violating local ordinances. Some of the most important undesirable consequences of prolonged homestay such as physical inactivity, weight gain, behavioral addiction disorders, insufficient sunlight exposure and social isolation will be critically addressed in this article, which also aims to provide some tentative recommendations for the alleviation of side effects.

show abstract

Section: Introductionmentioning

confidence: 99%

Health risks and potential remedies during prolonged lockdowns for coronavirus disease 2019 (COVID-19)

et al. 2020

View full text Add to dashboard Cite

show abstract

“…However, even with those restrictions in place, it can require the AI maker to set up a training program for their product, such as in the case of IDx-DR where the FDA required a training program including instructions on how to acquire and process quality images 12 . Regulators could require more, such as ongoing system monitoring, periodic retraining, software and usage inspections, review of aggregate usage statistics (e.g., to identify possible drifts in treatment frequencies and decision styles of users) 4 . They could also demand data and model validation and robustness analysis (e.g., via multiple re-trainings with different data subsets and data perturbations) of the AI/ML, such as due to data quality or adversarial attack issues 4 .…”

Section: Transitioning From a Product To A System Approach: First Stepsmentioning

confidence: 99%

“…Regulators could require more, such as ongoing system monitoring, periodic retraining, software and usage inspections, review of aggregate usage statistics (e.g., to identify possible drifts in treatment frequencies and decision styles of users) 4 . They could also demand data and model validation and robustness analysis (e.g., via multiple re-trainings with different data subsets and data perturbations) of the AI/ML, such as due to data quality or adversarial attack issues 4 . Further, regulators could also require testing variants that provide humans with different degrees of freedom: For example, users' discretion can be more or less limited in cases where devices provide probabilistic recommendations such as IDx-DR; or the AI/ML-based SaMD may provide more or fewer alternative recommended courses of action or even usage parameter choices.…”

Section: Transitioning From a Product To A System Approach: First Stepsmentioning

confidence: 99%

“…If the algorithm changes through usage, such SaMD will, at present, likely require another FDA round of review 3 . Thus, AI/ML makers will probably be inclined not to update their SaMD-both to reduce the cost and effort, but also because there are risks to proposing an update (and thus signaling a deficiency of the baseline product) if FDA does not approve the update or there is a significant delay 4 . For these reasons, the most valuable asset of AI/ML, its ability to improve by learning from data, may not be fully harnessed.…”

Section: Introductionmentioning

confidence: 99%

“…Elsewhere we have discussed the FDA's recent attempt to wrestle with what we call "the update problem", its treatment of "locked" versus "adaptive" algorithms, and have made proposals for how the FDA's approach might be improved, such as through continuous monitoring processes that consider specific risks of AI/ ML systems 4 . In this article, we make a more fundamental point: The FDA and its sister regulators in other countries have primarily been product regulators-they review and ultimately approve or reject medical products such as drugs and devices.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

The need for a system view to regulate artificial intelligence/machine learning-based software as medical device

et al. 2020

Self Cite

View full text Add to dashboard Cite

Artificial intelligence (AI) and Machine learning (ML) systems in medicine are poised to significantly improve health care, for example, by offering earlier diagnoses of diseases or recommending optimally individualized treatment plans. However, the emergence of AI/ML in medicine also creates challenges, which regulators must pay attention to. Which medical AI/ML-based products should be reviewed by regulators? What evidence should be required to permit marketing for AI/ML-based software as a medical device (SaMD)? How can we ensure the safety and effectiveness of AI/ML-based SaMD that may change over time as they are applied to new data? The U.S. Food and Drug Administration (FDA), for example, has recently proposed a discussion paper to address some of these issues. But it misses an important point: we argue that regulators like the FDA need to widen their scope from evaluating medical AI/ML-based products to assessing systems. This shift in perspective-from a product view to a system view-is central to maximizing the safety and efficacy of AI/ML in health care, but it also poses significant challenges for agencies like the FDA who are used to regulating products, not systems. We offer several suggestions for regulators to make this challenging but important transition.

show abstract

AI models and the future of genomic research and medicine: True sons of knowledge?

et al. 2021

View full text Add to dashboard Cite

The increasing availability of large‐scale, complex data has made research into how human genomes determine physiology in health and disease, as well as its application to drug development and medicine, an attractive field for artificial intelligence (AI) approaches. Looking at recent developments, we explore how such approaches interconnect and may conflict with needs for and notions of causal knowledge in molecular genetics and genomic medicine. We provide reasons to suggest that—while capable of generating predictive knowledge at unprecedented pace and scale—if and how these approaches will be integrated with prevailing causal concepts will not only determine the future of scientific understanding and self‐conceptions in these fields. But these questions will also be key to develop differentiated policies, such as for education and regulation, in order to harness societal benefits of AI for genomic research and medicine.

show abstract

Algorithms on regulatory lockdown in medicine

Abstract: Prioritize risk monitoring to address the “update problem”

Cited by 82 publications

References 8 publications

Health risks and potential remedies during prolonged lockdowns for coronavirus disease 2019 (COVID-19)

Health risks and potential remedies during prolonged lockdowns for coronavirus disease 2019 (COVID-19)

The need for a system view to regulate artificial intelligence/machine learning-based software as medical device

AI models and the future of genomic research and medicine: True sons of knowledge?

Contact Info

Product

Resources

About