Bias and ethical considerations in machine learning and the automation of perioperative risk assessment

O’Reilly-Shah, Vikas N; Gentry, Katherine R.; Walters, Andrew M.; Zivot, Joel; Anderson, Cole; Tighe, Patrick J.

doi:10.1016/j.bja.2020.07.040

Cited by 42 publications

(25 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As AI becomes more pervasive in both public and personal health across diverse populations, there have been increasing concerns, and related examples, of AI solutions leading to inadvertent bias of modeling results (45)(46)(47)(48). Broadly, such bias can originate from the data used for model training and testing, as well as the mechanics of the model itself (49).…”

Section: Potential For Bias In MLmentioning

confidence: 99%

Accessing Artificial Intelligence for Clinical Decision-Making

Giordano

Brennan

Mohamed

et al. 2021

Front. Digit. Health

125

View full text Add to dashboard Cite

Advancements in computing and data from the near universal acceptance and implementation of electronic health records has been formative for the growth of personalized, automated, and immediate patient care models that were not previously possible. Artificial intelligence (AI) and its subfields of machine learning, reinforcement learning, and deep learning are well-suited to deal with such data. The authors in this paper review current applications of AI in clinical medicine and discuss the most likely future contributions that AI will provide to the healthcare industry. For instance, in response to the need to risk stratify patients, appropriately cultivated and curated data can assist decision-makers in stratifying preoperative patients into risk categories, as well as categorizing the severity of ailments and health for non-operative patients admitted to hospitals. Previous overt, traditional vital signs and laboratory values that are used to signal alarms for an acutely decompensating patient may be replaced by continuously monitoring and updating AI tools that can pick up early imperceptible patterns predicting subtle health deterioration. Furthermore, AI may help overcome challenges with multiple outcome optimization limitations or sequential decision-making protocols that limit individualized patient care. Despite these tremendously helpful advancements, the data sets that AI models train on and develop have the potential for misapplication and thereby create concerns for application bias. Subsequently, the mechanisms governing this disruptive innovation must be understood by clinical decision-makers to prevent unnecessary harm. This need will force physicians to change their educational infrastructure to facilitate understanding AI platforms, modeling, and limitations to best acclimate practice in the age of AI. By performing a thorough narrative review, this paper examines these specific AI applications, limitations, and requisites while reviewing a few examples of major data sets that are being cultivated and curated in the US.

show abstract

Section: Potential For Bias In MLmentioning

confidence: 99%

Accessing Artificial Intelligence for Clinical Decision-Making

Giordano

Brennan

Mohamed

et al. 2021

Front. Digit. Health

125

View full text Add to dashboard Cite

show abstract

“…However, there is a growing awareness in the community that the presence of different sources of bias significantly decreases the overall generalisation ability of the models, leading to overestimated model performance reported in internal validation compared to evaluation on independent test data ( Soneson, Gerster, Delorenzi, 2014 , Cohen, Hashir, Brooks, Bertrand , Zech, Badgeley, Liu, Costa, Titano, Oermann, 2018 , Maguolo, Nanni ). In addition, numerous journal editorials are calling for better development, evaluation and reporting practices of machine learning models aimed for clinical application ( Mateen, Liley, Denniston, Holmes, Vollmer, 2020 , Nagendran, Chen, Lovejoy, Gordon, Komorowski, Harvey, Topol, Ioannidis, Collins, Maruthappu, 2020 , Campbell, Lee, Abrmoff, Keane, Ting, Lum, Chiang, 2020 , Health, 2020 , O’Reilly-Shah, Gentry, Walters, Zivot, Anderson, Tighe, 2020 , Health, 2019 , Stevens, Mortazavi, Deo, Curtis, Kao, 2020 ). Underneath, there are growing concerns about ethics and the risk of harmful outcomes of using AI in medical applications ( Campolo, Sanfilippo, Whittaker, Crawford, 2018 , Geis, Brady, Wu, Spencer, Ranschaert, Jaremko, Langer, Borondy Kitts, Birch, Shields, van den Hoven van Genderen, Kotter, Wawira Gichoya, Cook, Morgan, Tang, Safdar, Kohli, 2019 , Brady, Neri, 2020 ).…”

Section: Introductionmentioning

confidence: 99%

Public Covid-19 X-ray datasets and their impact on model bias – A systematic review of a significant problem

Cruz

Bossa

Sölter

et al. 2021

Medical Image Analysis

View full text Add to dashboard Cite

show abstract

“…But in recent years, a competing perspective has emerged --the perspective that algorithms often encode the biases of their developers or the surrounding society, producing predictions or inferences that are clearly discriminatory towards specific groups. Examples of algorithmic bias cross contexts, from criminal justice (Angwin et al, 2016), to medicine (O'Reilly-Shah et al, 2020), to computer vision (Klare et al, 2012), to hiring (Garcia, 2016). These limitations appear --and are particularly salient --for high-stakes decisions such as predicting recidivism (Angwin et al, 2016) or administering anaesthesia (O'Reilly-Shah et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

Algorithmic Bias in Education

Baker¹,

Hawn²

2021

Preprint

View full text Add to dashboard Cite

Draft Preprint. In this paper, we review algorithmic bias in education, discussing the causes of that bias and reviewing the empirical literature on the specific ways that algorithmic bias is known to have manifested in education. While other recent work has reviewed mathematical definitions of fairness and expanded algorithmic approaches to reducing bias, our review focuses instead on solidifying the current understanding of the concrete impacts of algorithmic bias in education—which groups are known to be impacted and which stages and agents in the development and deployment of educational algorithms are implicated. We discuss theoretical and formal perspectives on algorithmic bias, connect those perspectives to the machine learning pipeline, and review metrics for assessing bias. Next, we review the evidence around algorithmic bias in education, beginning with the most heavily-studied categories of race/ethnicity, gender, and nationality, and moving to the available evidence of bias for less-studied categories, such as socioeconomic status, disability, and military-connected status. Acknowledging the gaps in what has been studied, we propose a framework for moving from unknown bias to known bias and from fairness to equity. We discuss obstacles to addressing these challenges and propose four areas of effort for mitigating and resolving the problems of algorithmic bias in AIED systems and other educational technology.

show abstract

Bias and ethical considerations in machine learning and the automation of perioperative risk assessment

Cited by 42 publications

References 17 publications

Accessing Artificial Intelligence for Clinical Decision-Making

Accessing Artificial Intelligence for Clinical Decision-Making

Public Covid-19 X-ray datasets and their impact on model bias – A systematic review of a significant problem

Algorithmic Bias in Education

Contact Info

Product

Resources

About