AI Ethics Is Not a Panacea

McLennan, Stuart; Lee, Meredith M.; Fiske, Amelia; Celi, Leo Anthony

doi:10.1080/15265161.2020.1819470

Cited by 19 publications

(13 citation statements)

References 8 publications

(5 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Finally, it should be realized that ICU discharge and readmission decisions captured in AmsterdamUMCdb are subject to the preferences and biases of the intensivists practicing at our center and that the model learns this human subjectivity ( 40 ).…”

Section: Discussionmentioning

confidence: 99%

Explainable Machine Learning on AmsterdamUMCdb for ICU Discharge Decision Support: Uniting Intensivists and Data Scientists

Thoral

Fornasa²,

Bruin³

et al. 2021

Critical Care Explorations

View full text Add to dashboard Cite

Objectives: Unexpected ICU readmission is associated with longer length of stay and increased mortality. To prevent ICU readmission and death after ICU discharge, our team of intensivists and data scientists aimed to use AmsterdamUMCdb to develop an explainable machine learning–based real-time bedside decision support tool. Derivation Cohort: Data from patients admitted to a mixed surgical-medical academic medical center ICU from 2004 to 2016. Validation Cohort: Data from 2016 to 2019 from the same center. Prediction Model: Patient characteristics, clinical observations, physiologic measurements, laboratory studies, and treatment data were considered as model features. Different supervised learning algorithms were trained to predict ICU readmission and/or death, both within 7 days from ICU discharge, using 10-fold cross-validation. Feature importance was determined using SHapley Additive exPlanations, and readmission probability-time curves were constructed to identify subgroups. Explainability was established by presenting individualized risk trends and feature importance. Results: Our final derivation dataset included 14,105 admissions. The combined readmission/mortality rate within 7 days of ICU discharge was 5.3%. Using Gradient Boosting, the model achieved an area under the receiver operating characteristic curve of 0.78 (95% CI, 0.75–0.81) and an area under the precision-recall curve of 0.19 on the validation cohort ( n = 3,929). The most predictive features included common physiologic parameters but also less apparent variables like nutritional support. At a 6% risk threshold, the model showed a sensitivity (recall) of 0.72, specificity of 0.70, and a positive predictive value (precision) of 0.15. Impact analysis using probability-time curves and the 6% risk threshold identified specific patient groups at risk and the potential of a change in discharge management to reduce relative risk by 14%. Conclusions: We developed an explainable machine learning model that may aid in identifying patients at high risk for readmission and mortality after ICU discharge using the first freely available European critical care database, AmsterdamUMCdb. Impact analysis showed that a relative risk reduction of 14% could be achievable, which might have significant impact on patients and society. ICU data sharing facilitates collaboration between intensivists and data scientists to accelerate model development.

show abstract

Section: Discussionmentioning

confidence: 99%

Explainable Machine Learning on AmsterdamUMCdb for ICU Discharge Decision Support: Uniting Intensivists and Data Scientists

Thoral

Fornasa²,

Bruin³

et al. 2021

Critical Care Explorations

View full text Add to dashboard Cite

show abstract

“…ensuring that social, cultural, and ethical analyses are integrated into medical AI curriculum-as steps toward this goal. Clearly, technology alone is not the fix; large social problems that undergird structural inequality need to be addressed [72,73]. Nonetheless, researchers and educators can do their part to develop education and technologies that strive toward social justices.…”

Section: Discussionmentioning

confidence: 99%

Ensuring that biomedical AI benefits diverse populations

Zou

Schiebinger

2021

EBioMedicine

View full text Add to dashboard Cite

Artificial Intelligence (AI) can potentially impact many aspects of human health, from basic research discovery to individual health assessment. It is critical that these advances in technology broadly benefit diverse populations from around the world. This can be challenging because AI algorithms are often developed on non-representative samples and evaluated based on narrow metrics. Here we outline key challenges to biomedical AI in outcome design, data collection and technology evaluation, and use examples from precision health to illustrate how bias and health disparity may arise in each stage. We then suggest both short term approaches-more diverse data collection and AI monitoring-and longer term structural changes in funding, publications, and education to address these challenges.

show abstract

“…Sulla base di quanto detto sopra, andrebbe fatta qualche riflessione sulla tendenza attuale, che punta a superare il paradigma della ricerca clinica, sacrificandolo a un'illimitata fiducia nella sofisticatezza tecnologica (27)(28)(29). I problemi che stiamo affrontando necessitano di metodologie innovative, ma a condizione che siano inquadrate nel paradigma della ricerca clinica.…”

Section: Discussione E Proposta A) In Che Modo E a Quali Condizioni I Dati Dal Mondo Reale Possono Aiutarci A Curare Meglio?unclassified

Real-world data: how they can help to improve quality of care

Corrao

Alquati

Apolone

et al. 2021

Grhta

View full text Add to dashboard Cite

The current COVID pandemic crisis made it even clearer that the solutions to several questions that public health must face require the access to good quality data. Several issues of the value and potential of health data and the current critical issues that hinder access are discussed in this paper. In particular, the paper (i) focuses on “real-world data” definition; (ii) proposes a review of the real-world data availability in our country; (iii) discusses its potential, with particular focus on the possibility of improving knowledge on the quality of care provided by the health system; (iv) emphasizes that the availability of data alone is not sufficient to increase our knowledge, underlining the need that innovative analysis methods (e.g., artificial intelligence techniques) must be framed in the paradigm of clinical research; and (v) addresses some ethical issues related to their use. The proposal is to realize an alliance between organizations interested in promoting research aimed at collecting scientifically solid evidence to support the clinical governance of public health.

show abstract

AI Ethics Is Not a Panacea

Cited by 19 publications

References 8 publications

Explainable Machine Learning on AmsterdamUMCdb for ICU Discharge Decision Support: Uniting Intensivists and Data Scientists

Explainable Machine Learning on AmsterdamUMCdb for ICU Discharge Decision Support: Uniting Intensivists and Data Scientists

Ensuring that biomedical AI benefits diverse populations

Real-world data: how they can help to improve quality of care

Contact Info

Product

Resources

About