Machine intelligence in healthcare—perspectives on trustworthiness, explainability, usability, and transparency

Cutillo, Christine M.; Sharma, Karlie R.; Foschini, Luca; Kundu, Shinjini; Mackintosh, Maxine; Mandl, Kenneth D.

doi:10.1038/s41746-020-0254-2

Cited by 186 publications

(126 citation statements)

References 25 publications

Supporting

Mentioning

123

Contrasting

Unclassified

Order By: Relevance

“…As clinicians identify instances in which the system performs poorly, they can report these cases back to developers to foster quality assurance and product improvement. Given these considerations, explainability may be a key driver for the uptake of AI-driven CDSS in clinical practice, as trust in these systems is not yet established [22,23]. Here, it is important to note that any use of AI-based CDSS may influence a physician in reaching a decision.…”

Section: The Medical Perspectivementioning

confidence: 99%

Explainability for artificial intelligence in healthcare: a multidisciplinary perspective

Amann

Blasimme

Frey

et al. 2020

BMC Med Inform Decis Mak

746

417

View full text Add to dashboard Cite

Background Explainability is one of the most heavily debated topics when it comes to the application of artificial intelligence (AI) in healthcare. Even though AI-driven systems have been shown to outperform humans in certain analytical tasks, the lack of explainability continues to spark criticism. Yet, explainability is not a purely technological issue, instead it invokes a host of medical, legal, ethical, and societal questions that require thorough exploration. This paper provides a comprehensive assessment of the role of explainability in medical AI and makes an ethical evaluation of what explainability means for the adoption of AI-driven tools into clinical practice. Methods Taking AI-based clinical decision support systems as a case in point, we adopted a multidisciplinary approach to analyze the relevance of explainability for medical AI from the technological, legal, medical, and patient perspectives. Drawing on the findings of this conceptual analysis, we then conducted an ethical assessment using the “Principles of Biomedical Ethics” by Beauchamp and Childress (autonomy, beneficence, nonmaleficence, and justice) as an analytical framework to determine the need for explainability in medical AI. Results Each of the domains highlights a different set of core considerations and values that are relevant for understanding the role of explainability in clinical practice. From the technological point of view, explainability has to be considered both in terms how it can be achieved and what is beneficial from a development perspective. When looking at the legal perspective we identified informed consent, certification and approval as medical devices, and liability as core touchpoints for explainability. Both the medical and patient perspectives emphasize the importance of considering the interplay between human actors and medical AI. We conclude that omitting explainability in clinical decision support systems poses a threat to core ethical values in medicine and may have detrimental consequences for individual and public health. Conclusions To ensure that medical AI lives up to its promises, there is a need to sensitize developers, healthcare professionals, and legislators to the challenges and limitations of opaque algorithms in medical AI and to foster multidisciplinary collaboration moving forward.

show abstract

Section: The Medical Perspectivementioning

confidence: 99%

Explainability for artificial intelligence in healthcare: a multidisciplinary perspective

Amann

Blasimme

Frey

et al. 2020

BMC Med Inform Decis Mak

746

417

View full text Add to dashboard Cite

show abstract

“…A black-box model does not lend itself to interpretable and meaningful representations which may also make the model more susceptible for adversarial attacks [27], [28]. Recently, it has become increasingly clear that deep neural networks (DNNs) have the potential to identify biologically meaningful molecular representations directly from data [16], [29] and to revolutionize medicine [30], [31].…”

Section: Discussionmentioning

confidence: 99%

White-box Deep Neural Network Prediction of Genome-Wide Transcriptome Signatures

Magnusson¹,

Jn²,

Gustafsson³

2021

Preprint

View full text Add to dashboard Cite

Prediction algorithms for protein or gene structures, including transcription factor binding from sequence information, have been transformative in understanding gene regulation. Here we ask whether human transcriptomic profiles can be predicted solely from the expression of transcription factors (TFs). To this end, we explore whether a neural network (NN) could predict the transcriptome from TFs. Using at least one hidden layer, we find that the expression of 1,600 TFs can explain >95% of variance in 25,000 genes. Using the light-up technique to inspect the trained NN, we find an overrepresentation of known TF-gene regulations. Furthermore, the learned prediction network has a hierarchical organization. A smaller set of around 125 core TFs could explain close to 80% of the variance. Interestingly, reducing the number of TFs below 500 induces a rapid decline in prediction performance. Next, we evaluated the prediction model using transcriptional data from 22 human diseases. The TFs were sufficient to predict the target genes' dysregulation (rho=0.61, P < 10^-216). By inspecting the model, key causative TFs could be extracted for subsequent validation using disease-associated genetic variants. In conclusion, we demonstrate the construction of an interpretable neural network predictor. Analysis of the predictors revealed key TFs that were inducing transcriptional changes during disease.

show abstract

“…There are some limitations of the current study that should be addressed with regard to ML. Adequate enrollment by sex and race are common sampling issues in healthcare ML [43,44]. Details of this cohort have been extensively published and include even distributions by age and sex.…”

Section: Discussionmentioning

confidence: 99%

Predicting Olfactory Loss In Chronic Rhinosinusitis Using Machine Learning

Ramakrishnan¹,

Arbet²,

Mace

et al. 2020

Preprint

View full text Add to dashboard Cite

Objective Compare machine learning (ML) based predictive analytics methods compared to traditional logistic regression in classification of olfactory dysfunction in chronic rhinosinusitis (CRS-OD), and identify predictors within a large multi-institutional cohort of refractory CRS patients. Methods Adult CRS patients enrolled in a prospective, multi-institutional, observational cohort study were assessed for baseline CRS-OD using a smell identification test (SIT) or brief SIT (bSIT). Four different ML methods were compared to traditional logistic regression for classification of CRS normosmics versus CRS-OD. Results Data were collected for 611 study participants who met inclusion criteria between April 2011 and July 2015. 34% of enrolled patients demonstrated olfactory loss on objective testing. Differences between CRS normosmics and those with smell loss included objective disease measures (CT and endoscopy scores), age, sex, prior surgeries, socioeconomic status, steroid use, polyp presence, asthma, and aspirin sensitivity. Most ML methods outperformed traditional logistic regression in terms of predictive ability. Top predictors include known factors reported in the literature, as well as several socioeconomic factors. Conclusion Olfactory dysfunction is a variable phenomenon within a large multicenter cohort of refractory CRS patients. ML methods outperform traditional logistic regression in classification of normosmia versus smell loss in CRS, and are able to include numerous risk factors into prediction models. These results carry implications for basic science and clinical research in hyposmia secondary to sinonasal disease, the most common cause of persistent olfactory loss in the general population.

show abstract

Machine intelligence in healthcare—perspectives on trustworthiness, explainability, usability, and transparency

Cited by 186 publications

References 25 publications

Explainability for artificial intelligence in healthcare: a multidisciplinary perspective

Explainability for artificial intelligence in healthcare: a multidisciplinary perspective

White-box Deep Neural Network Prediction of Genome-Wide Transcriptome Signatures

Predicting Olfactory Loss In Chronic Rhinosinusitis Using Machine Learning

Contact Info

Product

Resources

About