Machine learning in medicine: should the pursuit of enhanced interpretability be abandoned?

Yoon, Chang Ho; Torrance, Robert; Scheinerman, Naomi

doi:10.1136/medethics-2020-107102

Cited by 64 publications

(55 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[17][18][19][20][21][22][23] , the DL structures can generate better feature representations and therefore are able to improve the TTE predictions. However, the low interpretability of DL-based models is a major obstacle to their application in healthcare where user (patients and clinicians) trust is critical [30][31][32] . The use of ROI can improve the TTE prediction performance of the CPH model while maintain its full interpretability.…”

Section: Tte Prediction Experiments On Synthetic Datamentioning

confidence: 99%

Clinical Time-to-Event Prediction Enhanced by Incorporating Compatible Related Outcomes

Gao

Cui

2022

Preprint

View full text Add to dashboard Cite

Accurate time-to-event (TTE) prediction of clinical outcomes from personal biomedical data is essential for precision medicine. It has become increasingly common that clinical datasets contain information for multiple related patient outcomes from comorbid diseases or multifaceted endpoints of a single disease. Various TTE models have been developed to handle competing risks that are related to mutually exclusive events. However, clinical outcomes are often non-competing and can occur at the same time or sequentially. Here we develop TTE prediction models with the capacity of incorporating data of compatible related clinical outcomes. We test our method on real and synthetic data and find that the incorporation of related auxiliary clinical outcomes can: 1) significantly improve the TTE prediction performance of convention Cox model while maintaining its interpretability; 2) further improve the performance of the state-of-the-art deep learning based models. While the auxiliary outcomes are utilized for model training, the model deployment is not limited by the availability of the auxiliary outcome data because the auxiliary outcome information is not required for the prediction of the primary outcome once the model is trained.

show abstract

Section: Tte Prediction Experiments On Synthetic Datamentioning

confidence: 99%

Clinical Time-to-Event Prediction Enhanced by Incorporating Compatible Related Outcomes

Gao

Cui

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Furthermore, another AI method similar to that of Attia et al reported similarly encouraging results 10 . However, although the results of such AI studies are promising, DNN-based AI techniques are inherently problematic in several respects, especially in relation to their lack of transparency and explainability, i.e., the 'black box' of AI 18,19 .Without the ability to know the exact features of the 12-lead ECG that are most important in a given DNN model's output, both interpretability and ethical accountability are compromised 45 . Moreover, it is effectively impossible for a clinician to identify, when critically evaluating the diagnostic output of a DNN-based AI model, the possible contribution to the result from methodological artifact or bias merely related to noise or to differing technical specifications between different ECG machines 46 .…”

Section: Comparison With Other Heart (Or Vascular) Agesmentioning

confidence: 99%

“…In addition, models in which the assessments were based on age predictions in healthy subjects will likely outperform models that were not. And finally, the use of more transparent regression models will also increase the ability of clinicians to better understand the origin of any unexpected result, and to thereafter relay it to the patient with a more convincing sense of trust and ethical accountability 45 .…”

Section: Comparison With Other Heart (Or Vascular) Agesmentioning

confidence: 99%

Heart age estimated using explainable advanced electrocardiography

Lindow

Palencia-Lamela

Schlegel

2021

Preprint

View full text Add to dashboard Cite

Background: Electrocardiographic (ECG) Heart Age conveying cardiovascular risk has been estimated by both Bayesian and artificial intelligence approaches. We hypothesized that explainable measures from the 10-second 12-lead ECG could successfully predict Bayesian ECG Heart Age. Methods: Advanced analysis was performed on ECGs from healthy subjects and patients with cardiovascular risk or proven heart disease. Regression models were used to predict a Bayesian 5-minute ECG Heart Age from the standard resting 10-second 12-lead ECG. The difference between 10-second ECG Heart Age and chronological age was compared. Results: In total, 2,771 subjects were included (n=1682 healthy volunteers, n=305 with cardiovascular risk factors, n=784 with cardiovascular disease). Overall, 10-second Heart Age showed strong agreement with the 5-minute Heart Age (R2=0.94, p<0.001, mean±SD bias 0.0±5.1 years). The difference between 10-second ECG Heart Age and chronological age was 0.0±5.7 years in healthy individuals, 7.4±7.3 years in subjects with cardiovascular risk factors (p<0.001), and 14.3±9.2 years for patients with cardiovascular disease (p<0.001). Conclusions: ECG Heart Age can be accurately estimated from a 10-second 12-lead ECG in a transparent and explainable fashion based on known ECG measures, without artificial intelligence techniques. The difference between ECG Heart Age and chronological age increases markedly with cardiovascular risk and disease.

show abstract

“…"It is very hard to trust any model ... without having transparency into how those models operate" [67]. The perception that interpretability is critical to trust is incredibly widespread [7,16,47,55,64,66,75,86]. However, progress on interpretability has been difficult to measure, as lack of a clear consensus definitions have exposed interpretability's inherent subjectivity and field-specific meanings [23,43,47,48,78].…”

Section: Introductionmentioning

confidence: 99%

Trust in AI: Interpretability is not necessary or sufficient, while black-box interaction is necessary and sufficient

Shen¹

2022

Preprint

View full text Add to dashboard Cite

The problem of human trust in artificial intelligence is one of the most fundamental problems in applied machine learning. Our processes for evaluating AI trustworthiness have substantial ramifications for ML's impact on science, health, and humanity, yet confusion surrounds foundational concepts. What does it mean to trust an AI, and how do humans assess AI trustworthiness? What are the mechanisms for building trustworthy AI? And what is the role of interpretable ML in trust?Here, we draw from statistical learning theory and sociological lenses on human-automation trust to motivate an AI-as-tool framework, which distinguishes human-AI trust from human-AI-human trust. Evaluating an AI's contractual trustworthiness involves predicting future model behavior using behavior certificates (BCs) that aggregate behavioral evidence from diverse sources including empirical out-of-distribution and out-of-task evaluation and theoretical proofs linking model architecture to behavior.We clarify the role of interpretability in trust with a ladder of model access. Interpretability (level 3) is not necessary or even sufficient for trust, while the ability to run a black-box model at-will (level 2) is necessary and sufficient. While interpretability can offer benefits for trust, it can also incur costs. We clarify ways interpretability can contribute to trust, while questioning the perceived centrality of interpretability to trust in popular discourse.How can we empower people with tools to evaluate trust? Instead of trying to understand how a model works, we argue for understanding how a model behaves. Instead of opening up black boxes, we should create more behavior certificates that are more correct, relevant, and understandable. We discuss how to build trusted and trustworthy AI responsibly with contract-aware model design, robustness testing, and favoring trust calibration over maximization. CCS Concepts: • Computing methodologies → Artificial intelligence; • Social and professional topics → Computing / technology policy; • Human-centered computing → Interaction design theory, concepts and paradigms.

show abstract

Machine learning in medicine: should the pursuit of enhanced interpretability be abandoned?

Cited by 64 publications

References 28 publications

Clinical Time-to-Event Prediction Enhanced by Incorporating Compatible Related Outcomes

Clinical Time-to-Event Prediction Enhanced by Incorporating Compatible Related Outcomes

Heart age estimated using explainable advanced electrocardiography

Trust in AI: Interpretability is not necessary or sufficient, while black-box interaction is necessary and sufficient

Contact Info

Product

Resources

About