Minimum information about clinical artificial intelligence modeling: the MI-CLAIM checklist

Norgeot, Beau; Quer, Giorgio; Beaulieu‐Jones, Brett K.; Torkamani, Ali; Dias, Raquel; Gianfrancesco, Milena; Arnaout, Rima; Kohane, Isaac S.; Saria, Suchi; Topol, Eric J.; Obermeyer, Ziad; Yu, Bin; Butte, Atul J.

doi:10.1038/s41591-020-1041-y

Cited by 286 publications

(212 citation statements)

References 17 publications

Supporting

Mentioning

212

Contrasting

Order By: Relevance

“…One way to argue for external validity is to attempt external validation, that is, to reproduce the results obtained on one dataset on a different but related dataset. Recent voices that urge the community to give external validation a higher priority across many domains (Debray et al, 2015;Krauss, 2018;Norgeot et al, 2020) are very much in accordance with Yu and Kumbier's (2020) call to statisticians to broaden the scope of their concern from data modelling to the entire data science life cycle as part of the PCS framework. This can be seen not only as one more predictive and stability check under the PCS framework but also as a special case of 'transfer learning' where the desiderata is the transferability of the conclusions or findings from one dataset to other related datasets.…”

Section: Complementary Analysis With the Approve Studymentioning

confidence: 89%

Stable Discovery of Interpretable Subgroups via Calibration in Causal Studies

Dwivedi

Yan

Park

et al. 2020

Int Statistical Rev

Self Cite

View full text Add to dashboard Cite

Building on Yu and Kumbier's predictability, computability and stability (PCS) framework and for randomised experiments, we introduce a novel methodology for Stable Discovery of Interpretable Subgroups via Calibration (StaDISC), with large heterogeneous treatment effects. StaDISC was developed during our re-analysis of the 1999-2000 VIGOR study, an 8076-patient randomised controlled trial that compared the risk of adverse events from a then newly approved drug, rofecoxib (Vioxx), with that from an older drug naproxen. Vioxx was found to, on average and in comparison with naproxen, reduce the risk of gastrointestinal events but increase the risk of thrombotic cardiovascular events. Applying StaDISC, we fit 18 popular conditional average treatment effect (CATE) estimators for both outcomes and use calibration to demonstrate their poor global performance. However, they are locally well-calibrated and stable, enabling the identification of patient groups with larger than (estimated) average treatment effects. In fact, StaDISC discovers three clinically interpretable subgroups each for the gastrointestinal outcome (totalling 29.4% of the study size) and the thrombotic cardiovascular outcome (totalling 11.0%). Complementary analyses of the found subgroups using the 2001-2004 APPROVe study, a separate independently conducted randomised controlled trial with 2587 patients, provide further supporting evidence for the promise of StaDISC.

show abstract

Section: Complementary Analysis With the Approve Studymentioning

confidence: 89%

Stable Discovery of Interpretable Subgroups via Calibration in Causal Studies

Dwivedi

Yan

Park

et al. 2020

Int Statistical Rev

Self Cite

View full text Add to dashboard Cite

show abstract

“…Since MSC therapies showed beneficial effects with complex undetermined components, AI may be well-suited to analyzing and revealing essential elements. Companies such as Merck, GSK, and Roche have developed partnerships with AI companies to construct suitable platforms [ 189 , 190 ]. However, the drug discovery process with AI is a long shot, which need to be verified in clinical trials.…”

Section: Advances and Perspectives To Overcome Challenges In Msc Clinmentioning

confidence: 99%

Challenges and advances in clinical applications of mesenchymal stromal cells

et al. 2021

View full text Add to dashboard Cite

Mesenchymal stromal cells (MSCs), also known as mesenchymal stem cells, have been intensely investigated for clinical applications within the last decades. However, the majority of registered clinical trials applying MSC therapy for diverse human diseases have fallen short of expectations, despite the encouraging pre-clinical outcomes in varied animal disease models. This can be attributable to inconsistent criteria for MSCs identity across studies and their inherited heterogeneity. Nowadays, with the emergence of advanced biological techniques and substantial improvements in bio-engineered materials, strategies have been developed to overcome clinical challenges in MSC application. Here in this review, we will discuss the major challenges of MSC therapies in clinical application, the factors impacting the diversity of MSCs, the potential approaches that modify MSC products with the highest therapeutic potential, and finally the usage of MSCs for COVID-19 pandemic disease.

show abstract

“…The MI‐CLAIM checklist highlights metrics for evaluating algorithm performance addition to those for model clinical utility 6 . For the former task, area under receiver operating characteristic (AU‐ROC, otherwise known as c‐statistic), which is synonymous to the area under a model‐derived sensitivity, 1‐specificity curve is most commonly used, but it is important to recall caveats to its use.…”

Section: What Is Ml?mentioning

confidence: 99%

“…In contrast, model “calibration” provides information on the reliability of risk estimates for individuals, matters significantly for patient decision‐making, and is often not reported 8,9 . Model clinical utility is also dependent on benchmarks that can be applied in the interpretation at the level of individual patients, namely, positive/negative predictive values, numbers needed to treat/harm, sensitivity, and specificity 6 …”

Section: What Is Ml?mentioning

confidence: 99%