A universal AutoScore framework to develop interpretable scoring systems for predicting common types of clinical outcomes

Xie, Feng; Ning, Yilin; Liu, Mingxuan; Li, Siqi; Saffari, Seyed Ehsan; Yuan, Han; Volovici, Victor; Ting, Daniel Shu Wei; Goldstein, Benjamin A.; Ong, Marcus Eng Hock; Vaughan, Roger; Chakraborty, Bibhas; Liu, Nan

doi:10.1016/j.xpro.2023.102302

Cited by 5 publications

(3 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The AutoScore framework was employed to develop the LC risk score [19]. This tool enables the automatic development of a clinically useful scoring model.…”

Section: Statistical Analysis and Development Of A Risk Scorementioning

confidence: 99%

“…In addition, besides COVID-19 disease itself, subsequent complications have become a significant health P r e p r i n t problem that persists in a substantial percentage of people. Currently, the definition of complications after COVID- 19 has not yet been fully systematised, but WHO defines this disease as a post-COVID condition (also known as long-COVID), which includes "the continuation or development of new symptoms 3 months after the initial SARS-CoV-2 infection, with these symptoms lasting for at least 2 months with no other explanation" [2].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

"LC Risk Score" - development and evaluation of a scale for assessing the risk of developing long COVID

Babicki,

Lejawa,

Osadnik

et al. 2024

Arch Med Sci

View full text Add to dashboard Cite

IntroductionObjective: To create a valuable practical tool for evaluating the risk of LC development.Material and methods1150 patients from the Polish STOP-COVID registry (PoLoCOV study) were used to develop the risk score. The patients were ill between 03/2020 and 04/2022. To develop a clinically useful scoring model. The LC risk score was generated using the machine learning-based framework AutoScore. Patient data were first randomised into a training (70% of output) and a test (30% of output) cohorts. Due to relatively small study group, cross-validation was used. Model predictive ability was evaluated based on the ROC curve and the AUC value. The result of the risk score for a given patient was the total value of points assigned to selected variables.ResultsTo create long COVID Risk Score, eight variables were ultimately selected due to their significance and clinical value. Female gender significantly contributed to higher final outcome values, with age range 40-49, BMI <18.5 kg/m2, hospitalisation during active disease, arthralgia, myalgia as well as loss of taste and smell during infection, COVID-19 symptoms lasting at least 14 days, and unvaccinated status. The final predictive value of the developed LC risk score for a cut-off of 58 points was AUC=0.630 (95% CI: 0.571-0.688) with sensitivity - 39.80%, specificity - 85.1%, positive predictive value - 80.8%, and negative predictive value 47.3%.ConclusionsConclusions: The LC risk score might be a practical and undemanding utility that employs basic sociodemographic data, vaccination status, and symptoms during COVID-19 to assess the risk of long-COVID.

show abstract

“…The AutoScore framework was employed to develop the LC risk score [19]. This tool enables the automatic development of a clinically useful scoring model.…”

Section: Statistical Analysis and Development Of A Risk Scorementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

"LC Risk Score" - development and evaluation of a scale for assessing the risk of developing long COVID

Babicki,

Lejawa,

Osadnik

et al. 2024

Arch Med Sci

View full text Add to dashboard Cite

show abstract

“…It includes a set of methods that attempts to combine a complex black-box model with an inherently interpretable model to build an interpretable model that achieves comparable performance to the black-box model. The AutoScore framework is an example of this hybrid interpretable model approach [ 35 , 36 ]. In this framework, development of an ML model is complicated, but the final result is familiar to users [ 5 , 35 , 37 ].…”

Section: How Does Explainable Ai Work?mentioning

confidence: 99%

Explainable artificial intelligence in emergency medicine: an overview

Okada,

Ning,

Ong

2023

Clin Exp Emerg Med

Self Cite

View full text Add to dashboard Cite

Artificial intelligence (AI) and machine learning (ML) have potential to revolutionize emergency medical care by enhancing triage systems, improving diagnostic accuracy, refining prognostication, and optimizing various aspects of clinical care. However, as clinicians often lack AI expertise, they might perceive AI as a “black box,” leading to trust issues. To address this, “explainable AI,” which teaches AI functionalities to end-users, is important. This review presents the definitions, importance, and role of explainable AI, as well as potential challenges in emergency medicine. First, we introduce the terms explainability, interpretability, and transparency of AI models. These terms sound similar but have different roles in discussion of AI. Second, we indicate that explainable AI is required in clinical settings for reasons of justification, control, improvement, and discovery and provide examples. Third, we describe three major categories of explainability: pre-modeling explainability, interpretable models, and post-modeling explainability and present examples (especially for post-modeling explainability), such as visualization, simplification, text justification, and feature relevance. Last, we show the challenges of implementing AI and ML models in clinical settings and highlight the importance of collaboration between clinicians, developers, and researchers. This paper summarizes the concept of “explainable AI” for emergency medicine clinicians. This review may help clinicians understand explainable AI in emergency contexts.

show abstract

Automated machine learning with interpretation: A systematic review of methodologies and applications in healthcare

Yuan,

Yu,

Xie

et al. 2024

Medicine Advances

View full text Add to dashboard Cite

Machine learning (ML) has achieved substantial success in performing healthcare tasks in which the configuration of every part of the ML pipeline relies heavily on technical knowledge. To help professionals with borderline expertise to better use ML techniques, Automated ML (AutoML) has emerged as a prospective solution. However, most models generated by AutoML are black boxes that are challenging to comprehend and deploy in healthcare settings. We conducted a systematic review to examine AutoML with interpretation systems for healthcare. We searched four databases (MEDLINE, EMBASE, Web of Science, and Scopus) complemented with seven prestigious ML conferences (AAAI, ACL, ICLR, ICML, IJCAI, KDD, and NeurIPS) that reported AutoML with interpretation for healthcare before September 1, 2023. We included 118 articles related to AutoML with interpretation in healthcare. First, we illustrated AutoML techniques used in the included publications, including automated data preparation, automated feature engineering, and automated model development, accompanied by a real‐world case study to demonstrate the advantages of AutoML over classic ML. Then, we summarized interpretation methods: feature interaction and importance, data dimensionality reduction, intrinsically interpretable models, and knowledge distillation and rule extraction. Finally, we detailed how AutoML with interpretation has been used for six major data types: image, free text, tabular data, signal, genomic sequences, and multi‐modality. To some extent, AutoML with interpretation provides effortless development and improves users' trust in ML in healthcare settings. In future studies, researchers should explore automated data preparation, seamless integration of automation and interpretation, compatibility with multi‐modality, and utilization of foundation models.

show abstract

A universal AutoScore framework to develop interpretable scoring systems for predicting common types of clinical outcomes

Cited by 5 publications

References 37 publications

"LC Risk Score" - development and evaluation of a scale for assessing the risk of developing long COVID

"LC Risk Score" - development and evaluation of a scale for assessing the risk of developing long COVID

Explainable artificial intelligence in emergency medicine: an overview

Automated machine learning with interpretation: A systematic review of methodologies and applications in healthcare

Contact Info

Product

Resources

About