Diagnostic Performance of Machine Learning-Derived OSA Prediction Tools in Large Clinical and Community-Based Samples

Holfinger, Steven; Lyons, M. Melanie; Keenan, Brendan T.; Mazzotti, Diego R.; Mindel, Jesse; Maislin, Greg; Cistulli, Peter A.; Sutherland, Kate; McArdle, Nigel; Singh, Bhajan; Chen, Ning‐Hung; Gíslason, Þórarinn; Penzel, Thomas; Han, Fang; Schwab, Richard J.; Magalang, Ulysses J.

doi:10.1016/j.chest.2021.10.023

Cited by 19 publications

(16 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The field of breathing sleep disorders could also benefit from improving ML Technology, using both its application in early diagnosis and the analysis of predictive factors of response to medical or surgical treatment [ 7 , 8 , 23 ].…”

Section: Discussionmentioning

confidence: 99%

“…Holfinger et al assessed the diagnostic performance of OSA machine learning prediction tools using readily available data, such as age, sex, BMI, and race, and compared the efficacy with a STOP-BANG-based model [ 23 ]. The authors included a wide cohort of 17,448 subjects in a retrospective study, demonstrating that AUCs (95% CI) of the kernel support vector machine (0.66 [0.65–0.67]) were significantly higher than logistic regression ones (0.61 [0.60–0.62]).…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Machine Learning Identification of Obstructive Sleep Apnea Severity through the Patient Clinical Features: A Retrospective Study

Riela

Iannella

Lechien

et al. 2023

Life

View full text Add to dashboard Cite

Objectives: To evaluate the role of clinical scores assessing the risk of disease severity in patients with clinical suspicion of obstructive sleep apnea syndrome (OSA). The hypothesis was tested by applying artificial intelligence (AI) to demonstrate its effectiveness in distinguishing between mild–moderate OSA and severe OSA risk. Methods: A support vector machine model (SVM) was developed from the samples included in the analysis (N = 498), and they were split into 75% for training (N = 373) with the remaining for testing (N = 125). Two diagnostic thresholds were selected for OSA severity: mild to moderate (apnea–hypopnea index (AHI) ≥ 5 events/h and AHI < 30 events/h) and severe (AHI ≥ 30 events/h). The algorithms were trained and tested to predict OSA patient severity. Results: The sensitivity and specificity for the SVM model were 0.93 and 0.80 with an accuracy of 0.86; instead, the logistic regression full mode reported a value of 0.74 and 0.63, respectively, with an accuracy of 0.68. After backward stepwise elimination for features selection, the reduced logistic regression model demonstrated a sensitivity and specificity of 0.79 and 0.56, respectively, and an accuracy of 0.67. Conclusion: Artificial intelligence could be applied to patients with symptoms related to OSA to identify individuals with a severe OSA risk with clinical-based algorithms in the OSA framework.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Machine Learning Identification of Obstructive Sleep Apnea Severity through the Patient Clinical Features: A Retrospective Study

Riela

Iannella

Lechien

et al. 2023

Life

View full text Add to dashboard Cite

show abstract

“…OSA can be predicted using patient-reported symptoms such as sleepiness, snoring, and observed apnea ( Holfinger et al, 2022 ). Questionnaires based on typical symptoms are widely used to assess OSA risk ( Chung et al, 2008 ).…”

Section: Discussionmentioning

confidence: 99%

“…Significant advances made in targeting driver genes for cancer diseases suggests that new OSA prediction tools based on genetic data could contribute to identifying OSA risk and improving outcomes. Compared to traditional statistical tools such as logistic regression models, machine learning algorithms are more efficient at detecting multilevel, nonlinear relationships between variables and outcomes ( Holfinger et al, 2022 ).…”

Section: Discussionmentioning

confidence: 99%

“…Machine learning also is rapidly being applied to various clinical models of diseases. According to Holfinger et al ( Holfinger et al, 2022 ), machine learning derived prediction tools based on age, sex, race, and BMI provide better diagnosis for OSA than logistic regression when used in community-based samples. Using sleep parameters and endoscopic findings to develop machine learning models for predicting the success rate of sleep surgery also showed higher accuracy than subjective prediction by sleep surgeons ( Kim et al, 2021 ).…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Multiple Machine Learning Methods Reveal Key Biomarkers of Obstructive Sleep Apnea and Continuous Positive Airway Pressure Treatment

et al. 2022

View full text Add to dashboard Cite

Obstructive sleep apnea (OSA) is a worldwide health issue that affects more than 400 million people. Given the limitations inherent in the current conventional diagnosis of OSA based on symptoms report, novel diagnostic approaches are required to complement existing techniques. Recent advances in gene sequencing technology have made it possible to identify a greater number of genes linked to OSA. We identified key genes in OSA and CPAP treatment by screening differentially expressed genes (DEGs) using the Gene Expression Omnibus (GEO) database and employing machine learning algorithms. None of these genes had previously been implicated in OSA. Moreover, a new diagnostic model of OSA was developed, and its diagnostic accuracy was verified in independent datasets. By performing Single Sample Gene Set Enrichment Analysis (ssGSEA) and Counting Relative Subsets of RNA Transcripts (CIBERSORT), we identified possible immunologic mechanisms, which led us to conclude that patients with high OSA risk tend to have elevated inflammation levels that can be brought down by CPAP treatment.

show abstract

STREAMLINE: A Simple, Transparent, End-To-End Automated Machine Learning Pipeline Facilitating Data Analysis and Algorithm Comparison

Urbanowicz

Zhang

Cui

et al. 2023

Genetic and Evolutionary Computation

View full text Add to dashboard Cite

Objective: While machine learning (ML) includes a valuable array of tools for analyzing biomedical data with multivariate and complex underlying associations, significant time and expertise is required to assemble effective, rigorous, comparable, reproducible, and unbiased pipelines. Automated ML (AutoML) tools seek to facilitate ML application by automating a subset of analysis pipeline elements. In this study we develop and validate a Simple, Transparent, End-to-end Automated Machine Learning Pipeline (STREAMLINE) and apply it to investigate the added utility of photography-based phenotypes for predicting obstructive sleep apnea (OSA); a common and underdiagnosed condition associated with a variety of health, economic, and safety consequences. Methods: STREAMLINE is designed to tackle biomedical binary classification tasks while (1) avoiding common mistakes, (2) accommodating complex associations and common data challenges, and (3) allowing scalability, reproducibility, and model interpretation. It automates the majority of established, generalizable, and reliably automatable aspects of an ML analysis pipeline while incorporating cutting edge algorithms and providing opportunities for human-in-the-loop customization. We present a broadly refactored and extended release of STREAMLINE, validating and benchmarking performance across simulated and real-world datasets. Then we applied STREAMLINE to evaluate the utility of demographics (DEM), self-reported comorbidities (DX), symptoms (SYM), and photography-based craniofacial (CF) and intraoral (IO) anatomy measures in predicting 'any OSA' or 'moderate/severe OSA' using 3,111 participants from Sleep Apnea Global Interdisciplinary Consortium (SAGIC). Results: Benchmarking analyses validated the efficacy of STREAMLINE across data simulations with increasingly complex patterns of association including epistatic interactions and genetic heterogeneity. OSA analyses identified a significant increase in ROC-AUC when adding CF to DEM+DX+SYM to predict 'moderate/severe' OSA. Additionally, a consistent but non-significant increase in PRC-AUC was observed with the addition of each subsequent feature set to predict 'any OSA', with CF and IO yielding minimal improvements. Conclusion: STREAMLINE is an effective, rigorous, transparent, and easy-to-use AutoML approach to a comparative ML analysis that adheres to best practices in data science. Application of STREAM-LINE to OSA data suggests that CF features provide additional value in predicting moderate/severe OSA, but neither CF nor IO features meaningfully improved the prediction of 'any OSA' beyond established demographics, comorbidity and symptom characteristics.Keywords automated machine learning • obstructive sleep apnea • data science • predictive modeling • craniofacial traits • intraoral anatomy user-specification of feature types (which cannot always be reliably automated) and one-hot-encoding of categorical features for modeling, (3) engineering of 'missingness features' to consider missingness as a potentially informati...

show abstract

Diagnostic Performance of Machine Learning-Derived OSA Prediction Tools in Large Clinical and Community-Based Samples

Cited by 19 publications

References 46 publications

Machine Learning Identification of Obstructive Sleep Apnea Severity through the Patient Clinical Features: A Retrospective Study

Machine Learning Identification of Obstructive Sleep Apnea Severity through the Patient Clinical Features: A Retrospective Study

Multiple Machine Learning Methods Reveal Key Biomarkers of Obstructive Sleep Apnea and Continuous Positive Airway Pressure Treatment

STREAMLINE: A Simple, Transparent, End-To-End Automated Machine Learning Pipeline Facilitating Data Analysis and Algorithm Comparison

Contact Info

Product

Resources

About