Ultrasound-based radiomics XGBoost model to assess the risk of central cervical lymph node metastasis in patients with papillary thyroid carcinoma: Individual application of SHAP

Shi, Yan; Zou, Ying; Liu, Jihua; Wang, Yuanyuan; Chen, Yingbin; Sun, Fang; Yang, Zhi; Cui, Guanghe; Zhu, Xijun; Chen, Xu; Liu, Feifei

doi:10.3389/fonc.2022.897596

Cited by 11 publications

(10 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…All 47 studies are summarized in Supplementary Table 1 [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48] and were divided based on the tumor of origin: breast cancer, head and neck cancer, and cervical cancer. Most of the papers were published between 2019 and 2022 (n = 19 articles), and the analyses were conducted using MATLAB, R software, SPSS, OriginPro, a papillary thyroid carcinoma (PTC), cervical LN metastasis (LNM) prediction system (custom-built, n = 3), MIPAV, and Mazda.…”

Section: Resultsmentioning

confidence: 99%

“…Feature extraction was mainly performed with PyRadiomics, ITK-SNAP, LifeX, MedCalc, ImageJ, PASW and Prism. The values of the area under the curve (AUC), sensitivity, specificity, and accuracy are summarized in ▶ Table 1 [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48].…”

Section: Resultsmentioning

confidence: 99%

“…Head and neck cancer studies mainly focused on thyroid cancer, specifically PTC and head and neck squamous cell carcinoma (HNSCC). We identified 20 studies investigating LN status in patients with PTC [13,14,15,16,17,18,19,20,21,22,23,32,35,36,37,38,46,47,48], one study on follicular thyroid carcinoma [25], one on treatment outcome prediction in head and neck malignancies [44], and three studies on HNSCC [29,30,31].…”

Section: Head and Neck Cancermentioning

confidence: 99%

See 2 more Smart Citations

Quantitative ultrasound radiomics analysis to evaluate lymph nodes in patients with cancer: a systematic review

Guerrisi,

Miseo,

Falcone

et al. 2024

Ultraschall Med

View full text Add to dashboard Cite

This systematic review aims to evaluate the role of ultrasound (US) radiomics in assessing lymphadenopathy in patients with cancer and the ability of radiomics to predict metastatic lymph node involvement. A systematic literature search was performed in the PubMed (MEDLINE), Cochrane Central Register of Controlled Trials (CENTRAL), and EMBASE (Ovid) databases up to June 13, 2023. 42 articles were included in which the lymph node mass was assessed with a US exam, and the analysis was performed using radiomics methods. From the survey of the selected articles, experimental evidence suggests that radiomics features extracted from US images can be a useful tool for predicting and characterizing lymphadenopathy in patients with breast, head and neck, and cervical cancer. This noninvasive and effective method allows the extraction of important information beyond mere morphological characteristics, extracting features that may be related to lymph node involvement. Future studies are needed to investigate the role of US-radiomics in other types of cancers, such as melanoma.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Head and Neck Cancermentioning

confidence: 99%

See 1 more Smart Citation

Quantitative ultrasound radiomics analysis to evaluate lymph nodes in patients with cancer: a systematic review

Guerrisi,

Miseo,

Falcone

et al. 2024

Ultraschall Med

View full text Add to dashboard Cite

show abstract

“…While SHAP was often used to explain features in machine learning algorithms and neural network models ( Bang et al, 2021 ; Park et al, 2022 ; Shaji et al, 2022 ; Shi et al, 2022 ), SHAP analysis of logistic regression models had not yet been mentioned. Although logistic regression algorithms were simpler and more explicit than other machine learning algorithms and neural networks, logistic regression models were more challenging to understand than they may seem.…”

Section: Discussionmentioning

confidence: 99%

Development and validation of an interpretable radiomic nomogram for severe radiation proctitis prediction in postoperative cervical cancer patients

et al. 2023

View full text Add to dashboard Cite

BackgroundRadiation proctitis is a common complication after radiotherapy for cervical cancer. Unlike simple radiation damage to other organs, radiation proctitis is a complex disease closely related to the microbiota. However, analysis of the gut microbiota is time-consuming and expensive. This study aims to mine rectal information using radiomics and incorporate it into a nomogram model for cheap and fast prediction of severe radiation proctitis prediction in postoperative cervical cancer patients.MethodsThe severity of the patient’s radiation proctitis was graded according to the RTOG/EORTC criteria. The toxicity grade of radiation proctitis over or equal to grade 2 was set as the model’s target. A total of 178 patients with cervical cancer were divided into a training set (n = 124) and a validation set (n = 54). Multivariate logistic regression was used to build the radiomic and non-raidomic models.ResultsThe radiomics model [AUC=0.6855(0.5174-0.8535)] showed better performance and more net benefit in the validation set than the non-radiomic model [AUC=0.6641(0.4904-0.8378)]. In particular, we applied SHapley Additive exPlanation (SHAP) method for the first time to a radiomics-based logistic regression model to further interpret the radiomic features from case-based and feature-based perspectives. The integrated radiomic model enables the first accurate quantitative assessment of the probability of radiation proctitis in postoperative cervical cancer patients, addressing the limitations of the current qualitative assessment of the plan through dose-volume parameters only.ConclusionWe successfully developed and validated an integrated radiomic model containing rectal information. SHAP analysis of the model suggests that radiomic features have a supporting role in the quantitative assessment of the probability of radiation proctitis in postoperative cervical cancer patients.

show abstract

“…At the higher end, an AUROC of 0.947 implies near perfect fit, while an AUROC of 0.771, while still significantly more predictive than random chance, provides a much decreased level of confidence in the predictions of the model. This highlights a potential issue in replication of machine-learning methods on similar cohorts [22,[44][45][46][47]. Two studies may find vastly different results in the predictive accuracy of machine-learning methods even if they use near identical models, covariates, and model summary statistics just due to the choice of the train-test sets (which are determined strictly by random number generation) [32,35,36,48,49].…”

Section: Overall Variability In Model Accuracymentioning

confidence: 99%

Increasing transparency in machine learning through bootstrap simulation and shapely additive explanations

Huang

2023

PLoS ONE

View full text Add to dashboard Cite

Machine learning methods are widely used within the medical field. However, the reliability and efficacy of these models is difficult to assess, making it difficult for researchers to identify which machine-learning model to apply to their dataset. We assessed whether variance calculations of model metrics (e.g., AUROC, Sensitivity, Specificity) through bootstrap simulation and SHapely Additive exPlanations (SHAP) could increase model transparency and improve model selection. Data from the England National Health Services Heart Disease Prediction Cohort was used. After comparison of model metrics for XGBoost, Random Forest, Artificial Neural Network, and Adaptive Boosting, XGBoost was used as the machine-learning model of choice in this study. Boost-strap simulation (N = 10,000) was used to empirically derive the distribution of model metrics and covariate Gain statistics. SHapely Additive exPlanations (SHAP) to provide explanations to machine-learning output and simulation to evaluate the variance of model accuracy metrics. For the XGBoost modeling method, we observed (through 10,000 completed simulations) that the AUROC ranged from 0.771 to 0.947, a difference of 0.176, the balanced accuracy ranged from 0.688 to 0.894, a 0.205 difference, the sensitivity ranged from 0.632 to 0.939, a 0.307 difference, and the specificity ranged from 0.595 to 0.944, a 0.394 difference. Among 10,000 simulations completed, we observed that the gain for Angina ranged from 0.225 to 0.456, a difference of 0.231, for Cholesterol ranged from 0.148 to 0.326, a difference of 0.178, for maximum heart rate (MaxHR) ranged from 0.081 to 0.200, a range of 0.119, and for Age ranged from 0.059 to 0.157, difference of 0.098. Use of simulations to empirically evaluate the variability of model metrics and explanatory algorithms to observe if covariates match the literature are necessary for increased transparency, reliability, and utility of machine learning methods. These variance statistics, combined with model accuracy statistics can help researchers identify the best model for a given dataset.

show abstract

Ultrasound-based radiomics XGBoost model to assess the risk of central cervical lymph node metastasis in patients with papillary thyroid carcinoma: Individual application of SHAP

Cited by 11 publications

References 43 publications

Quantitative ultrasound radiomics analysis to evaluate lymph nodes in patients with cancer: a systematic review

Quantitative ultrasound radiomics analysis to evaluate lymph nodes in patients with cancer: a systematic review

Development and validation of an interpretable radiomic nomogram for severe radiation proctitis prediction in postoperative cervical cancer patients

Increasing transparency in machine learning through bootstrap simulation and shapely additive explanations

Contact Info

Product

Resources

About