Prediction of lymph node metastasis in patients with breast invasive micropapillary carcinoma based on machine learning and SHapley Additive exPlanations framework

Abstract

Background and purpose: Machine learning (ML) is applied for outcome prediction and treatment support. This study aims to develop different ML models to predict risk of axillary lymph node metastasis (LNM) in breast invasive micropapillary carcinoma (IMPC) and to explore the risk factors of LNM.Methods: From the Surveillance, Epidemiology, and End Results (SEER) database and the records of our hospital, a total of 1547 patients diagnosed with breast IMPC were incorporated in this study. The ML model is built and the external validation is carried out. SHapley Additive exPlanations (SHAP) framework was applied to explain the optimal model; multivariable analysis was performed with logistic regression (LR); and nomograms were constructed according to the results of LR analysis.Results: Age and tumor size were correlated with LNM in both cohorts. The luminal subtype is the most common in patients, with the tumor size <=20mm. Compared to other models, Xgboost was the best ML model with the biggest AUC of 0.813 (95% CI: 0.7994 -0.8262) and the smallest Brier score of 0.186 (95% CI: 0.799-0.826). SHAP plots demonstrated that tumor size was the most vital risk factor for LNM. In both training and test sets, Xgboost had better AUC (0.761 vs 0.745; 0.813 vs 0.775; respectively), and it also achieved a smaller Brier score (0.202 vs 0.204; 0.186 vs 0.191; 0.220 vs 0.221; respectively) than the nomogram model based on LR in those three different sets. After adjusting for five most influential variables (tumor size, age, ER, HER-2, and PR), prediction score based on the Xgboost model was still correlated with LNM (adjusted OR:2.73, 95% CI: 1.30-5.71, P=0.008). Conclusions: The Xgboost model outperforms the traditional LR-based nomogram model in predicting the LNM of IMPC patients. Combined with SHAP, it can more intuitively reflect the influence of different variables on the LNM. The tumor size was the most important risk factor of LNM for breast IMPC Frontiers in Oncology frontiersin.org 01

The Role of AI in Breast Cancer Lymph Node Classification: A Comprehensive Review

“…Therefore, model performances can be overly optimistic, and can underperform when evaluated on unseen real-world data. That can be seen in the results reported by Jiang et al, where the XGBoost AUC on the test set was 0.81, which dropped to 0.70 when evaluated on an external validation cohort [ 47 ]. Moreover, one systematic review has shown that in DL radiology applications, validation scores will demonstrate a modest to substantial decrease when compared to internal performance [ 54 ].…”

“…One other study that evaluated ML methods only on breast cancer patients with invasive micropapillary carcinoma also had XGBoost as the best performing algorithm [ 47 ]. As determined by Shapley values, tumor size, and patient age were the two most important features.…”

“…Models were trained on data extracted from 1405 patients and evaluated on an external validation cohort of 142 patients. Interestingly, while XGBoost achieved an AUC of 0.76 [0.746–0.776] on the training set and an AUC of 0.81 [0.799–0.826] on the test set, when evaluated on an external validation cohort the AUC dropped to 0.700 [0.683–0.716], hence indicating that the model was overfitted on the previously seen data [ 47 ].…”

The prediction of distant metastasis risk for male breast cancer patients based on an interpretable machine learning model

“…If we can develop a tool or model to predict the probability of mortality, it would be helpful to urge MBC patients to receive timely profession examination or treatment. In recent years, ML models have also been widely applied to predict survival or lymph node metastasis of breast cancer [23,35,36]. However, it has not been used to predict the distant metastasis risk of MBC patients.…”

“…To intuitively understand the nature of the ML model with the feature of 'black-box' , the SHAP framework was introduced into this study to interpret the optimal ML model. Its interpretability performance has been validated in many models [21][22][23]. The SHAP framework could present global (e.g., summary plot) and local (e.g., force plot) interpretability plots based on SHAP values.…”