The prediction of distant metastasis risk for male breast cancer patients based on an interpretable machine learning model

Abstract

Objectives This research was designed to compare the ability of different machine learning (ML) models and nomogram to predict distant metastasis in male breast cancer (MBC) patients and to interpret the optimal ML model by SHapley Additive exPlanations (SHAP) framework. Methods Four powerful ML models were developed using data from male breast cancer (MBC) patients in the SEER database between 2010 and 2015 and MBC patients from our hospital between 2010 and 2020. The area under curve (AUC) and Brier score were used to assess the capacity of different models. The Delong test was applied to compare the performance of the models. Univariable and multivariable analysis were conducted using logistic regression. Results Of 2351 patients were analyzed; 168 (7.1%) had distant metastasis (M1); 117 (5.0%) had bone metastasis, and 71 (3.0%) had lung metastasis. The median age at diagnosis is 68.0 years old. Most patients did not receive radiotherapy (1723, 73.3%) or chemotherapy (1447, 61.5%). The XGB model was the best ML model for predicting M1 in MBC patients. It showed the largest AUC value in the tenfold cross validation (AUC:0.884; SD:0.02), training (AUC:0.907; 95% CI: 0.899—0.917), testing (AUC:0.827; 95% CI: 0.802—0.857) and external validation (AUC:0.754; 95% CI: 0.739—0.771) sets. It also showed powerful ability in the prediction of bone metastasis (AUC: 0.880, 95% CI: 0.856—0.903 in the training set; AUC: 0.823, 95% CI:0.790—0.848 in the test set; AUC: 0.747, 95% CI: 0.727—0.764 in the external validation set) and lung metastasis (AUC: 0.906, 95% CI: 0.877—0.928 in training set; AUC: 0.859, 95% CI: 0.816—0.891 in the test set; AUC: 0.756, 95% CI: 0.732—0.777 in the external validation set). The AUC value of the XGB model was larger than that of nomogram in the training (0.907 vs 0.802) and external validation (0.754 vs 0.706) sets. Conclusions The XGB model is a better predictor of distant metastasis among MBC patients than other ML models and nomogram; furthermore, the XGB model is a powerful model for predicting bone and lung metastasis. Combining with SHAP values, it could help doctors intuitively understand the impact of each variable on outcome.

Predicting mortality and recurrence in colorectal cancer: Comparative assessment of predictive models

“…Xuhai Zhao et al (2022) conducted research comparing ML models and a nomogram for predicting distant metastasis in male breast cancer patients. The XGB model outperformed other models and the nomogram, demonstrating superior AUC values and accurate predictions of distant, bone, and lung metastasis [ 44 ].…”

Development and validation of AI models using LR and LightGBM for predicting distant metastasis in breast cancer: a dual-center study

“…For example, Zhao et al. used four machine learning algorithms to predict the risk of breast cancer distant metastasis, with XGBoost performing best (AUC of 0.907 in the training set and 0.754 in the validation set) ( 20 ). Burak Yagin et al.…”

“…Additionally, Zhao et al. ( 20 ) used the SEER database and four machine learning algorithms, including Extreme Gradient Boosting (XGBoost), k-Nearest Neighbors (KNN), Decision Tree (DT), and Support Vector Machine (SVM) to predict the risk of distant metastasis in breast cancer, with XGBoost performing the best. Furthermore, Burak Yagin et al.…”

Prediction models for postoperative recurrence of non-lactating mastitis based on machine learning

“…The concept of SHAP value in game theory is introduced into the interpretation process of the ML model, which can not only reflect the influence of each sample feature, but also show the positivity and negativity of the influence of each feature on the prediction results. Its interpretability is verified in many models [ 9 , 10 ]. The trained model is subjected to tenfold cross-validation to test the performance of the model to reduce problems such as overfitting, and selection bias, and to give the generalization ability of the model on an independent dataset.…”