Machine learning model successfully identifies important clinical features for predicting outpatients with rotator cuff tears

Abstract

Purpose The aim of this study is to develop a machine learning model to identify important clinical features related to rotator cuf tears (RCTs) using explainable artiicial intelligence (XAI) for eiciently predicting outpatients with RCTs. Methods A retrospective review of a local clinical registry dataset was performed to include patients with shoulder pain and dysfunction who underwent questionnaires and physical examinations between 2019 and 2022. RCTs were diagnosed by shoulder arthroscopy. Six machine-learning algorithms (Stacking, Gradient Boosting Machine, Bagging, Random Forest, Extreme Gradient Boost (XGBoost), and Adaptive Boosting) were developed for the prediction. The performance of the models was assessed by the area under the receiver operating characteristic curve (AUC), Brier scores, and Decision curve. The interpretability of the predicted outcomes was evaluated using Shapley additive explanation (SHAP) values. Results A total of 1684 patients who completed questionnaires and clinical tests were included, and 417 patients with RCTs underwent shoulder arthroscopy. In six machining learning algorithms for predicting RCTs, the accuracy, AUC values, and Brier scores were in the range of 0.81-0.86, 0.75-0.92, and 0.15-0.19, respectively. The XGBoost model showed superior performance with accuracy, AUC, and Brier scores of 0.85(95% conidence interval, 0.82-0.87), 0.92 (95% conidence interval,0.90-0.94), and 0.15 (95% conidence interval,0.14-0.16), respectively. The Shapley plot showed the impact of the clinical features on predicting RCTs. The most important variables were Jobe test, Bear hug test, and age for prediction, with mean SHAP values of 1.458, 0.950, and 0.790, respectively. Conclusion The machine learning model successfully identiied important clinical variables for predicting patients with RCTs. In addition, the best algorithm was also integrated into a digital application to provide predictions in outpatient settings. This tool may assist patients in reducing their pain experience and providing prompt treatments. Level of Evidence Level III.

Artificial Intelligence and Machine Learning in Rotator Cuff Tears

“…The AI algorithm determined the most influential clinical features in predicting RCTs, through mean Shapley additive explanation values, such as the Jobe test, Bear hug test, and age prediction. The trained algorithm also demonstrated an accuracy of 0.85, an AUC of 0.92, a Brier score of 0.15, and is now freely accessible as a digital application 30 …”

“…In many clinical settings, physical examination can provide an initial diagnosis for shoulder pathology related to pain and limited range of motion. However, clinical testing derived from RCT diagnosis may be contentious given the complexity of shoulder movements 30 . Therefore, various imaging modalities are utilized to assist with an accurate diagnosis.…”

“…However, clinical testing derived from RCT diagnosis may be contentious given the complexity of shoulder movements. 30 Therefore, various imaging modalities are utilized to assist with an accurate diagnosis. AI, along with its subdivisions ML and DL, has been widely applied to further characterize RCTs and provide a more precise and timely diagnosis.…”

“…Investigators have advanced methods for ML models to predict RCT diagnosis and outcome in an outpatient setting, regardless of imaging. 30 More recently, an ML algorithm has been used to predict postoperative American Shoulder and Elbow Surgeons (ASES) scores after arthroscopic rotator cuff repair. Several key factors were identified as influential in predicting the patients' postoperative outcome after arthroscopic rotator cuff repair, determined by the Shapley additive explanations approach, such as preoperative ASES score, pre-treatment Visual Analog Pain Scale, body mass index, age, and tendon quality.…”

Artificial intelligence (AI) and large data registries: Understanding the advantages and limitations of contemporary data sets for use in AI research

“…The use of large data registries has gained much attention for developing and validating predictive models using AI [18][19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36]. These analyses have leveraged the statistical power of large data sets to better understand the propensity for adverse events, cost of episodes of care and resource utilizationphenomenon either not readily available or too rare to be studied using institutional data sets.…”