Background and purpose — External validation of machine learning (ML) prediction models is an essential step before clinical application. We assessed the proportion, performance, and transparent reporting of externally validated ML prediction models in orthopedic surgery, using the Transparent Reporting for Individual Prognosis or Diagnosis (TRIPOD) guidelines. Material and methods — We performed a systematic search using synonyms for every orthopedic specialty, ML, and external validation. The proportion was determined by using 59 ML prediction models with only internal validation in orthopedic surgical outcome published up until June 18, 2020, previously identified by our group. Model performance was evaluated using discrimination, calibration, and decision-curve analysis. The TRIPOD guidelines assessed transparent reporting. Results — We included 18 studies externally validating 10 different ML prediction models of the 59 available ML models after screening 4,682 studies. All external validations identified in this review retained good discrimination. Other key performance measures were provided in only 3 studies, rendering overall performance evaluation difficult. The overall median TRIPOD completeness was 61% (IQR 43–89), with 6 items being reported in less than 4/18 of the studies. Interpretation — Most current predictive ML models are not externally validated. The 18 available external validation studies were characterized by incomplete reporting of performance measures, limiting a transparent examination of model performance. Further prospective studies are needed to validate or refute the myriad of predictive ML models in orthopedics while adhering to existing guidelines. This ensures clinicians can take full advantage of validated and clinically implementable ML decision tools.
Machine learning (ML) studies are becoming increasingly popular in orthopedics but lack a critically appraisal of their adherence to peer‐reviewed guidelines. The objective of this review was to (1) evaluate quality and transparent reporting of ML prediction models in orthopedic surgery based on the transparent reporting of multivariable prediction models for individual prognosis or diagnosis (TRIPOD), and (2) assess risk of bias with the Prediction model Risk Of Bias ASsessment Tool. A systematic review was performed to identify all ML prediction studies published in orthopedic surgery through June 18th, 2020. After screening 7138 studies, 59 studies met the study criteria and were included. Two reviewers independently extracted data and discrepancies were resolved by discussion with at least two additional reviewers present. Across all studies, the overall median completeness for the TRIPOD checklist was 53% (interquartile range 47%–60%). The overall risk of bias was low in 44% (n = 26), high in 41% (n = 24), and unclear in 15% (n = 9). High overall risk of bias was driven by incomplete reporting of performance measures, inadequate handling of missing data, and use of small datasets with inadequate outcome numbers. Although the number of ML studies in orthopedic surgery is increasing rapidly, over 40% of the existing models are at high risk of bias. Furthermore, over half incompletely reported their methods and/or performance measures. Until these issues are adequately addressed to give patients and providers trust in ML models, a considerable gap remains between the development of ML prediction models and their implementation in orthopedic practice.
Background: Predicting oncologic outcomes is essential for optimizing the treatment for patients with cancer. This review examines the feasibility of using Computed Tomography (CT) images of fat density as a prognostic factor in patients with cancer. Methods: A systematic literature search was performed in PubMed, Embase and Cochrane up to March 2020. All studies that mentioned using subcutaneous or visceral adipose tissue (SAT and VAT, respectively) CT characteristics as a prognostic factor for patients with cancer were included. The primary endpoints were any disease-related outcomes in patients with cancer. Results: After screening 1043 studies, ten studies reporting a total of 23ten for SAT and thirteen for VATcomparisons on survival, tumor recurrence and postsurgical infection were included. All ten studies included different types of malignancy: six localized, two metastatic disease, and two both. Five different anatomic landmarks were used to uniformly measure fat density on CT: lumbar (L)4 (n ¼ 4), L3 (n ¼ 2), L4-L5 intervertebral space (n ¼ 2), L5-S1 intervertebral space (n ¼ 1), and the abdomen (n ¼ 1). Overall, six of ten SAT comparisons (60%) and six of thirteen VAT comparisons (46%) reported a significant (p < .05) association of increased SAT or VAT density with an adverse outcome. All remaining nonsignificant comparisons, except one, deviated in the same direction of being predictive for adverse outcomes but failed to reach significance. The median hazard ratio (HR) for the nine SAT and thirteen VAT associations where HRs were given were 1.45 (95% confidence interval [CI] 1.01-1.97) and 1.90 (95% CI 1.12-2.74), respectively. The binomial sign test and Fisher's method both reported a significant association between both SAT and VAT and adverse outcomes. Conclusion: This review may support the feasibility of using SAT or VAT on CT as a prognostic tool for patients with cancer in predicting adverse outcomes such as survival and tumor recurrence. Future research should standardize radiologic protocol in prospective homogeneous series of patients on each cancer diagnosis group in order to establish accurate parameters to help physicians use CT scan defined characteristics in clinical practice.
Background: The outcome differences following surgery for an impending versus a completed pathological fracture have not been clearly defined. The purpose of the present study was to assess differences in outcomes following the surgical treatment of impending versus completed pathological fractures in patients with long-bone metastases in terms of (1) 90day and 1-year survival and (2) intraoperative blood loss, perioperative blood transfusion, anesthesia time, duration of hospitalization, 30-day postoperative systemic complications, and reoperations. Methods:We retrospectively performed a matched cohort study utilizing a database of 1,064 patients who had undergone operative treatment for 462 impending and 602 completed metastatic long-bone fractures. After matching on 22 variables, including primary tumor, visceral metastases, and surgical treatment, 270 impending pathological fractures were matched to 270 completed pathological fractures. The primary outcome was assessed with the Cox proportional hazard model. The secondary outcomes were assessed with the McNemar test and the Wilcoxon signed-rank test. Results:The 90-day survival rate did not differ between the groups (HR, 1.13 [95% CI, 0.81 to 1.56]; p = 0.48), but the 1year survival rate was worse for completed pathological fractures (46% versus 38%) (HR, 1.28 [95% CI, 1.02 to 1.61]; p = 0.03). With regard to secondary outcomes, completed pathological fractures were associated with higher intraoperative estimated blood loss (p = 0.03), a higher rate of perioperative blood transfusions (p = 0.01), longer anesthesia time (p = 0.04), and more reoperations (OR, 2.50 [95% CI, 1.92 to 7.86]; p = 0.03); no differences were found in terms of the rate of 30-day postoperative complications or the duration of hospitalization.Conclusions: Patients undergoing surgery for impending pathological fractures had lower 1-year mortality rates and Disclosure: The Disclosure of Potential Conflicts of Interest forms are provided with the online version of the article (http://links.lww.com/JBJS/G816).
Background and Objectives Body composition measurements using computed tomography (CT) may serve as imaging biomarkers of survival in patients with and without cancer. This study assesses whether body composition measurements obtained on abdominal CTs are independently associated with 90‐day and 1‐year mortality in patients with long‐bone metastases undergoing surgery. Methods This single institutional retrospective study included 212 patients who had undergone surgery for long‐bone metastases and had a CT of the abdomen within 90 days before surgery. Quantification of cross‐sectional areas (CSA) and CT attenuation of abdominal subcutaneous adipose tissue, visceral adipose tissue, and paraspinous and abdominal muscles were performed at L4. Multivariate Cox proportional‐hazards analyses were performed. Results Sarcopenia was independently associated with 90‐day mortality (hazard ratio [HR] = 1.87; 95% confidence interval [CI] = 1.11–3.16; p = 0.019) and 1‐year mortality (HR = 1.50; 95% CI = 1.02–2.19; p = 0.038) in multivariate analysis while controlling for clinical variables such as primary tumors, comorbidities, and chemotherapy. Abdominal fat CSAs and muscle attenuation were not associated with mortality. Conclusions The presence of sarcopenia assessed by CT is predictive of 90‐day and 1‐year mortality in patients undergoing surgery for long‐bone metastases. This body composition measurement can be used as novel imaging biomarker supplementing existing prognostic tools to optimize patient selection for surgery and improve shared decision making.
Background: The widespread use of electronic patient-generated health data has led to unprecedented opportunities for automated extraction of clinical features from free-text medical notes. However, processing this rich resource of data for clinical and research purposes, depends on laborintensive and potentially error-prone manual review. The aim of this study was to develop a natural language processing (NLP) algorithm for binary classification (single metastasis versus two or more metastases) in bone scintigraphy reports of patients undergoing surgery for bone metastases. Material and methods: Bone scintigraphy reports of patients undergoing surgery for bone metastases were labeled each by three independent reviewers using a binary classification (single metastasis versus two or more metastases) to establish a ground truth. A stratified 80:20 split was used to develop and test an extreme-gradient boosting supervised machine learning NLP algorithm. Results: A total of 704 free-text bone scintigraphy reports from 704 patients were included in this study and 617 (88%) had multiple bone metastases. In the independent test set (n ¼ 141) not used for model development, the NLP algorithm achieved an 0.97 AUC-ROC (95% confidence interval [CI], 0.92-0.99) for classification of multiple bone metastases and an 0.99 AUC-PRC (95% CI, 0.99-0.99). At a threshold of 0.90, NLP algorithm correctly identified multiple bone metastases in 117 of the 124 who had multiple bone metastases in the testing cohort (sensitivity 0.94) and yielded 3 false positives (specificity 0.82). At the same threshold, the NLP algorithm had a positive predictive value of 0.97 and F1score of 0.96. Conclusions: NLP has the potential to automate clinical data extraction from free text radiology notes in orthopedics, thereby optimizing the speed, accuracy, and consistency of clinical chart review. Pending external validation, the NLP algorithm developed in this study may be implemented as a means to aid researchers in tackling large amounts of data.
Objective: A systematic review and meta-analysis was conducted to assess the quality of life (QoL) after open surgery for spinal metastases, and how surgery affects physical, social/family, emotional, and functional well-being. Summary of Background Data: It remains questionable to whatextent open surgery improves QoL for metastatic spinal disease, it would be interesting to quantify the magnitude and duration of QoL benefits-if any-after surgery for spinal metastases.Materials and Methods: Included were studies measuring QoL before and after nonpercutaneous, open surgery for spinal metastases for various indications including pain, spinal cord compression, instability, or tumor control. A random-effect model assessed standardized mean differences (SMDs) of summary QoL scores between baseline and 1, 3, 6, or 9-12 months after surgery. Results:The review yielded 10 studies for data extraction. The pooled QoL summary score improved from baseline to 1 month (SMD = 1.09, P < 0.001), to 3 months (SMD = 1.28, P < 0.001), to 6 months (SMD = 1.21, P < 0.001), and to 9-12 months (SMD = 1.08, P = 0.001). The surgery improved physical well-being during the first 3 months (SMD = 0.94, P = 0.022), improved emotional (SMD = 1.19, P = 0.004), and functional well-being (SMD = 1.08, P = 0.005) during the first 6 months, and only improved social/ family well-being at month 6 (SMD = 0.28, P = 0.001). Conclusions:The surgery improved QoL for patients with spinal metastases, and rapidly improved physical, emotional, and functional well-being; it had minimal effect on social/family wellbeing. However, choosing the optimal candidate for surgical intervention in the setting of spinal metastases remains paramount: otherwise postoperative morbidity and complications may outbalance the intended benefits of surgery. Future research should report clear definitions of selection criteria and surgical indication and provide stratified QoL results by indication and clinical characteristics such as primary tumor type, preoperative Karnofsky, and Bilsky scores to elucidate the optimal candidate for surgical intervention.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.