An Electronic Medical Record–Based Discharge Disposition Tool Gets Bundle Busted: Decaying Relevance of Clinical Data Accuracy in Machine Learning

Greenstein, Alexander; Teitel, Jack; Mitten, David J.; Ricciardi, Benjamin F.; Myers, Thomas G.

doi:10.1016/j.artd.2020.08.007

Cited by 15 publications

(9 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thirty-one studies assessed the prediction of clinical outcome and/or resource utilization for TJA 4,5,10,11,16,18-20,29-31,38-57 . The median sample size was 5,282 (range: 51 to 295,605).…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Efficacy and Applications of Artificial Intelligence and Machine Learning Analyses in Total Joint Arthroplasty

Polce

Kunze

Dooley

et al. 2022

Journal of Bone and Joint Surgery

View full text Add to dashboard Cite

Background: There has been a considerable increase in total joint arthroplasty (TJA) research using machine learning (ML). Therefore, the purposes of this study were to synthesize the applications and efficacies of ML reported in the TJA literature, and to assess the methodological quality of these studies.Methods: PubMed, OVID/MEDLINE, and Cochrane libraries were queried in January 2021 for articles regarding the use of ML in TJA. Study demographics, topic, primary and secondary outcomes, ML model development and testing, and model presentation and validation were recorded. The TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) guidelines were used to assess the methodological quality.Results: Fifty-five studies were identified: 31 investigated clinical outcomes and resource utilization; 11, activity and motion surveillance; 10, imaging detection; and 3, natural language processing. For studies reporting the area under the receiver operating characteristic curve (AUC), the median AUC (and range) was 0.80 (0.60 to 0.97) among 26 clinical outcome studies, 0.99 (0.83 to 1.00) among 6 imaging-based studies, and 0.88 (0.76 to 0.98) among 3 activity and motion surveillance studies. Twelve studies compared ML to logistic regression, with 9 (75%) reporting that ML was superior. The average number of TRIPOD guidelines met was 11.5 (range: 5 to 18), with 38 (69%) meeting greater than half of the criteria. Presentation and explanation of the full model for individual predictions and assessments of model calibration were poorly reported (<30%). Conclusions:The performance of ML models was good to excellent when applied to a wide variety of clinically relevant outcomes in TJA. However, reporting of certain key methodological and model presentation criteria was inadequate. Despite the recent surge in TJA literature utilizing ML, the lack of consistent adherence to reporting guidelines needs to be addressed to bridge the gap between model development and clinical implementation.T he application of artificial intelligence (AI) within orthopaedic surgery research has undergone a substantial increase in recent years [1][2][3][4][5][6][7][8][9][10][11][12] . Machine learning (ML) represents a subset within AI that includes algorithms that autonomously learn from data and make predictions without explicit programmed instructions. The adoption of ML in orthopaedic research has been a function of increasing data availability and several purported benefits including (1) optimization of model performance through "learning," (2) avoidance of limitations inherent to traditional regression analyses, and (3) ability to integrate individualized patient elements to provide individu-alized explanations of anticipated outcomes 6,[13][14][15] . Despite the proliferation of ML literature and the theoretical benefits of ML, it is important to assess whether these analyses have resulted in meaningful results that may influence clinical practice.Current investigations using ML within the total j...

show abstract

Section: Resultsmentioning

confidence: 99%

“…Thirty-one studies assessed the prediction of clinical outcome and/or resource utilization for TJA 4,5,10,11,16,[18][19][20][29][30][31][38][39][40][41][42][43][44][45][46][47][48][49][50][51][52][53][54][55][56][57] . The median sample size was 5,282 (range: 51 to 295,605).…”

Section: Clinical Outcomes and Resource Utilizationmentioning

confidence: 99%

Efficacy and Applications of Artificial Intelligence and Machine Learning Analyses in Total Joint Arthroplasty

Polce

Kunze

Dooley

et al. 2022

Journal of Bone and Joint Surgery

View full text Add to dashboard Cite

show abstract

“…31-34 Other studies, seeking to improve on currently available tools, have sought to develop ML models to predict discharge disposition. 9,35 Greenstein et al 35 tested an electronic medical record–integrated artificial NN and compared it with RAPT; while providing a proof of concept of ML utility in predicting disposition outcomes, they concluded that their artificial NN, with an AUC of 0.804 and an accuracy of 61.3%, did worse than currently available tools, such as RAPT. In comparison, this present study developed ML models with improved reliability and responsiveness and found that LSVM had the highest AUC at 0.801 with a reliability of 82.37%.…”

Section: Discussionmentioning

confidence: 99%

Comparative Analysis of the Ability of Machine Learning Models in Predicting In-hospital Postoperative Outcomes After Total Hip Arthroplasty

El-Othmani

Zalikha

Shah

2022

J Am Acad Orthop Surg

View full text Add to dashboard Cite

Background: Machine learning (ML) methods have shown promise in a wide range of applications including the development of patientspecific predictive models before surgical interventions. The purpose of this study was to develop, test, and compare four distinct ML models to predict postoperative parameters after primary total hip arthroplasty. Methods: Data from the Nationwide Inpatient Sample were used to identify patients undergoing total hip arthroplasty from 2016 to 2017. Linear support vector machine (LSVM), random forest (RF), neural network (NN), and extreme gradient boost trees (XGBoost) predictive of mortality, length of stay, and discharge disposition were developed and validated using 15 predictive patient-specific and hospital-specific factors. Area under the curve of the receiver operating characteristic (AUCROC) curve and accuracy were used as validity metrics, and the strongest predictive variables under each model were assessed. Results: A total of 177,442 patients were included in this analysis. For mortality, the XGBoost, NN, and LSVM models all had excellent responsiveness during validation while RF had fair responsiveness. LSVM had the highest responsiveness with an AUCROC of 0.973 during validation. For the length of stay, the LSVM and NN models had fair responsiveness while the XGBoost and random forest models had poor responsiveness. LSVM had the highest responsiveness with an AUCROC of 0.744 during validation. For the discharge disposition outcome, LSVM had good responsiveness while the XGBoost, NN, and RF models all had fair responsiveness. LSVM had the highest responsiveness with an AUCROC of 0.801. Discussion: The ML methods tested demonstrated a range of poorto-excellent responsiveness and accuracy in the prediction of the assessed metrics, with LSVM being the best performer. Such models

show abstract

“…It can be defined as a fully connected, feed-forward ANN with a single hidden layer. According to Greenstein et al [ 9 ], this method works by taking the input variables, multiplying them by weights (amount of impact), and using a nonlinear function to scale this information within a range. Moreover, this process occurs at each node within the neural network to optimize it ( Fig.…”

Section: Methodsmentioning

confidence: 99%

Predicting Prolonged Hospital Stays in Elderly Patients With Hip Fractures Managed During the COVID-19 Pandemic in Chile: An Artificial Neural Networks Study

Díaz-Ledezma

Mardones

2022

HSS Journal®: The Musculoskeletal Journal of Hospital for Speci

View full text Add to dashboard Cite

Background: Prolonged length of stay (LOS) after a hip fracture is associated with increased mortality. Purpose: We sought to create a model to predict prolonged LOS in elderly Chilean patients with hip fractures managed during the COVID-19 pandemic. Methods: Employing an official database, we created an artificial neural network (ANN), a computational model corresponding to a subset of machine learning, to predict prolonged LOS (≥14 days) among 2686 hip fracture patients managed in 43 Chilean public hospitals during 2020. We identified 18 clinically relevant variables as potential predictors; 80% of the sample was used to train the ANN and 20% was used to test it. The performance of the ANN was evaluated via measuring its discrimination power through the area under the curve of the receiver operating characteristic curve (AUC-ROC). Results: Of the 2686 patients, 820 (30.2%) had prolonged LOS. In the training sample (2,125 cases), the ANN correctly classified 1,532 cases (72.09%; AUC-ROC: 0.745). In the test sample (561 cases), the ANN correctly classified 401 cases (71.48%; AUC-ROC: 0.742). The most relevant variables to predict prolonged LOS were the patient’s admitting hospital (relative importance [RI]: 0.11), the patient’s geographical health service providing health care (RI: 0.11), and the patient’s surgery being conducted within 2 days of admission (RI: 0.10). Conclusions: Using national-level big data, we developed an ANN that predicted with fair accuracy prolonged LOS in elderly Chilean patients with hip fractures during the COVID-19 pandemic. The main predictors of a prolonged LOS were unrelated to the patient’s individual health and concerned administrative and organizational factors.

show abstract

An Electronic Medical Record–Based Discharge Disposition Tool Gets Bundle Busted: Decaying Relevance of Clinical Data Accuracy in Machine Learning

Cited by 15 publications

References 22 publications

Efficacy and Applications of Artificial Intelligence and Machine Learning Analyses in Total Joint Arthroplasty

Efficacy and Applications of Artificial Intelligence and Machine Learning Analyses in Total Joint Arthroplasty

Comparative Analysis of the Ability of Machine Learning Models in Predicting In-hospital Postoperative Outcomes After Total Hip Arthroplasty

Predicting Prolonged Hospital Stays in Elderly Patients With Hip Fractures Managed During the COVID-19 Pandemic in Chile: An Artificial Neural Networks Study

Contact Info

Product

Resources

About