Deep Learning With Electronic Health Records for Short-Term Fracture Risk Identification: Crystal Bone Algorithm Development and Validation

Abstract

Background Fractures as a result of osteoporosis and low bone mass are common and give rise to significant clinical, personal, and economic burden. Even after a fracture occurs, high fracture risk remains widely underdiagnosed and undertreated. Common fracture risk assessment tools utilize a subset of clinical risk factors for prediction, and often require manual data entry. Furthermore, these tools predict risk over the long term and do not explicitly provide short-term risk estimates necessary to identify patients likely to experience a fracture in the next 1-2 years. Objective The goal of this study was to develop and evaluate an algorithm for the identification of patients at risk of fracture in a subsequent 1- to 2-year period. In order to address the aforementioned limitations of current prediction tools, this approach focused on a short-term timeframe, automated data entry, and the use of longitudinal data to inform the predictions. Methods Using retrospective electronic health record data from over 1,000,000 patients, we developed Crystal Bone, an algorithm that applies machine learning techniques from natural language processing to the temporal nature of patient histories to generate short-term fracture risk predictions. Similar to how language models predict the next word in a given sentence or the topic of a document, Crystal Bone predicts whether a patient’s future trajectory might contain a fracture event, or whether the signature of the patient’s journey is similar to that of a typical future fracture patient. A holdout set with 192,590 patients was used to validate accuracy. Experimental baseline models and human-level performance were used for comparison. Results The model accurately predicted 1- to 2-year fracture risk for patients aged over 50 years (area under the receiver operating characteristics curve [AUROC] 0.81). These algorithms outperformed the experimental baselines (AUROC 0.67) and showed meaningful improvements when compared to retrospective approximation of human-level performance by correctly identifying 9649 of 13,765 (70%) at-risk patients who did not receive any preventative bone-health-related medical interventions from their physicians. Conclusions These findings indicate that it is possible to use a patient’s unique medical history as it changes over time to predict the risk of short-term fracture. Validating and applying such a tool within the health care system could enable automated and widespread prediction of this risk and may help with identification of patients at very high risk of fracture.

Machine Learning Solutions for Osteoporosis—A Review

“…( 103 ) By taking advantage of ML models applied to sequential data, Almog and colleagues investigated the use of natural language processing techniques, such as recurrent neural networks, to predict short‐term incident fractures in electronic health records (EHRs). ( 106 ) They observed a better performance of their model in the prediction of subsequent fractures compared with first fractures. Three studies clearly documented methods for correcting class imbalance caused by the very low incidence of positive cases.…”

“…( 99 ) The 12 remaining studies used supervised learning for the prediction of risk of osteoporosis by bone density loss at 10 years, ( 100 ) incident falls at 6 months ( 102 ) and 1 year, ( 101 ) incident vertebral fracture at ≈ 8 months, ( 103 ) hip fracture prediction at 4, 5, or 10 years, ( 107–111 ) vertebral or hip fractures at ≈ 7.5 years, ( 104 ) major osteoporotic fractures (hip, spine, wrist, or humerus) at ≈ 4.5 years, ( 105 ) and all sort of fracture sites at 1 and 2 years. ( 106 ) Because unsupervised learning is not intended to predict a predetermined outcome, no performance metrics were reported. However, interesting features were identified, such as antiresorptive treatment response compliance, which could improve osteoporosis treatment.…”

“…Three studies clearly documented methods for correcting class imbalance caused by the very low incidence of positive cases. ( 105,106,109 ) Ye and colleagues and Kruse and colleagues calibrated their prediction models by aligning the predicted with the observed probabilities, to aid its understandability and applicability in clinical practice. ( 101,108 ) Good visualization of predictors was obtained using SHAP values.…”

“…Predicting the risk of bone loss, osteoporotic fractures, falls, or comorbidities in osteoporotic patients over time was investigated in 14 studies (Table 4). ( 98–111 ) Two of them used unsupervised learning to identify fracture and comorbidity risk groups, respectively. ( 98,99 ) Kruse and colleagues developed a fracture risk clustering model to categorize subgroups of patients at risk.…”

Machine Learning–Driven Models to Predict Prognostic Outcomes in Patients Hospitalized With Heart Failure Using Electronic Health Records: Retrospective Study

“…Besides this, predictive models have been constructed in various health care domains with a certain degree of success by automated mining of EHRs, combined with machine learning (ML) approaches [17], specifically for the prediction of HF outcomes [18][19][20]. In contrast to previous studies of predictive models for HF outcomes that apply traditional methods, recent research is adopting ML techniques for predicting HF mortality, readmission, and medication adherence, which might demonstrate better performance in their predictions because of their consideration of higher order and nonlinear relationships between multidimensional variables [21].…”