A framework for building a clinically relevant risk model.

Abstract

6554 Background: Acute care accounts for half of cancer expenditures and is a measure of poor quality care. Identifying patients at high risk for emergency department (ED) visits enables institutions to target resources to those most likely to benefit. Risk stratification models developed to date have not been meaningfully employed in oncology, and there is a need for clinically relevant models to improve patient care. Methods: We established and applied a predictive framework for clinical use with attention to modeling technique, clinician feedback, and application metrics. The model employs electronic health record data from initial visit to first antineoplastic administration for patients at our institution from January 2014 to June 2017. The binary dependent variable is occurrence of an ED visit within the first 6 months of treatment. The final regularized multivariable logistic regression model was chosen based on clinical and statistical significance. In order to accommodate for the needs to the program, parameter selection and model calibration were optimized to suit the positive predictive value of the top 25% of observations as ranked by model-determined risk. Results: There are 5,752 antineoplastic administration starts in our training set, and 1,457 in our test set. The positive predictive value of this model for the top 25% riskiest new start antineoplastic patients is 0.53. From over 1,400 data features, the model was refined to include 400 clinically relevant ones spanning demographics, pathology, clinician notes, labs, medications, and psychosocial information. At the patient level, specific features determining risk are surfaced in a web application, RiskExplorer, to enable clinician review of individual patient risk. This physician facing application provides the individual risk score for the patient as well as their quartile of risk when compared to the population of new start antineoplastic patients. For the top quartile of patients, the risk for an ED visit within the first 6 months of treatment is greater than or equal to 49%. Conclusions: We have constructed a framework to build a clinically relevant risk model. We are now piloting it to identify those likely to benefit from a home-based, digital symptom management intervention.

Natural Language Processing Methods to Identify Oncology Patients at High Risk for Acute Care with Clinical Notes

“…In total, they trained their model using 760 inputs and retained 125 to predict the risk of ACU. This work, and others, highlight the potential of data-driven models to predict ACU risk using SHD [6][7][8] . However, most EHRs are not mapped to a common data model and they are not necessarily standardised between different facilities.…”