Machine Learning Approaches to Predict 6-Month Mortality Among Patients With Cancer

Parikh, Ravi B.; Manz, Christopher R; Chivers, Corey; Regli, Susan Harkness; Braun, Johanna; Draugelis, Michael; Schuchter, Lynn M.; Shulman, Lawrence N.; Navathe, Amol S.; Patel, Mitesh S.; O’Connor, Nina

doi:10.1001/jamanetworkopen.2019.15997

Cited by 164 publications

(155 citation statements)

References 35 publications

Supporting

Mentioning

152

Contrasting

Unclassified

Order By: Relevance

“…The mortality prediction algorithm has been previously validated. This trial followed the Consolidated Standards of Reporting Trials ( CONSORT ) reporting guideline. The trial protocol was approved by the University of Pennsylvania institutional review board (trial protocol in Supplement 1 ), which also granted a waiver for the requirement to obtain informed consent because this study was an evaluation of a health system initiative that posed minimal risk to clinicians and patients.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Effect of Integrating Machine Learning Mortality Estimates With Behavioral Nudges to Clinicians on Serious Illness Conversations Among Patients With Cancer

et al. 2020

Self Cite

View full text Add to dashboard Cite

IMPORTANCE Serious illness conversations (SICs) are structured conversations between clinicians and patients about prognosis, treatment goals, and end-of-life preferences. Interventions that increase the rate of SICs between oncology clinicians and patients may improve goal-concordant care and patient outcomes. OBJECTIVE To determine the effect of a clinician-directed intervention integrating machine learning mortality predictions with behavioral nudges on motivating clinician-patient SICs. DESIGN, SETTING, AND PARTICIPANTS This stepped-wedge cluster randomized clinical trial was conducted across 20 weeks (from June 17 to November 1, 2019) at 9 medical oncology clinics (8 subspecialty oncology and 1 general oncology clinics) within a large academic health system in Pennsylvania. Clinicians at the 2 smallest subspecialty clinics were grouped together, resulting in 8 clinic groups randomly assigned to the 4 intervention wedge periods. Included participants in the intention-to-treat analyses were 78 oncology clinicians who received SIC training and their patients (N = 14 607) who had an outpatient oncology encounter during the study period. INTERVENTIONS (1) Weekly emails to oncology clinicians with SIC performance feedback and peer comparisons; (2) a list of up to 6 high-risk patients (Ն10% predicted risk of 180-day mortality) scheduled for the next week, estimated using a validated machine learning algorithm; and (3) opt-out text message prompts to clinicians on the patient's appointment day to consider an SIC. Clinicians in the control group received usual care consisting of weekly emails with cumulative SIC performance. MAIN OUTCOMES AND MEASURES Percentage of patient encounters with an SIC in the intervention group vs the usual care (control) group. RESULTS The sample consisted of 78 clinicians and 14 607 patients. The mean (SD) age of patients was 61.9 (14.2) years, 53.7% were female, and 70.4% were White. For all encounters, SICs were conducted among 1.3% in the control group and 4.6% in the intervention group, a significant difference (adjusted difference in percentage points, 3.3; 95% CI, 2.3-4.5; P < .001). Among 4124 high-risk patient encounters, SICs were conducted among 3.6% in the control group and 15.2% in the intervention group, a significant difference (adjusted difference in percentage points, 11.6; 95% CI, 8.2-12.5; P < .001). CONCLUSIONS AND RELEVANCE In this stepped-wedge cluster randomized clinical trial, an intervention that delivered machine learning mortality predictions with behavioral nudges to oncology clinicians significantly increased the rate of SICs among all patients and among patients with high mortality risk who were targeted by the intervention. Behavioral nudges combined with machine learning mortality predictions can positively influence clinician behavior and may be applied more broadly to improve care near the end of life.

show abstract

Section: Methodsmentioning

confidence: 99%

“…We used a validated ML algorithm that has been shown to accurately predict 180-day mortality risk. We hypothesized that this multimodal intervention would increase the rate of SICs between oncology clinicians and all patients as well as between oncology clinicians and high-risk patients.…”

Section: Introductionmentioning

confidence: 99%

Effect of Integrating Machine Learning Mortality Estimates With Behavioral Nudges to Clinicians on Serious Illness Conversations Among Patients With Cancer

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…In medicine, deep learning has led to advances such as a tool that predicts diabetic retinopathy using retinal fundus photographs and a test that distinguishes COVID-19 from community-acquired pneumonia using chest computed tomography imaging [ [2] , [3] , [4] ]. Machine learning algorithms using patient data from the electronic medical record have been designed to predict acute kidney injury, cancer mortality rate, and prognosis following solid organ transplantation [ [5] , [6] , [7] ]. New applications are sure to emerge as this technology matures [ 8 ].…”

Section: Introductionmentioning

confidence: 99%

Modern Internet Search Analytics and Total Joint Arthroplasty: What Are Patients Asking and Reading Online?

Shen

Driscoll

Islam

et al. 2021

The Journal of Arthroplasty

100

View full text Add to dashboard Cite

Introduction Patients considering total joint arthroplasty often search for information online regarding surgery, however little is known about the specific topics that patients search for and the nature of the information provided. Google compiles frequently asked questions associated with a search term using machine learning and natural language processing. Links to individual websites are provided to answer each question. Analysis of this data may help improve understanding of patient concerns and inform more effective counseling. Methods Search terms were entered into Google for total hip and total knee arthroplasty. Frequently asked questions and associated websites were extracted to a database using customized software. Questions were categorized by topic; websites were categorized by type. JAMA Benchmark Criteria were used to assess website quality. Pearson’s chi-square and Student’s t-tests were performed as appropriate. Results A total of 620 questions (305 TKA, 315 THA) were extracted with 602 associated websites. The most popular question topics were Specific Activities (23.5%), Indications/Management (15.6%), and Restrictions (13.4%). Questions related to Pain were more common in the TKA group (23.0% vs 2.5%, p<0.001) compared to THA. The most common website types were Academic (31.1%), Commercial (29.2%), and Social Media (17.1%). JAMA scores (0-4) were highest for Government websites (mean 3.92, p=0.005). Conclusion The most frequently asked questions on Google related to total joint arthroplasty are related to arthritis management, rehabilitation, and ability to perform specific tasks. A sizeable proportion of health information provided originate from non-academic, non-government sources (64.4%), with 17.1% from social media websites.

show abstract

“…Three classifiers were considered ( eMethods ): logistic regression with lasso regularization [ 15 ], XGBoost [ 16 ], and random forest [ 17 ]. The later two are tree-based algorithms, known for their consistent performance on a variety of datasets [ 18 ] and within similar mortality work [ 10 , 19 ]. Parameter selection was conducted within 5-fold cross validation by sampling patients (not admissions [ 20 ]) before retraining one model on the entire training set ( eMethods ).…”

Section: Methodsmentioning

confidence: 99%

Development, implementation, and prospective validation of a model to predict 60-day end-of-life in hospitalized adults upon admission at three sites

Major

Aphinyanaphongs

2020

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

Background Automated systems that use machine learning to estimate a patient’s risk of death are being developed to influence care. There remains sparse transparent reporting of model generalizability in different subpopulations especially for implemented systems. Methods A prognostic study included adult admissions at a multi-site, academic medical center between 2015 and 2017. A predictive model for all-cause mortality (including initiation of hospice care) within 60 days of admission was developed. Model generalizability is assessed in temporal validation in the context of potential demographic bias. A subsequent prospective cohort study was conducted at the same sites between October 2018 and June 2019. Model performance during prospective validation was quantified with areas under the receiver operating characteristic and precision recall curves stratified by site. Prospective results include timeliness, positive predictive value, and the number of actionable predictions. Results Three years of development data included 128,941 inpatient admissions (94,733 unique patients) across sites where patients are mostly white (61%) and female (60%) and 4.2% led to death within 60 days. A random forest model incorporating 9614 predictors produced areas under the receiver operating characteristic and precision recall curves of 87.2 (95% CI, 86.1–88.2) and 28.0 (95% CI, 25.0–31.0) in temporal validation. Performance marginally diverges within sites as the patient mix shifts from development to validation (patients of one site increases from 10 to 38%). Applied prospectively for nine months, 41,728 predictions were generated in real-time (median [IQR], 1.3 [0.9, 32] minutes). An operating criterion of 75% positive predictive value identified 104 predictions at very high risk (0.25%) where 65% (50 from 77 well-timed predictions) led to death within 60 days. Conclusion Temporal validation demonstrates good model discrimination for 60-day mortality. Slight performance variations are observed across demographic subpopulations. The model was implemented prospectively and successfully produced meaningful estimates of risk within minutes of admission.

show abstract

Machine Learning Approaches to Predict 6-Month Mortality Among Patients With Cancer

Cited by 164 publications

References 35 publications

Effect of Integrating Machine Learning Mortality Estimates With Behavioral Nudges to Clinicians on Serious Illness Conversations Among Patients With Cancer

Effect of Integrating Machine Learning Mortality Estimates With Behavioral Nudges to Clinicians on Serious Illness Conversations Among Patients With Cancer

Modern Internet Search Analytics and Total Joint Arthroplasty: What Are Patients Asking and Reading Online?

Development, implementation, and prospective validation of a model to predict 60-day end-of-life in hospitalized adults upon admission at three sites

Contact Info

Product

Resources

About