Prospective Comparison of Medical Oncologists and a Machine Learning Model to Predict 3-Month Mortality in Patients With Metastatic Solid Tumors

Zachariah, Finly; Rossi, Lorenzo; Roberts, Laura Morgan; Bosserman, Linda D.

doi:10.1001/jamanetworkopen.2022.14514

Cited by 22 publications

(20 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Study 3. In a study published by Zachariah and colleagues, 24 the positive likelihood ratios (PLRs) support the conclusion drawn by the authors that the machine learning (ML) model outperforms the oncologists at utilizing the surprise question to predict the patients with a higher 3-month mortality (Table 3). Of great significance however, calculation of the NLRs suggest that negative or "yes" responses to both SQ and ML model are not an effective method to screen for 3-month mortality estimation.…”

Section: Receiver Operating Characteristics Curvesmentioning

confidence: 63%

Likelihood Ratios: An Important Concept for Palliative Physicians to Understand

Davis

Soni

Strobel

2022

Am J Hosp Palliat Care

View full text Add to dashboard Cite

Palliative care has several tools and questionnaires which are commonly used for patient-related outcomes and prognosis. As an example, the Surprise Question (I would or would not be surprised that this person would have died in a year) has been used as a screen for palliative care referral but also used as a prognostic tool. Diagnostic tests, prognostic tools, and tools for gauging outcomes have certain sensitivity and specificity in predicting a diagnosis or outcome. Clinicians often use positive and negative predictive values in judging the merits of a diagnostic tool or questionnaire. However positive and negative predictive values are highly dependent on the prevalence of disease or outcome in a population and thus are not portable across studies. Likelihood ratios are both portable across populations but also provide the strength of the diagnostic or predictive measure of a test or questionnaire. In this article, we review the value and limitations of likelihood ratios and illustrate the value of using likelihood ratios using 3 studies centered on the Surprise Question published in 2022.

show abstract

Section: Receiver Operating Characteristics Curvesmentioning

confidence: 63%

Likelihood Ratios: An Important Concept for Palliative Physicians to Understand

Davis

Soni

Strobel

2022

Am J Hosp Palliat Care

View full text Add to dashboard Cite

show abstract

“…The integration of mortality risk into clinical practice remains a subject of ongoing investigation. Several academic hospitals have recently published promising workflows and initiatives to enhance serious illness conversations, 35 , 36 , 37 , 38 including a recent study of 20 506 oncology patients using a machine-learning algorithm to generate prompts, which successfully increased serious illness conversations from 3.4% to 13.5% and decreased end-of-life systemic therapy. 39 In these studies, multiple barriers have been identified, including lack of clinician engagement, alert fatigue, restricted or biased prediction models, and limited practice capacity for serious illness conversations.…”

Section: Discussionmentioning

confidence: 99%

Advanced Care Planning for Hospitalized Patients Following Clinician Notification of Patient Mortality by a Machine Learning Algorithm

et al. 2023

View full text Add to dashboard Cite

ImportanceGoal-concordant care is an ongoing challenge in hospital settings. Identification of high mortality risk within 30 days may call attention to the need to have serious illness conversations, including the documentation of patient goals of care.ObjectiveTo examine goals of care discussions (GOCDs) in a community hospital setting with patients identified as having a high risk of mortality by a machine learning mortality prediction algorithm.Design, Setting, and ParticipantsThis cohort study took place at community hospitals within 1 health care system. Participants included adult patients with a high risk of 30-day mortality who were admitted to 1 of 4 hospitals between January 2 and July 15, 2021. Patient encounters of inpatients in the intervention hospital where physicians were notified of the computed high risk mortality score were compared with patient encounters of inpatients in 3 community hospitals without the intervention (ie, matched control).InterventionPhysicians of patients with a high risk of mortality within 30 days received notification and were encouraged to arrange for GOCDs.Main Outcomes and MeasuresThe primary outcome was the percentage change of documented GOCDs prior to discharge. Propensity-score matching was completed on a preintervention and postintervention period using age, sex, race, COVID-19 status, and machine learning-predicted mortality risk scores. A difference-in-difference analysis validated the results.ResultsOverall, 537 patients were included in this study with 201 in the preintervention period (94 in the intervention group; 104 in the control group) and 336 patients in the postintervention period. The intervention and control groups included 168 patients per group and were well-balanced in age (mean [SD], 79.3 [9.60] vs 79.6 [9.21] years; standardized mean difference [SMD], 0.03), sex (female, 85 [51%] vs 85 [51%]; SMD, 0), race (White patients, 145 [86%] vs 144 [86%]; SMD 0.006), and Charlson comorbidities (median [range], 8.00 [2.00-15.0] vs 9.00 [2.00 to 19.0]; SMD, 0.34). Patients in the intervention group from preintervention to postintervention period were associated with being 5 times more likely to have documented GOCDs (OR, 5.11 [95% CI, 1.93 to 13.42]; P = .001) by discharge compared with matched controls, and GOCD occurred significantly earlier in the hospitalization in the intervention patients as compared with matched controls (median, 4 [95% CI, 3 to 6] days vs 16 [95% CI, 15 to not applicable] days; P &lt; .001). Similar findings were observed for Black patient and White patient subgroups.Conclusions and RelevanceIn this cohort study, patients whose physicians had knowledge of high-risk predictions from machine learning mortality algorithms were associated with being 5 times more likely to have documented GOCDs than matched controls. Additional external validation is needed to determine if similar interventions would be helpful at other institutions.

show abstract

“…Zachariah, Rossi, Roberts, and Bosserman (2022) used a ML model based on artificial neural networks (ANN) to compare the treatment suggestions made by medical oncologists and the AI algorithm. The ANN was trained on a data set of breast cancer patients, and the algorithm was able to suggest treatments that were similar to those suggested by the oncologists.…”

Section: Adoption Of Machine Learning In the Health Sectormentioning

confidence: 99%

Adoption of machine learning systems within the health sector: a systematic review, synthesis and research agenda

Bundi

2023

DTS

View full text Add to dashboard Cite

PurposeThe purpose of this study is to examine the state of research into adoption of machine learning systems within the health sector, to identify themes that have been studied and observe the important gaps in the literature that can inform a research agenda going forward.Design/methodology/approachA systematic literature strategy was utilized to identify and analyze scientific papers between 2012 and 2022. A total of 28 articles were identified and reviewed.FindingsThe outcomes reveal that while advances in machine learning have the potential to improve service access and delivery, there have been sporadic growth of literature in this area which is perhaps surprising given the immense potential of machine learning within the health sector. The findings further reveal that themes such as recordkeeping, drugs development and streamlining of treatment have primarily been focused on by the majority of authors in this area.Research limitations/implicationsThe search was limited to journal articles published in English, resulting in the exclusion of studies disseminated through alternative channels, such as conferences, and those published in languages other than English. Considering that scholars in developing nations may encounter less difficulty in disseminating their work through alternative channels and that numerous emerging nations employ languages other than English, it is plausible that certain research has been overlooked in the present investigation.Originality/valueThis review provides insights into future research avenues for theory, content and context on adoption of machine learning within the health sector.

show abstract

Prospective Comparison of Medical Oncologists and a Machine Learning Model to Predict 3-Month Mortality in Patients With Metastatic Solid Tumors

Cited by 22 publications

References 60 publications

Likelihood Ratios: An Important Concept for Palliative Physicians to Understand

Likelihood Ratios: An Important Concept for Palliative Physicians to Understand

Advanced Care Planning for Hospitalized Patients Following Clinician Notification of Patient Mortality by a Machine Learning Algorithm

Adoption of machine learning systems within the health sector: a systematic review, synthesis and research agenda

Contact Info

Product

Resources

About