Eye of the beholder: Risk calculators and barriers to adoption in surgical trainees

Leeds, Ira L.; Rosenblum, Andrew J.; Wise, Paul E.; Watkins, Anthony C.; Goldblatt, Matthew I.; Haut, Elliott R.; Efron, Jonathan E.; Johnston, Fabian M.

doi:10.1016/j.surg.2018.07.002

Cited by 27 publications

(32 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Online risk calculators like the National Surgical Quality Improvement Program Surgical Risk Calculator can reduce variability and increase the likelihood that patients will engage in prehabilitation, but they have time-consuming manual data acquisition and entry requirements, which hinders their clinical adoption. [13][14][15][16][17][18] Emerging technologies can circumvent this problem. The MySurgeryRisk platform autonomously draws data from multiple input sources and uses machine learning techniques to predict postoperative complications and mortality.…”

Section: Discussionmentioning

confidence: 99%

Added Value of Intraoperative Data for Predicting Postoperative Complications: The MySurgeryRisk PostOp Extension

Datta

Loftus

Ruppert

et al. 2020

Journal of Surgical Research

View full text Add to dashboard Cite

Background: Models that predict postoperative complications often ignore important intraoperative events and physiological changes. This study tested the hypothesis that accuracy, discrimination, and precision in predicting postoperative complications would improve when using both preoperative and intraoperative data input data compared with preoperative data alone. Methods: This retrospective cohort analysis included 43,943 adults undergoing 52,529 inpatient surgeries at a single institution during a 5-y period. Random forest machine learning models in the validated MySurgeryRisk platform made patient-level predictions for seven postoperative complications and mortality occurring during hospital admission using electronic health record data and patient neighborhood characteristics. For each outcome, one model trained with preoperative data alone; one model trained with both preoperative and intraoperative data. Models were compared by accuracy, discrimination (expressed as area under the receiver operating characteristic curve), precision (expressed as area under the precision-recall curve), and reclassification indices. Results: Machine learning models incorporating both preoperative and intraoperative data had greater accuracy, discrimination, and precision than models using preoperative data alone for predicting all seven postoperative complications (intensive care unit length of stay >48 h, mechanical ventilation >48 h, neurologic complications including delirium, cardiovascular complications, acute kidney injury, venous thromboembolism, and wound complications), and in-hospital mortality (accuracy: 88% versus 77%; area under the receiver operating characteristic curve: 0.93 versus 0.87; area under the precision-recall

show abstract

Section: Discussionmentioning

confidence: 99%

Added Value of Intraoperative Data for Predicting Postoperative Complications: The MySurgeryRisk PostOp Extension

Datta

Loftus

Ruppert

et al. 2020

Journal of Surgical Research

View full text Add to dashboard Cite

show abstract

“…Traditional clinical decision-support systems require timeconsuming manual data acquisition and entry, which impairs their adoption. 8,33 Even the most successful and widely used static variable cutoff values do not accurately represent individual patient pathophysiology, as reflected by their suboptimal accuracy. 34,43,56 Parametric regression equations also fail to represent the complex, nonlinear associations among input variables, further limiting the accuracy of traditional multivariable regression models.…”

Section: Artificial Intelligence Predictive Analytics and Augmented Decision-makingmentioning

confidence: 99%

Artificial Intelligence and Surgical Decision-making

et al. 2020

View full text Add to dashboard Cite

IMPORTANCE-Surgeons make complex, high-stakes decisions under time constraints and uncertainty, with significant effect on patient outcomes. This review describes the weaknesses of traditional clinical decision-support systems and proposes that artificial intelligence should be used to augment surgical decision-making.OBSERVATIONS-Surgical decision-making is dominated by hypothetical-deductive reasoning, individual judgment, and heuristics. These factors can lead to bias, error, and preventable harm. Traditional predictive analytics and clinical decision-support systems are intended to augment surgical decision-making, but their clinical utility is compromised by time-consuming manual data management and suboptimal accuracy. These challenges can be overcome by automated artificial intelligence models fed by livestreaming electronic health record data with mobile device outputs. This approach would require data standardization, advances in model interpretability, careful implementation and monitoring, attention to ethical challenges involving algorithm bias and accountability for errors, and preservation of bedside assessment and human intuition in the decision-making process.

show abstract

“…Other studies have shown that risk assessment tools are underutilized by trainees, favoring traditional models of communication due to lack of an electronic health record and clinical work ow interruption [25]. This remains true for attending surgeons as well.…”

Section: Discussionmentioning

confidence: 99%

Surgeons’ Attitudes About Use of Preoperative Risk Assessment Tools

Pradhan

Dyas

Bronsert

et al. 2021

Preprint

View full text Add to dashboard Cite

Background: Formal surgical risk assessment tools have been developed to predict risk of adverse postoperative patient outcomes. Such tools accurately predict common postoperative complications, inform patients and providers of likely perioperative outcomes, guide decision making, and improve patient care. However, these are underutilized. We studied the attitudes towards and techniques of how surgeons preoperatively assess risk.Methods: 108 surgical faculty, 95 surgical residents, and 58 affiliate surgeons at a large academic tertiary referral hospital and affiliate community hospitals were emailed a 16-question survey between 8/2019-6/2020. Surveys were analyzed using descriptive statistics (frequency distributions and percentages for categorical variables, means, and standard deviations for continuous variables), and Fisher’s exact test and unpaired t-tests comparing responses by surgical attendings vs. residents.Results: Overall response rates were 50.0% for faculty surgeons, 47.4% for residents, and 36.2% for affiliate surgeons. Only 20.8% of surgeons used risk calculators most or all of the time. Attending surgeons were more likely to use prior experience and current literature while residents used risk calculators more frequently. Risk assessment tools were more likely to be used when predicting major complications and death in older patients with significant risk factors. Greatest barriers for use of risk assessment tools included time, inaccessibility, and trust in accuracy.Conclusions: Even though risk assessment tools are available, few surgeons use them as part of their routine practices. Developers of the tools need to design the tools to better promote the positive features and to avoid the burdens commonly reported by users.

show abstract

Eye of the beholder: Risk calculators and barriers to adoption in surgical trainees

Abstract: Risk assessment tools are underutilized by trainees. Of importance, preference sets of clinicians appear to be unpredictable and may benefit more from a customizable, bespoke approach.

Cited by 27 publications

References 42 publications

Added Value of Intraoperative Data for Predicting Postoperative Complications: The MySurgeryRisk PostOp Extension

Added Value of Intraoperative Data for Predicting Postoperative Complications: The MySurgeryRisk PostOp Extension

Artificial Intelligence and Surgical Decision-making

Surgeons’ Attitudes About Use of Preoperative Risk Assessment Tools

Contact Info

Product

Resources

About