Evaluation and Enhancement of Calibration in the American College of Surgeons NSQIP Surgical Risk Calculator

Liu, Yaoming; Cohen, Mark E.; Hall, Bruce L.; Ko, Clifford Y.; Bilimoria, Karl Y.

doi:10.1016/j.jamcollsurg.2016.03.040

Cited by 137 publications

(100 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The ACS-NSQIP is well recognized as the leading nationally validated outcomesbased program to measure and improve the quality of surgical care; the accuracy and reproducibility of the data have also been previously demonstrated. [25][26][27] The ACS-NSQIP data are collected in a standardized fashion by dedicated surgical clinical reviewers. Patients are followed up throughout their hospital course and after discharge from hospital to 30 postoperative days.…”

Section: Data Sourcementioning

confidence: 99%

Association of Perioperative Red Blood Cell Transfusions With Venous Thromboembolism in a North American Registry

et al. 2018

View full text Add to dashboard Cite

show abstract

Section: Data Sourcementioning

confidence: 99%

Association of Perioperative Red Blood Cell Transfusions With Venous Thromboembolism in a North American Registry

et al. 2018

View full text Add to dashboard Cite

show abstract

“…33 Additionally, these efforts have also demonstrated the need for continuous improvement of risk prediction models as further data has refined all-procedure risk prediction over time. 17 …”

Section: Discussionmentioning

confidence: 99%

“…12,13 Much of the prior work assessing risk prediction has focused on systematic revisions needed in data collection to reduce the risk modeling variability in a cancer surgery patient population (e.g., oncologic omissions such as cancer staging and neoadjuvant chemotherapy). 14–17 However, a clinically practical question remains unanswered. What is the comparative performance of risk models in cancer versus benign disease populations?…”

Section: Introductionmentioning

confidence: 99%

The independent effect of cancer on outcomes: a potential limitation of surgical risk prediction

Leeds

Canner

Efron

et al. 2017

Journal of Surgical Research

View full text Add to dashboard Cite

BACKGROUND Cancer patients are often thought to have worse surgical outcomes. There is a growing view that risk models do not adequately predict these outcomes. This study aims to compare the use of common risk models for benign versus malignant gastrointestinal disease. MATERIALS AND METHODS The NSQIP 2005–2015 participant use files were queried for patients undergoing elective surgery for benign and malignant diseases with a primary procedure code for major colon, pancreas, or stomach resection. Multivariate logistic regression was performed to identify independent predictors of mortality and morbidity. RESULTS We identified 264,401 cases (111,563 malignant). The gastrointestinal cancer population was disproportionately male, older than 65, non-white, and less functionally independent. Comorbidities more common in the cancer population included diabetes, hypertension, dyspnea, and COPD. Cancer patients had a longer length of stay (+0.9 days), higher mortality rate (1.7% vs. 1.1%), and higher complication rate (27.4% vs. 23.2%). NSQIP prediction models for complications in cancer versus non-cancer patients underperformed for predicting mortality (p < 0.001). Multivariate regression demonstrated that a diagnosis of cancer requiring surgery independently conferred an 18% increased odds of death, a 9% increased odds of a complication, and an 8% increased odds of multiple complications compared to patients with benign disease. CONCLUSIONS NSQIP prediction models less effectively evaluate the risk of death in cancer patients as compared to patients with benign disease. A diagnosis of cancer is independently associated with an increased risk of surgical complications. Incorporating cancer diagnosis into surgical risk models may better inform patient and surgeon expectations.

show abstract

“…One major shortcoming of the calculator for proctectomy was its failure to include bleeding—the most frequent complication in this practice—as a complication 7. Other studies have found that the risk calculator was inaccurate in its prediction of outcomes when serious complications occur,8 and often had a tendency for predicted risk to be overestimated for lowest and highest risk patients and underestimated for moderate-risk patients 10…”

Section: Discussionmentioning

confidence: 99%

Examining the Validity of the Acs-Nsqip Risk Calculator in Plastic Surgery: Lack of Input Specificity, Outcome Variability and Imprecise Risk Calculations

Johnson

Campwala

Gupta

2017

Journal of Investigative Medicine

View full text Add to dashboard Cite

American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) created the Surgical Risk Calculator, to allow physicians to offer patients a risk-adjusted 30-day surgical outcome prediction. This tool has not yet been validated in plastic surgery. A retrospective analysis of all plastic surgery-specific complications from a quality assurance database from September 2013 through July 2015 was performed. Patient preoperative risk factors were entered into the ACS Surgical Risk Calculator, and predicted outcomes were compared with actual morbidities. The difference in average predicted complication rate versus the actual rate of complication within this population was examined. Within the study population of patients with complications (n=104), the calculator accurately predicted an above average risk for 20.90% of serious complications. For surgical site infections, the average predicted risk for the study population was 3.30%; this prediction was proven only 24.39% accurate. The actual incidence of any complication within the 4924 patients treated in our plastic surgery practice from September 2013 through June 2015 was 1.89%. The most common plastic surgery complications include seroma, hematoma, dehiscence and flap-related complications. The ACS Risk Calculator does not present rates for these risks. While most frequent outcomes fall into general risk calculator categories, the difference in predicted versus actual complication rates indicates that this tool does not accurately predict outcomes in plastic surgery. The ACS Surgical Risk Calculator is not a valid tool for the field of plastic surgery without further research to develop accurate risk stratification tools.

show abstract

Evaluation and Enhancement of Calibration in the American College of Surgeons NSQIP Surgical Risk Calculator

Cited by 137 publications

References 18 publications

Association of Perioperative Red Blood Cell Transfusions With Venous Thromboembolism in a North American Registry

Association of Perioperative Red Blood Cell Transfusions With Venous Thromboembolism in a North American Registry

The independent effect of cancer on outcomes: a potential limitation of surgical risk prediction

Examining the Validity of the Acs-Nsqip Risk Calculator in Plastic Surgery: Lack of Input Specificity, Outcome Variability and Imprecise Risk Calculations

Contact Info

Product

Resources

About