Can the American College of Surgeons Risk Calculator Predict 30-Day Complications After Cervical Spine Surgery?

Abstract

Study Design: This was a retrospective cohort study. Objective: The objective of this study was to assess the American College of Surgeons (ACS) Risk Calculator’s ability to accurately predict complications after cervical spine surgery. Summary of Background Data: Surgical risk calculators exist in many fields and may assist in the identification of patients at increased risk for complications and readmissions. Risk calculators may allow for improved outcomes, an enhanced informed consent process, and management of modifiable risk factors. The American College of Surgeons National Surgery Quality Improvement Program (ACS NSQIP) Risk Calculator was developed from a cohort of over 1.4 million patients, using 2805 unique Current Procedural Terminology (CPT) codes. The risk calculator uses 21 patient predictors (eg, age, American Society of Anesthesiologists class, body mass index, hypertension) and the planned procedure (CPT code) to predict the chance that patients will have any of 12 different outcomes (eg, death, any complication, serious complication, reoperation) within 30 days following surgery. The purpose of this study is to determine if the ACS NISQIP risk calculator can predict 30-day complications after cervical fusion. Methods: A retrospective chart review was performed on patients that underwent primary cervical fusions between January 2009 and 2015 at a single institution, utilizing cervical fusion CPT codes. Patients without 30 days of postoperative follow-up were excluded. Descriptive statistics were calculated for the overall sample, anterior versus posterior fusion, and single versus multilevel fusion. Logistic regression models were fit with actual complication occurrence as the dependent variable in each model and ACS estimated risk as the independent variable. The c-statistic was used as the measure of concordance for each model. Receiver operating characteristic curves were plotted to visually depict the predictive ability of the estimated risks. Acceptable concordance was set at c>0.80. All analyses were conducted using SAS, v9.4. Results: A total of 404 patients met the inclusion criteria for this study. Age, body mass index, sex, and a number of levels of fusion were gathered as input data the ACS NSQIP Risk Calculator. Results of Risk Calculator were compared with observed complication rates. Descriptive statistics of the Risk Calculator risk estimates showed a significant prediction of “any complication” and “discharge to skilled nursing facility” among the cohort. Because there were no deaths or urinary tract infections, no models were fit for these outcomes. Conclusions: The ACS Risk Calculator accurately predicted complications in the categories of “any complication” (P<0.0001) and “discharge to the skilled nursing facility” (P<0.001) for our cohort. We conclude that the ACS Risk Calculator was unable to accurately predict specific complications on a more granular basis for the patients of this study. Although the ACS risk calculator may be useful in the field of general surgery and in the development of new institutional strategies for risk mitigation, our findings demonstrate that it does not necessarily provide accurate information for patients undergoing cervical spinal surgery.

Validation of the ACS-NSQIP Risk Calculator: A Machine-Learning Risk Tool for Predicting Complications and Mortality Following Adult Spinal Deformity Corrective Surgery

“…A study by McCarthy et al applied the calculator to a retrospective cohort of 237 lumbar and 404 cervical spine surgery patients and found no significant outcome prediction in the lumbar cohort. 26 This inability to detect a difference may have been due to an insufficient sample size, as the cervical cohort was able to significantly (P < 0.001) predict any complication as well as discharge to a skilled nursing facility. 26,27 The studies on spine surgery have been typically underpowered to demonstrate true efficacy of the ACS-NSQIP risk calculator at predicting postoperative outcomes.…”

“…26 This inability to detect a difference may have been due to an insufficient sample size, as the cervical cohort was able to significantly (P < 0.001) predict any complication as well as discharge to a skilled nursing facility. 26,27 The studies on spine surgery have been typically underpowered to demonstrate true efficacy of the ACS-NSQIP risk calculator at predicting postoperative outcomes. However, there are several independent risk calculators being utilized that have showed promise in predicting spine surgery outcomes and can be used in conjunction with the risk calculator, or to further validate its usefulness in spine surgery.…”

Evaluation of complications after laparoscopic and open appendectomy by the American College of Surgeon National Surgical Quality Improvement Program Surgical Risk Calculator

“…McCarthy et al[ 22 ] found that the SRC predicted with a high degree of accuracy, the development of any complications, and discharge to nursing home (p<0.0001). They concluded that the SRC could not accurately predict specific complications in greater detail for these patients nor in cervical spinal surgery.…”

Performance Assessment of the American College of Surgeons Risk Calculator in Metastatic Spinal Tumor Surgery

“…Since its conception, the accuracy of the ACS-NSQIP calculator in predicting outcomes has been investigated in many patient populations and procedures 14–21. To date, no study has been done to validate the ACS-NSQIP calculator in patients receiving surgical intervention for spinal metastases, despite previous work showing this calculator to be unreliable in other cancer types 22–24.…”

“…[11][12][13] Since its conception, the accuracy of the ACS-NSQIP calculator in predicting outcomes has been investigated in many patient populations and procedures. [14][15][16][17][18][19][20][21] To date, no study has been done to validate the ACS-NSQIP calculator in patients receiving surgical intervention for spinal metastases, despite previous work showing this calculator to be unreliable in other cancer types. [22][23][24] Thus, the purpose of this study was to investigate the accuracy of the ACS-NSQIP surgical risk calculator in predicting postoperative outcomes in the metastatic spine disease population.…”