The Dangers of Gathering Data: Surgeon-Specific Outcomes Revisited

Abstract

The accuracy of risk adjustment is important in developing surgeon profiles. As surgeon profiles are obtained from observational, nonrandomized data, we hypothesized that selection bias exists in how patients are matched with surgeons and that this bias might influence surgeon profiles. We used the Society of Thoracic Surgeons risk model to calculate observed to expected (O/E) mortality ratios for each of six cardiac surgeons at a single institution. Propensity scores evaluated selection bias that might influence development of risk-adjusted mortality profiles. Six surgeons (four high and two low O/E ratios) performed 2298 coronary artery bypass grafting (CABG) operations over 4 years. Multivariate predictors of operative mortality included preoperative shock, advanced age, and renal dysfunction, but not the surgeon performing CABG. When patients were stratified into quartiles based on the propensity score for operative death, 83% of operative deaths (50 of 60) were in the highest risk quartile. There were significant differences in the number of high-risk patients operated upon by each surgeon. One surgeon had significantly more patients in the highest risk quartile and two surgeons had significantly less patients in the highest risk quartile (p < 0.05 by chi-square). Our results show that high-risk patients are preferentially shunted to certain surgeons, and away from others, for unexplained (and unmeasured) reasons. Subtle unmeasured factors undoubtedly influence how cardiac surgery patients are matched with surgeons. Problems may arise when applying national database benchmarks to local situations because of this unmeasured selection bias.

Creating Individual Surgeon Performance Assessments in a Statewide Hospital Surgical Quality Improvement Collaborative

“…Researchers have long attempted to profile provider performance, targeting different types of providers using various measures, with limited success. [1][2][3][4][5][6][7][8] Surgeons have been the focus of numerous profiling efforts focused on postoperative patient outcomes, dating back to the New York Department of Health's publication of mortality rates after coronary artery bypass grafting surgery in the early 1990s. 9 In 2013, the United Kingdom's National Health Service began publishing surgeon-specific patient outcomes.…”

The Effect of Feedback on Surgeon Performance: A Narrative Review

“…Looking at six surgeons within one institution, Ferraris et al found that 83% of deaths occurred within the highest quartile for risk. One surgeon had significantly more high-risk patients ( p < 0.05), demonstrating a shunting of high-risk patients and an inordinate risk burden on select physicians for unmeasured reasons [ 19 ].…”

Mapping standard ophthalmic outcome sets to metrics currently reported in eight eye hospitals

“…At present, hierarchical regression is the gold standard for risk adjustment of outcomes and for producing provider report cards; however this gold standard is rarely used [ 41 ] and no equivalent modeling has been developed in ophthalmology to date. The National Ophthalmic Database (NOD) in the U.K. has compiled a national cataract data set, and participating cataract surgeons have the ability to compare their individual surgical data to others in a risk-adjusted manner [ 28 ].…”