Confusion Instead of Clarity: Publicly Reported Cardiac Surgery Ratings for Coronary Artery Bypass Grafting and Aortic Valve Replacement

Abstract

BACKGROUND: Public reporting of cardiac surgery ratings has been advocated to inform patient selection of hospitals. Although Society of Thoracic Surgeons (STS) ratings are based on audited riskadjusted patient outcomes, other rating systems rely on administrative databases. In this study, we evaluate correlation among 4 widely used hospital rating systems for coronary artery bypass grafting (CABG) and aortic valve replacement (AVR). STUDY DESIGN: We identified an initial cohort of 602 hospitals from US News & World Report's (USN) listing of the 2016-2017 "Best Hospitals for Cardiology & Heart Surgery." From this cohort, current publicly available CABG and AVR ratings were collected from the STS, USN, Centers for Medicare & Medicaid Services, and Healthgrades. All 4 rating systems rated hospitals as high, average, or below average performers for each procedure. We then determined the match rate between rating systems for individual hospitals and assessed interrater reliability with Cohen's k. RESULTS: Rating systems had different distributions of high and low performing ratings assigned. USN rated hospitals as high performing for both CABG and AVR more frequently compared with STS, Healthgrades, and Centers for Medicare & Medicaid Services. For CABG, the match rate between systems varied from 50% to 85%, with the best match between STS and Centers for Medicare & Medicaid Services. Similarly for AVR, the match rate varied from 50% to 73%, with the best match between STS and Healthgrades. Interrater reliability was poor among the 4 rating systems (k < 0.2) and consistent with no agreement for CABG and AVR ratings. CONCLUSIONS: Publicly reported cardiac surgery ratings have significant discrepancy and poor correlation. This might confuse instead of clarify public perception of hospital quality for cardiac surgery.

Association ofUS News & World ReportTop Ranking for Gastroenterology and Gastrointestinal Operation With Patient Outcomes in Abdominal Procedures

“…[14][15][16] Besides the USNWR ratings, alternative ranking systems have been proposed, including Healthgrades, Consumer Reports, CMS Overall Hospital Quality Star Rating, and the American College of Surgeons National Surgical Quality Improvement Program Meritorious Hospital scoring. 8,17,18 These ranking systems still may be inadequate. Raghuram and colleagues 19 found little correlation among 4 different publicly available hospital ranking systems (USNWR, Healthgrades, CMS, and Society of Thoracic Surgeons) for coronary artery bypass graft or aortic valve replacement.…”

“…These ranking systems still may be inadequate. Raghuram and colleagues found little correlation among 4 different publicly available hospital ranking systems ( USNWR , Healthgrades, CMS, and Society of Thoracic Surgeons) for coronary artery bypass graft or aortic valve replacement. It appears that further studies are needed into the best metrics and methods for assessing and comparing hospitals.…”

Evaluating Quality in Adult Cardiac Surgery

“…62 Poor interrater reliability between hospital rating systems has been found, as well as poor correlation between private media organizations' findings and the STS adult cardiac surgery database. 63 The impact on surgical training is not fully understood. 64 Surgeon-specific mortality data inaccurately apply to patients undergoing multidisciplinary care.…”

Lack of associations between hospital rating and outcomes in patients with an acute coronary syndrome

“…However, due to variation in the focus and measures between rating systems, high performance hospitals within one rating system may not be high performers within another rating system. This confuses patients, providers and purchasers, and makes it difficult for hospitals to understand where to focus their efforts for improvement 5–7. Further, patients may be unable or unwilling to use hospital rankings in decisions regarding choice of healthcare providers.…”