Limitations of Ranking Lists Based on Cardiac Surgery Mortality Rates

Abstract

Background-Ranking lists are a common way of reporting performance in cardiac surgery; however, rankings have shown to be imprecise, yet the extent of this imprecision is unknown. We aimed to determine the precision of, and fluctuations in, ranking lists in the comparison of cardiac surgery mortality rates. Methods and Results-Information on all adult cardiac surgery patients in all 16 cardiothoracic centers in The Netherlands from January 1, 2007, until December 31, 2009, was extracted from the database of the Netherlands Association for Cardio-Thoracic Surgery (nϭ46883). Ranks were assessed using crude and adjusted mortality rates, using a random effects logistic regression model. Risk adjustment was performed using the logistic EuroSCORE. Statistical precision of ranks was assessed with 95% confidence intervals. Additional analyses were performed for patients with isolated coronary artery bypass grafting. The ranking lists, based on mortality rates in 3 consecutive years, showed considerable reshuffling. When all data were pooled, the mean width of the 95% confidence intervals was 10 ranks using crude and 8 ranks using adjusted mortality rates. The large overlap of the confidence intervals across hospitals indicates that rank statistics were not materially different. Results were similar in the isolated coronary artery bypass grafting subgroup. Conclusions-Rankings are an imprecise statistical method to report cardiac surgery mortality rates and prone to (random) fluctuation. Hence, reshuffling of ranks can be expected solely due to chance. Therefore, we strongly discourage the use of ranking lists in the comparison of mortality rates. (Circ Cardiovasc Qual Outcomes. 2012;5:403-409.)

Prevalence of comorbidities and their influence on complications and outcomes of coronary bypass surgery for coronary heart disease in gender perspective

“…При анализе данных с помощью системы ES II оценивались факторы риска, связанные с характеристикой пациента, состоянием сердца и с операцией. Число факторов риска составило 6,9 [5,7], при гендерном анализе -у мужчин 6,3 [5,[3][4][5][6][7]1], у женщин 7,1 [6,[2][3][4][5][6][7]8] (р<0,05). Прогнозируемая госпитальная летальность -1,8 [1,5], в т. ч. у мужчин -1,3 [1,0-2,1], у женщин -3,3 [2,5-4,1] (р<0,05).…”

“…В современной кардиохирургии и анестезиологии отсутствуют общепризнанные методы оценки риска и предоперационной подготовки при КШ в зависимости от наличия сопутствующей и/или ассоциированной патологии с учетом гендерных различий. Так, несмотря на высокую оценку использования в кардиохирургии системы EuroSCORE (ES -European System for Cardiac Operative Risk Evaluation), в публикациях последнего времени накопились данные о недооценке или переоценке летальности на основе оценочной шкалы, в результате чего использование системы ES стало вопросом для дискуссий [7,8]. В обзорной статье, 2012г, обобщены результаты 67 исследований по определению прогностической точности показателя ES; сделан вывод, что оценка риска на основе этой модели не может применяться ни для отбора пациентов на кардиохирургическую операцию, ни для сравнения результатов между отдельными клиниками [9].…”

What is a performance outlier?

“…As a consequence, there is substantial fluctuation from one sampling period to another in the rates of adverse events and performance rankings among providers 32. Longitudinal assessment of provider performance over longer periods of time and investigation of trends are more prudent approaches than relying on results in one sampling period 33…”

Impact of type of procedure and surgeon on EuroSCORE operative risk validation

“…The influence of the hospital unit or a system of national health outcome in cardiac surgery has been studied in this context [ 11 ] . Siregar et al [ 12 ] showed the limitations of showing lists of performance of hospitals and surgeons in the study population of patients undergoing cardiovascular surgery in the Netherlands. The study shows that in a three-year period, there was great variability in the results of hospitals with wide confidence intervals, thus questioning the validity of such comparisons.…”