Assessing the reliability of standardized performance indicators

Williams, Scott C.; Watt, Ann; Schmaltz, Stephen; Koss, Richard G.; Loeb, Jerod M.

doi:10.1093/intqhc/mzi098

Cited by 37 publications

(19 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The measures then underwent extensive pilot testing and validation by the JCAHO, with most measures demonstrating agreement rates of more than 90% on reabstractions done at a sample of hospitals. 8 Samples of the CMS data are audited to ensure that the data being reported are accurate and, through a quality improvement organization, the data are validated by reabstracting a sample of medical records. 6,9 We focused on processes of care for pneumonia, AMI, and CHF ( Table 1).…”

Section: Hospital Quality Datamentioning

confidence: 99%

Quality of Care for the Treatment of Acute Medical Conditions in US Hospitals

Landon

Normand²,

Lessler³

et al. 2006

Arch Intern Med

100

View full text Add to dashboard Cite

Section: Hospital Quality Datamentioning

confidence: 99%

Quality of Care for the Treatment of Acute Medical Conditions in US Hospitals

Landon

Normand²,

Lessler³

et al. 2006

Arch Intern Med

100

View full text Add to dashboard Cite

“…2 Each of the measures included in the composite has been precisely defined, standardized, rigorously tested, and implemented on a national scale. 22,23 When viewed together, through the window of the national comparative performance measurement database of the Joint Commission, the measures allow valid comparisons of the provision of healthcare processes across organizations. 5 Aggregating these process measures into a cardiovascular composite measure or index combines many aspects of care into a single score or rating.…”

Section: Williams Et Al Hospital Performance 561mentioning

confidence: 99%

Performance of Top-Ranked Heart Care Hospitals on Evidence-Based Process Measures

et al. 2006

Self Cite

View full text Add to dashboard Cite

Background— Despite the increasing availability of evidence-based clinical performance measure data that compares the performances of US hospitals, the general public continues to rely on more popular resources such as the US News & World Report annual publication of “America’s Best Hospitals” for information on hospital quality. This study evaluated how well hospitals ranked on the US News & World Report list of top heart and heart surgery hospitals performed on acute myocardial infarction and heart failure measures derived from American College of Cardiology and American Heart Association clinical treatment guidelines. Methods and Results— This study identified 774 hospitals, including 41 of the US News & World Report top 50 heart and heart surgery hospitals. To compare hospitals, 10 rate-based performance measures (6 addressing processes of acute myocardial infarction care and 4 addressing heart failure care), were aggregated into a cardiovascular composite measure. As a group, the US News & World Report hospitals performed statistically better than their peers (mean, 86% versus 83%; P <0.05). Individually, however, only 23 of the US News & World Report hospitals achieved statistically better-than-average performance compared with the population average, whereas 9 performed significantly worse ( P <0.05). One hundred sixty-seven hospitals in this study routinely implemented evidenced-based heart care ≥90% of the time. Conclusions— A number of the US News & World Report top hospitals fell short in regularly applying evidenced-based care for their heart patients. At the same time, many lesser known hospitals routinely provided cardiovascular care that was consistent with nationally established guidelines.

show abstract

“…From one perspective in our study, the ability of TJC criteria to track patients with STEMI is excellent (66/66, 100%): this actually exceeds the accepted accuracy of the acute myocardial infarction ICD-9 code and TJC/CMS margin of error for abstraction using that ICD-9 code. [21][22][23][24] Viewed from another perspective, the ability of TJC criteria to track patients with STEMI is incomplete: for all individuals presenting directly to the PCI center who underwent PCI, TJC identified only 66/97 (68%). When transfer patients who underwent PCI are included, the ability of TJC criteria to track STEMI patients is poor (66/402, 16%), and it is important to recognize that a significant number of patients with STEMI who do not undergo PCI remain excluded from this denominator.…”

Section: Discussionmentioning

confidence: 99%

ST-Elevation Myocardial Infarction

Campbell

Satran

Larson

et al. 2009

Circ: Cardiovascular Quality and Outcomes

View full text Add to dashboard Cite

Background-In the United States, efforts are underway to improve timely access to percutaneous coronary intervention in ST-elevation myocardial infarction (STEMI). The Joint Commission (TJC) and the American College of Cardiology National Cardiovascular Data Registry (NCDR) have developed standardized definitions and clinical performance measures for STEMI. The purpose of this study was to determine differences in 3 quality-assurance registries for STEMI patients. Methods and Results-STEMI patients presenting to the Minneapolis Heart Institute at Abbott Northwestern Hospital (Minneapolis, Minn) are tracked by 3 distinct quality assurance programs: NCDR, TJC, and the level 1 MI registry (a regional system for percutaneous coronary intervention in STEMI which includes transfer patients). Over 1 year, we examined consecutive STEMI patients in level 1 and compared them with individuals meeting NCDR and TJC inclusion criteria. Of 501 STEMI patients treated using the level 1 MI protocol, 422 patients had a clear culprit (402 percutaneous coronary intervention, 13 coronary artery bypass grafting, 7 medical management). In the same period, 282 patients met inclusion criteria for NCDR (56% of the level 1 population), and 66 met inclusion criteria for TJC (13% of the level 1 population). Transfer patients (nϭ380) accounted for 87% of the discrepancy between level 1 and TJC. Pharmacoinvasive percutaneous coronary intervention (nϭ102) accounted for 47% of the discrepancy between level 1 and NCDR. Conclusions-Current inclusion criteria for enrollment in STEMI registries are not uniform. This may lead to variable quality assurance outcomes for the same patient cohort and has important implications for standardized quality measurement. (Circ Cardiovasc Qual Outcomes. 2009;2:648-655.)

show abstract

Assessing the reliability of standardized performance indicators

Abstract: Although improvement in the accuracy and completeness of the self-reported data is possible and desirable, the baseline level of data reliability appears to be acceptable for indicators used to assess and improve hospital performance on selected clinical topics.

Cited by 37 publications

References 11 publications

Quality of Care for the Treatment of Acute Medical Conditions in US Hospitals

Quality of Care for the Treatment of Acute Medical Conditions in US Hospitals

Performance of Top-Ranked Heart Care Hospitals on Evidence-Based Process Measures

ST-Elevation Myocardial Infarction

Contact Info

Product

Resources

About