Developing an Outcomes Measurement System: The Value of Testing

Abstract

Outcomes measurement systems often experience similar challenges to implementing data collection and demonstrating value. This article examines the experiences of 451 hospitals participating in a 2-year research effort designed to assess the ability of participants to successfully implement collection of obstetric and peri-operative indicator data for development of an outcomes measurement system. Measures of ability to implement include rate of attrition and reasons for withdrawal, resources expended, months of data transmitted, internal factors that affected ability to operationalize data collection, and assessment of value of participating in the testing process. The findings indicate considerable variation in implementation ability, challenges encountered, and satisfaction with the experience. Several changes in the operational system were made in response to the findings. Many of the lessons learned from the testing experience may be applicable to sponsors of and participants in other outcomes measurement systems.

Intellectual capital and performance measurement in healthcare organizations

“…• Concerning the step (c) "Data Collection and Processing", the following issues could affect the healthcare organization: first, costs increase due to need both for new information technology and related maintenance, and for the necessary managerial expertise; second, people working in the organization could be motivated because they have the impression that, measuring everything, nothing is significant ( Johnston and Fitzgerald, 2001); third, the management turnover, problems with the information systems and the presence of budget constraints (Braun and Zibrat, 1996;Voelker et al, 2001); fourth, lack of the availability of data, poor quality and reflex of performance (Kelman and Smith, 2000;Brown, 2002); and finally, the connections between the institutional, technical and managerial levels could influence the analysis and interpretation of data (Lemieux-Charles et al, 2003).…”

Specifying and Standardizing Performance Measures for Use at a National Level

“…Efforts to develop, test, and implement priority sets of performance measures addressing perioperative care, obstetrical care, trauma care, oncology care, cardiovascular care, medication use, and infection control required the assembly of expert panels to help ensure that measures reflected the best available science and clinical knowledge. Extensive testing and evaluation processes, requiring years of work, were designed to stringently evaluate the reliability and validity of the measures (Braun and Zibrat 1996;Turpin et al 1996). In addition to developing and testing the measures, a reference database was created to receive patient-level data and support the production of quarterly comparative feedback reports (Nadzam, et al 1993;McGreevey, Nadzam, and Corbin 1997).…”

Evidence-based practice for young people who self harm: can it be sustained and does it improve outcomes?

“…It is also necessary to determine if interventions shown to be efficacious in experimental designs also improve patient outcomes in real clinical settings (Braun & Zibrat 1996). As Weisz, Donenberg and Weiss (1995) suggest, the outcomes of clinic therapy may be less positive than for laboratory-based research therapy, and this is likely to be due to differences in the severity of problems, the setting of therapy and the treatment methods used.…”