Unintended Consequences of Implementing a National Performance Measurement System into Local Practice

Abstract

Facility-level strategies undertaken to implement national PM systems may result in inappropriate clinical care, can distract providers from patient concerns, and may have a negative effect on patient education and autonomy. Further research is needed to ascertain how features of centralized PM systems influence whether measures are translated locally by facilities into more or less patient-centered policies and processes.

Developing and Testing an Electronic Measure of Screening Colonoscopy Overuse in a Large Integrated Healthcare System

“…We obtained or constructed several potential facility-level predictors of screening overuse: (1) proportion of screen-eligible patients in FY13 who were "up to date" for screening according to current guidelines (assessed by chart review conducted as part of VA's ongoing External Peer Review Program); (2) median number of days between positive fecal occult blood test (FOBT) and colonoscopy ("FOBT wait time") (see below); (3) complexity score (a measure of facility complexity based on factors such as patient risk, academic affiliation, and patient volume); 19 (4) academic affiliation (obtained from the VA Office of Academic Affiliations); (5) number of colonoscopies performed in FY13 (obtained from CDW); and, (6) proportion of colonoscopies "outsourced" to non-VA facilities in FY10 (also known as "fee basis" colonoscopies) (obtained from CDW). To quantify the FOBT wait time, we identified all positive FOBTs in FY12 (N=55,494) and all colonoscopies performed within 12 months after positive FOBT (N = 22,728).…”

“…For example, several recent studies have shown that implementation of underuse measures can unintentionally encourage overuse of care. [4][5][6][7] In contrast, overuse measures have the potential to discourage utilization when care is actually indicated, promoting underuse. In light of such concerns, Mathias and Baker have recommended that the potential unintended effects of overuse measures be carefully considered during measure development, and that efforts be taken to mitigate and monitor for such effects.…”

Factors Underlying Quality Problems with Alcohol Screening Prompted by a Clinical Reminder in Primary Care: A Multi-site Qualitative Study

“…[21][22][23][24]40 Although not without limitation, 41,42 clinical decision support systems have been used to implement evidence-based practices across multiple conditions, [43][44][45][46][47][48][49] and many of these systems are being developed to be shareable across multiple healthcare systems, 50 especially in the U.S. in response to healthcare reform. Findings from the present and prior studies in VA suggest that while VA's alcohol screening clinical reminder has effectively prompted clinical staff to conduct screening and facilitated documentation that screening occurred, 9 it has not facilitated screening in a validated, standardized, reproducible way.…”

The Double Edged Sword of Performance Measurement

“…In this issue of the Journal, Powell and colleagues describe a number of unintended consequences of implementing performance measurement in the Department of Veterans Affairs (VA) Health Care System, 1 an early adopter of system-wide performance management. [2][3][4][5] While there are significant limitations in generalizing the findings of this retrospective qualitative study, it is an important contribution to the growing body of evidence documenting the complexities of health care performance measurement and a poignant reminder that performance measurement is a tool that cuts both ways.…”

“…1 They found local implementation of VA's national performance measurement system led in some instances to provision of inappropriate clinical care, decreased provider attention to patient concerns and service, and compromised patient education and autonomy, as well as some adverse effects on primary care team dynamics. They additionally observed notable variation among the facilities in how performance data were shared with front line clinicians, strategies to improve performance, and application of rules.…”