Measuring Prescribing Improvements in Pragmatic Trials of Educational Tools for General Practitioners

Abstract

Randomized pragmatic trials of drugs, physician education and drug policies are needed to improve pharmacosurveillance and cost-effectiveness of prescribing. Since 1994, we have developed and tested methods for low-cost education and policy trials to improve prescribing in primary care in Canada. We review methodology for using drug claims and other health services data to evaluate prescribing improvement programs and policies. We apply the lessons to a proposed trial of physician education tools (PET) for quality improvement of prescribing. Design issues for the trial include defining the potential programme in causal terms using counterfactuals, narrowing the denominator to the population affected, excluding noise from the numerator, calculating the prescribing preference, adjusting for baseline differences, controlling for modifiers and confounders, accounting for uncertainty when measuring impacts, and grouping practices for feedback and recognition. Data from a randomized trial of academic detailing illustrate measurement challenges. A decade of progress on methods for evaluating prescribing improvement programs with drug claims data has enabled planning of routine randomized pragmatic trials of education and policies in primary care in Canada.Two connected challenges in pharmacology are how to improve pharmacosurveillance (Laupacis et al. 2003), and how providers and payers can share responsibility and credit for improving the cost-effectiveness of prescribing (Ashworth et al. 2004). For both purposes, we need low-cost randomized pragmatic trials of drugs, physician education and drug policies in primary care. Recalling the philosophy of pragmatism with its focus on outcomes in normal life (Wiener 1973), the adjective 'pragmatic' means the trials are done in real-world settings with everyday patients and practices (Godwin et al. 2003). To establish capacity and infrastructure for such trials, we are focusing initially on the second challenge.A systematic review of research on implementing guidelines for practice improvement concluded that a) disseminating printed educational materials, b) audit with feedback, and c) multifaceted interventions with educational outreach, change physician behaviours by only about 6% to 8%

Change in antihypertensive drug prescribing after guideline implementation: a controlled before and after study

“…In the following, we will focus on measuring the impact of an intervention accurately in the context of pharmacy claims databases and give specific consideration to our outcome measures in the light of the issues raised by Maclure et al (2006) in their review of measuring prescription-related improvements in the primary care setting [39]. …”

“…the definition of the patient group in which the outcome is intended to be measured, is critical. As Maclure and co-workers stated, every patient in the denominator should be at risk of entering the numerator [39]. Therefore, new patients visiting a physician - i.e.…”

Rerandomization to Balance Tiers of Covariates

“…When covariates vary in importance, a natural choice is to impose balance restrictions on each covariate individually, as suggested by Moulton (2004), Maclure et al (2006), and Cox (2009). We illustrate the sub-optimal nature of this with two simple examples, the first with two uncorrelated covariates, and the second with two correlated covariates, both shown in Figure 4.…”

A Brief Overview of Academic Detailing in Canada: Another Role for Pharmacists

“…The research is also a testing ground for the development of common and practical impact evaluation methods for the ongoing assessment of health improvement strategies (e.g., use of the randomized designed delay trial method for comparing physician prescribing). 32 Changes in prescribing will be reviewed and the magnitude of impact on health outcomes will be analyzed with a meta-analysis of aggregate level results performed across the provinces. The study also includes a qualitative review of physician and pharmacist/detailer feedback to better understand the factors that contribute to the effective delivery of academic detailing programs.…”