Methodological challenges and proposed solutions for evaluating opioid policy effectiveness

Schuler, Megan S.; Griffin, Beth Ann; Cerdá, Magdalena; McGinty, Emma E.; Stuart, Elizabeth A.

doi:10.1007/s10742-020-00228-2

Cited by 25 publications

(19 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This narrative review presents a comprehensive review of the literature covering recent years to inform conclusions about the gaps, methodologic challenges, and future opportunities for studies evaluating OA prescribing limits. Many methodologic challenges we describe in this review are consistent with those previously described in the literature 53,54 ; however, our review includes additional challenges for consideration and suggestions for improving future research. In addition, our review expands on such previous work by identifying patient and public health outcomes potentially impacted by prescribing limits with little or no published quantitative evaluations.…”

Section: Strengths and Limitationssupporting

confidence: 66%

Evaluating opioid analgesic prescribing limits: A narrative review

Seitz

Janiszewski

Guy

et al. 2022

Pharmacoepidemiology and Drug

View full text Add to dashboard Cite

Purpose In response to the opioid crisis, opioid analgesic guidelines and prescribing limits have proliferated. The purpose of this narrative review is to examine evidence from studies evaluating the patient or public health impact of federal and state opioid analgesic prescribing guidelines and laws, describe gaps and challenges in current research, and highlight opportunities for improving future research. Methods We focused on evidence from a literature review covering 2013 through 2019. We identified 30 studies evaluating opioid analgesic thresholds based on federal policies and guidelines, state laws, and Medicaid state plans that attempt to influence the course of patient care at or when the limit is exceeded (e.g., prior authorization). Results Most studies evaluated changes in prescribing or dispensing patterns of opioid analgesics, largely finding decreases in prescribing after policy enactment. Fewer studies evaluated patient or public health outcomes beyond changes in prescribing and dispensing patterns; results were infrequently stratified by potentially important sociodemographic and clinical factors. No studies assessed the potential for adverse patient outcomes for which we have emerging evidence of harms. Conclusions We describe knowledge gaps and propose opportunities for future research to sufficiently assess the potential impact and unintended consequences of opioid analgesic prescribing laws, regulations, guidelines, and policies.

show abstract

Section: Strengths and Limitationssupporting

confidence: 66%

Evaluating opioid analgesic prescribing limits: A narrative review

Seitz

Janiszewski

Guy

et al. 2022

Pharmacoepidemiology and Drug

View full text Add to dashboard Cite

show abstract

“…( 1)). Indeed, when there are only 2 time periods, a first-difference estimator and fixed effects estimator are identical; with 3 or more time periods, the relative performance of these estimators depends on the degree of autocorrelation in the outcome [17,26].…”

Section: Empirical Models Consideredmentioning

confidence: 99%

“…Additionally, it has been well established that standard error corrections that adjust for violations of the assumed independence of the repeated measures in longitudinal datasets are needed to obtain accurate Type I error rates [12][13][14][15][16]. Despite the wealth of knowledge concerning challenges of and best practices for DID designs in various settings, the applied literature largely does not reflect these insights [17][18][19][20].…”

Section: Introductionmentioning

confidence: 99%

Moving beyond the classic difference-in-differences model: a simulation study comparing statistical methods for estimating effectiveness of state-level policies

Griffin

Schuler

Stuart

et al. 2021

BMC Med Res Methodol

Self Cite

View full text Add to dashboard Cite

Background Reliable evaluations of state-level policies are essential for identifying effective policies and informing policymakers’ decisions. State-level policy evaluations commonly use a difference-in-differences (DID) study design; yet within this framework, statistical model specification varies notably across studies. More guidance is needed about which set of statistical models perform best when estimating how state-level policies affect outcomes. Methods Motivated by applied state-level opioid policy evaluations, we implemented an extensive simulation study to compare the statistical performance of multiple variations of the two-way fixed effect models traditionally used for DID under a range of simulation conditions. We also explored the performance of autoregressive (AR) and GEE models. We simulated policy effects on annual state-level opioid mortality rates and assessed statistical performance using various metrics, including directional bias, magnitude bias, and root mean squared error. We also reported Type I error rates and the rate of correctly rejecting the null hypothesis (e.g., power), given the prevalence of frequentist null hypothesis significance testing in the applied literature. Results Most linear models resulted in minimal bias. However, non-linear models and population-weighted versions of classic linear two-way fixed effect and linear GEE models yielded considerable bias (60 to 160%). Further, root mean square error was minimized by linear AR models when we examined crude mortality rates and by negative binomial models when we examined raw death counts. In the context of frequentist hypothesis testing, many models yielded high Type I error rates and very low rates of correctly rejecting the null hypothesis (< 10%), raising concerns of spurious conclusions about policy effectiveness in the opioid literature. When considering performance across models, the linear AR models were optimal in terms of directional bias, root mean squared error, Type I error, and correct rejection rates. Conclusions The findings highlight notable limitations of commonly used statistical models for DID designs, which are widely used in opioid policy studies and in state policy evaluations more broadly. In contrast, the optimal model we identified--the AR model--is rarely used in state policy evaluation. We urge applied researchers to move beyond the classic DID paradigm and adopt use of AR models.

show abstract

“…But which policies are effective? In their paper, Schuler et al ( 2021 ) provide an overview of methodological challenges facing opioid policy researchers and propose ways forward. Their rigorous evaluation of the key challenges, ranging from obtaining high-quality policy and outcome data to addressing confounding and heterogeneity and overcoming limited statistical power, will be of interest not just to opioid policy researchers but also other researchers conducting policy evaluation using observational data.…”

Section: The Opioid Crisismentioning

confidence: 99%

Guest Editorial: Articles selected from the 2020 International Conference on Health Policy Statistics

Crespi¹,

Harel

2021

Health Serv Outcomes Res Method

View full text Add to dashboard Cite

With the theme "Leveraging Data to Shape the Future", the conference featured an ambitious program covering a broad range of topics (https ://ww2.amsta t.org/meeti ngs/ichps /2020/onlin eprog ram/index .cfm). Conference speakers were invited to submit manuscripts for selection into a special issue of Health Services and Outcomes Research Methodology, with us (OH and CC) serving as guest coeditors. After a peer review process, fifteen articles were selected and are appearing in the December 2020 and March 2021 issues of the journal. These articles showcase the people, methods and applications that characterize the ICHPS conference series.

show abstract

Methodological challenges and proposed solutions for evaluating opioid policy effectiveness

Cited by 25 publications

References 70 publications

Evaluating opioid analgesic prescribing limits: A narrative review

Evaluating opioid analgesic prescribing limits: A narrative review

Moving beyond the classic difference-in-differences model: a simulation study comparing statistical methods for estimating effectiveness of state-level policies

Guest Editorial: Articles selected from the 2020 International Conference on Health Policy Statistics

Contact Info

Product

Resources

About