Misspecification issues in risk adjustment and construction of outcome-based quality indicators

Abstract

Hospital ''report cards'' reporting risk-adjusted health outcomes are increasingly used to benchmark quality of care. However, risk adjustment methods that do not fully account for the interrelationship between quality, risks and outcomes may lead to biased quality measures. This study aims to determine whether the current approach based on logistic regression and observed-to-expected outcome comparisons (O-E difference or O/E ratio) provides unbiased measures of quality. We first provided a conceptual framework to demonstrate that O-E difference or O/E ratio is inconsistently specified when estimates are based on logistic risk adjustment models. To examine the misspecification issue empirically, risk adjustment was performed based on coronary artery bypass graft (CABG) surgery data from New York's Cardiac Surgery Reporting System, and quality indicators (QI) of different specifications were calculated for hospital profiling. Computer simulations further explored the issue of misspecified QIs. Results showed that risk-adjusted mortality rates (RAMR) calculated from different QIs identified the same hospital outliers based on 95% confidence intervals, but generated different rank orders for hospitals in both high-quality and low-quality tails of the quality distributions. Simulation results further showed that, compared to O-E and O/E, logistically transformed QIs were superior regarding their abilities to identify hospitals of true extreme rankings, especially when the outcome was less prevalent or the number of patients per hospital was small. Based on our findings, we recommend that analysts consider the use of logistically transformed QI prior to publicly releasing quality rankings using measures based on O-E or O/E.

The “Nursing Home Compare” Measure of Urinary/Fecal Incontinence: Cross‐Sectional Variation, Stability over Time, and the Impact of Case Mix

“…Expected incontinence rate for each facility ( E j ) was then calculated as ${\displaystyle {\sum }_{j=1}^{n_j}{p}_{ij}}/n_j$. Finally, for each facility in each quarter, we first calculated the risk-adjusted QM as $$\text{logit}\left(Q{M}_j\right)=lntrue(\frac{O_j}{1-O_j}true)-lntrue(\frac{E_j}{1-E_j}true)+lntrue(\frac{\overline{O}}{1-\overline{O}}true)$$ where Ō is the overall incontinence rate for all residents in the quarterly sample ( Ō = 47.4%, 47.6%, and 47.7%, for the 1 st , 2 nd , and 3 rd quarter of 2005, respectively), and then back-transformed logit ( QM j ) to the probability scale (Li et al 2007): $$Q{M}_j={\left(1+{\text{logit}}^{-1}\left(Q{M}_j\right)\right)}^{-1}.$$.…”

Does Mandating Nursing Home Participation in Quality Reporting Make a Difference? Evidence From Massachusetts

“…Studies focused in particular on issues related to measurement of quality 13-20 and salient presentation of the information to consumers. 21-26 Despite these efforts, reports cards are not always effective, 27-30 suggesting there is room for further improvement.…”

Measuring quality in diabetes care: an expert-based statistical approach

“…This intervention-oriented patient-centered view is somewhat distinct from previous studies of quality of care. Existing studies have measured quality in order to assess the US healthcare system ( Schuster et al 1998 ; Corrigan 2001 ; Berwick 2002 ; Davis et al 2007 ; Agency for Healthcare Research and Quality 2006 ; Schoenbaum et al 2007 ; Asch et al 2004 ; Leatherman and McCarthy 2002 ; Hayward 2007 ); to study health plans ( Wholey et al 2003a Wholey et al 2003b ); to identify practices and factors that improve quality ( Heisler and Wagner 2004 ; Solberg et al 2008 ; Hanchate et al 2010 ); to provide rankings of doctors and healthcare facilities ( United States Department Of Health and Human Services 2012 ; Hofer et al 1999 ; Gandhi et al 2002 ; Li et al 2007 ; Shwartz et al 2009 ); and to rate providers for “pay-for-performance” type reimbursement ( Pham et al 2007 ; Scholle et al 2009 ; Romano and Mutter 2004 ).…”