Validity of the risk adjustment approach to compare outcomes

Silva, Letícia Krauss

doi:10.1590/s0102-311x2003000100032

Cited by 13 publications

(12 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In previous studies, attributional validation has used explicit or implicit review 2 6. Both involve investigators examining site-level clinical and other material and then reaching a judgement as to the quality of care being delivered.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of the DAVROS (Development And Validation of Risk-adjusted Outcomes for Systems of emergency care) risk-adjustment model as a quality indicator for healthcare

et al. 2013

View full text Add to dashboard Cite

Background and objectiveRisk-adjusted mortality rates can be used as a quality indicator if it is assumed that the discrepancy between predicted and actual mortality can be attributed to the quality of healthcare (ie, the model has attributional validity). The Development And Validation of Risk-adjusted Outcomes for Systems of emergency care (DAVROS) model predicts 7-day mortality in emergency medical admissions. We aimed to test this assumption by evaluating the attributional validity of the DAVROS risk-adjustment model.MethodsWe selected cases that had the greatest discrepancy between observed mortality and predicted probability of mortality from seven hospitals involved in validation of the DAVROS risk-adjustment model. Reviewers at each hospital assessed hospital records to determine whether the discrepancy between predicted and actual mortality could be explained by the healthcare provided.ResultsWe received 232/280 (83%) completed review forms relating to 179 unexpected deaths and 53 unexpected survivors. The healthcare system was judged to have potentially contributed to 10/179 (8%) of the unexpected deaths and 26/53 (49%) of the unexpected survivors. Failure of the model to appropriately predict risk was judged to be responsible for 135/179 (75%) of the unexpected deaths and 2/53 (4%) of the unexpected survivors. Some 10/53 (19%) of the unexpected survivors died within a few months of the 7-day period of model prediction.ConclusionsWe found little evidence that deaths occurring in patients with a low predicted mortality from risk-adjustment could be attributed to the quality of healthcare provided.

show abstract

Section: Methodsmentioning

confidence: 99%

“…Whether this assumption is valid is an important but often neglected aspect of the validation of risk-adjustment methods. Attributional validity refers to that aspect of validating a measure that encompasses testing the assumption that changes seen in the risk-adjusted outcome measure reflect differences in care quality 2 6…”

Section: Introductionmentioning

confidence: 99%

Evaluation of the DAVROS (Development And Validation of Risk-adjusted Outcomes for Systems of emergency care) risk-adjustment model as a quality indicator for healthcare

et al. 2013

View full text Add to dashboard Cite

show abstract

“…Silva [9] attempted to evaluate the worthiness of riskadjustment methodology in technologically advanced healthcare environments. The research found that risk-adjustment becomes more complex and biased, as we add variables con-cerning the technological level of healthcare services.…”

Section: Literature Reviewmentioning

confidence: 99%

Two-stage hospital efficiency analysis including qualitative evidence: A Greek case

Nektarios

Constantopoulos

et al. 2016

JHA

View full text Add to dashboard Cite

Background: The European Union health policy agenda stresses the importance of environmental and qualitative factors in structural hospital reforms. In response to the economic crisis, both cost containment and performance improvements of the Greek hospital sector, have become a pertinent issue for overall reforms. Objective: The study examines the efficiency of 112 Greek public hospitals, by applying bootstrapping techniques and investigating the effect of contextual factors on hospital efficiency. Furthermore, the effect of qualitative evidence, on hospital efficiency is explored by focusing on a subset of 28 large hospitals. Methods: The quality aspects of the Greek hospitals are investigated by applying two models of Data Envelopment Analysis (DEA), augmented by bootstrapping techniques, in order to assess the importance of quality dimensions on the efficiency of hospital scores. In addition, two Tobit regression models are estimated assessing the contribution of contextual factors, in the efficiency and bias-corrected efficiency scores. Results: Efficiency analysis indicated that only 23.2% of the hospitals are fully efficient (0.96-1.00), 37.5% are efficient (0.71-0.95) while 39.3% are inefficient (0.30-0.70). The Kolmogorov-Smirnov test, between the original and the bootstrapcorrected efficiency, indicates that their distributions are significantly different (p-value < .001). The environmental factors, influencing efficiency, are Occupancy Rate and the ratio between Outpatient Visits and Inpatient Days. Results indicate that the inclusion of Risk-Adjustment Mortality Rate significantly influences (p-value < .05) the efficiency of the hospitals. Conclusions: In the era of economic crisis, the inclusion of quality variables and the use of bootstrapping techniques provide a vital framework in assessing the efficiency of the hospital sector.

show abstract

“…This is defined as the degree to which variation in the risk-adjusted outcome can be attributed to the quality of care provided 22. The attributional validity of risk-adjustment methods is rarely formally evaluated, and methods are poorly developed 22 23. Review methods can be used to evaluate quality of care at institutions, and then compared with risk-adjusted outcomes.…”

Section: Can Variation In Risk-adjusted Mortality Be Explained By Variation In Healthcare?mentioning

confidence: 99%

What do hospital mortality rates tell us about quality of care?: Table 1

2013

View full text Add to dashboard Cite

Hospital mortality rates could be useful indicators of quality of care, but careful statistical analysis is required to avoid erroneously attributing variation in mortality to differences in health care when it is actually due to differences in case mix. The summary hospital mortality indicator is currently used by the English National Health Service (NHS). It adjusts mortality rates up to 30 days after discharge for patient age, sex, type of admission, year of discharge, comorbidity, deprivation and diagnosis. Such risk-adjustment methods have been used to identify poor performance, most notably at mid-Staffordshire NHS Foundation Trust, but their use is subject to a number of limitations. Studies exploring whether variation in risk-adjusted mortality can be explained by variation in healthcare have reached conflicting conclusions. Furthermore, concerns have been raised that the proportion of preventable deaths among hospital admissions is too small to produce a reliable 'signal' in risk-adjusted mortality rates. This provides hospital managers, regulators and clinicians with a considerable dilemma. Variation in mortality rates cannot be ignored, as they might indicate unacceptable variation in healthcare and avoidable mortality, but they also cannot be reliably used to judge the quality of healthcare, based on current evidence.

show abstract

Validity of the risk adjustment approach to compare outcomes

Cited by 13 publications

References 26 publications

Evaluation of the DAVROS (Development And Validation of Risk-adjusted Outcomes for Systems of emergency care) risk-adjustment model as a quality indicator for healthcare

Evaluation of the DAVROS (Development And Validation of Risk-adjusted Outcomes for Systems of emergency care) risk-adjustment model as a quality indicator for healthcare

Two-stage hospital efficiency analysis including qualitative evidence: A Greek case

What do hospital mortality rates tell us about quality of care?: Table 1

Contact Info

Product

Resources

About