Evaluation of the DAVROS (Development And Validation of Risk-adjusted Outcomes for Systems of emergency care) risk-adjustment model as a quality indicator for healthcare

Wilson, Richard W.; Goodacre, Steve; Klingbajl, Marcin; Kelly, Anne‐Maree; Rainer, Timothy; Coats, Timothy J; Holloway, V J; Townend, Will; Crane, Steven

doi:10.1136/emermed-2013-202359

Cited by 5 publications

(10 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Risk adjustment for quality improvement spans all medical disciplines from alternative medicine to surgery and is performed by researchers, administrators, government agencies and health insurances. Yet we have not found a published solution on how to effectively isolate quality-of-care differences since Iezzoni’s comment other than approaches that entailed visiting the participating clinics and assessing the attributional validity of their risk adjustment model on site, such as the DAVROS group [ 13 ]. But besides having its own bias issues, this approach, requiring independent experts, is personnel and time intensive.…”

Section: Discussionmentioning

confidence: 99%

“…If the observed outcome exceeds the predicted outcome, then this discrepancy is assumed to be due to poor care. To which degree this assumption is valid is an important but often neglected aspect of the validation of risk-adjustment methods [ 12 , 13 ]. Literature therefore recommends observing predictive and attributional validity to assess the validity of the risk adjustment approaches.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Comparison of three different methods for risk adjustment in neonatal medicine

et al. 2017

View full text Add to dashboard Cite

BackgroundQuality improvement in health care requires identification of areas in need of improvement by comparing processes and patient outcomes within and between health care providers. It is critical to adjust for different case-mix and outcome risks of patient populations but it is currently unclear which approach has higher validity and how limitations need to be dealt with. Our aim was to compare 3 approaches towards risk adjustment for 7 different major quality indicators in neonatal intensive care (21 models).MethodsWe compared an indirect standardization, logistic regression and multilevel approach. Parameters for risk adjustment were chosen according to literature and the condition that they may not depend on processes performed by treating clinics. Predictive validity was tested using the mean Brier Score and by comparing area under curve (AUC) using high quality population based data separated into training and validation sets. Changes in attributional validity were analysed by comparing the effect of the models on the observed-to-expected ratios of the clinics in standardized mortality/morbidity ratio charts.ResultsRisk adjustment based on indirect standardization revealed inferior c-statistics but superior Brier scores for 3 of 7 outcomes. Logistic regression and multilevel modelling were equivalent to one another. C-statistics revealed that predictive validity was high for 8 and acceptable for 11 of the 21 models. Yet, the effect of all forms of risk adjustment on any clinic’s comparison with the standard was small, even though there was clear risk heterogeneity between clinics.ConclusionsAll three approaches to risk adjustment revealed comparable results. The limited effect of risk adjustment on clinic comparisons indicates a small case-mix influence on observed outcomes, but also a limited ability to isolate quality improvement potential based on risk-adjustment models. Rather than relying on methodological approaches, we instead recommend that clinics build small collaboratives and compare their indicators both in risk-adjusted and unadjusted form together. This allows qualitatively investigating and discussing the residual risk-differences within networks. The predictive validity should be quantified and reported and stratification into risk groups should be more widely used to correct for confounding.Electronic supplementary materialThe online version of this article (doi:10.1186/s12887-017-0861-5) contains supplementary material, which is available to authorized users.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Comparison of three different methods for risk adjustment in neonatal medicine

et al. 2017

View full text Add to dashboard Cite

show abstract

“…This supports findings that restricting the calculation of standardised mortality ratios to include only certain conditions reduces over-dispersion of data and may yield a more useful comparative statistic [ 48 ]. This clinically meaningful case selection addresses the criticism of risk-standardised outcomes that there is a poor correlation between quality of care provided and probability of death [ 42 , 49 ]. Until complete and accurate disease-staged outcomes data are available on the same scale as current administrative data, risk-standardised outcomes remain the best measure of clinical outcomes for national studies.…”

Section: Discussionmentioning

confidence: 99%

Variations and inter-relationship in outcome from emergency admissions in England: a retrospective analysis of Hospital Episode Statistics from 2005–2010

Holt

Sinha

Ozdemir

et al. 2014

BMC Health Serv Res

View full text Add to dashboard Cite

BackgroundThe quality of care delivered and clinical outcomes of care are of paramount importance. Wide variations in the outcome of emergency care have been suggested, but the scale of variation, and the way in which outcomes are inter-related are poorly defined and are critical to understand how best to improve services. This study quantifies the scale of variation in three outcomes for a contemporary cohort of patients undergoing emergency medical and surgical admissions. The way in which the outcomes of different diagnoses relate to each other is investigated.MethodsA retrospective study using the English Hospital Episode Statistics 2005–2010 with one-year follow-up for all patients with one of 20 of the commonest and highest-risk emergency medical or surgical conditions. The primary outcome was in-hospital all-cause risk-standardised mortality rate (in-RSMR). Secondary outcomes were 1-year all-cause risk-standardised mortality rate (1 yr-RSMR) and 28-day all-cause emergency readmission rate (RSRR).Results2,406,709 adult patients underwent emergency medical or surgical admissions in the groups of interest. Clinically and statistically significant variations in outcome were observed between providers for all three outcomes (p < 0.001). For some diagnoses including heart failure, acute myocardial infarction, stroke and fractured neck of femur, more than 20% of hospitals lay above the upper 95% control limit and were statistical outliers. The risk-standardised outcomes within a given hospital for an individual diagnostic group were significantly associated with the aggregated outcome of the other clinical groups.ConclusionsHospital-level risk-standardised outcomes for emergency admissions across a range of specialties vary considerably and cross traditional speciality boundaries. This suggests that global institutional infra-structure and processes of care influence outcomes. The implications are far reaching, both in terms of investigating performance at individual hospitals and in understanding how hospitals can learn from the best performers to improve outcomes.

show abstract

“…[1] An accurate assessment of outcomes is imperative because it can promote early appropriate interventions and improve the outcomes of ED patients. [2] There are certain disease states and conditions for which evaluation methods are appropriate and effective, such as ST-elevation myocardial infarction (STEMI), sepsis, and acute stroke. Moreover, several physiologic scoring systems have been demonstrated to be appropriate predictors of mortality for patients admitted to the ED.…”

Section: Introductionmentioning

confidence: 99%

Comparing the effectiveness of three scoring systems in predicting adult patient outcomes in the emergency department

Xiao-jun

Liu

et al. 2019

Medicine

View full text Add to dashboard Cite

This study aimed to evaluate the performance of the rapid acute physiology score (RAPS), the rapid emergency medicine score (REMS), and the modified early warning score (MEWS) in predicting the outcomes of adult patients presenting to the emergency department (ED). A retrospective review was undertaken between February 2014 and February 2018 in an adult ED of a 3300-bed university hospital. The RAPS, REMS, and MEWS were calculated to assess their capability to predict hospital admission, length of hospital stay, and in-hospital mortality, using area under receiver operating characteristic analysis. Multivariate analysis was used to identify variables that were independent predictors of the outcomes. We included 39,977 patients who had presented to the ED during 48 consecutive months, of whom 4857 were admitted and 213 died in hospital. The predictabilities of REMS, RAPS, and MEWS for hospital admission were 0.76, 0.59, and 0.55, respectively; the predictability of REMS, RAPS, and MEWS for hospital mortality were 0.88, 0.72, and 0.73, respectively; and the predictability of REMS, RAPS, and MEWS for length of hospital stay were 0.76, 0.67, and 0.65, respectively. Multivariate analysis showed that the Glasgow coma scale (GCS) (odds ratio (OR), 1.61; P < .001), age (OR, 1.50; P < .001), and MAP (OR, 1.27; P < .001) were independent predictors for hospital admission; GCS (OR, 2.92; P < .001), respiratory rate (RR) (OR, 2.69; P < .001), peripheral oxygen saturation (OR, 2.67; P < .001), MAP (OR, 2.11; P < .001), age (OR, 1.75; P < .001), and pulse rate (PR) (OR, 1.73; P < .001) were independent predictors for in-hospital mortality; and RR (OR, 1.41; P < .001), temperature (OR, 1.05; P = .01), and PR (OR, 0.96; P = .04) were independent predictors for length of hospital stay. Our study evaluated and confirmed the REMS as a powerful predictor of ED adult patient outcomes, including hospital admission, length of hospital stay, and in-hospital mortality compared to RAPS and MEWS.

show abstract

Evaluation of the DAVROS (Development And Validation of Risk-adjusted Outcomes for Systems of emergency care) risk-adjustment model as a quality indicator for healthcare

Cited by 5 publications

References 7 publications

Comparison of three different methods for risk adjustment in neonatal medicine

Comparison of three different methods for risk adjustment in neonatal medicine

Variations and inter-relationship in outcome from emergency admissions in England: a retrospective analysis of Hospital Episode Statistics from 2005–2010

Comparing the effectiveness of three scoring systems in predicting adult patient outcomes in the emergency department

Contact Info

Product

Resources

About