Improving risk adjustment in the PRAiS (Partial Risk Adjustment in Surgery) model for mortality after paediatric cardiac surgery and improving public understanding of its use in monitoring outcomes

Pagel, Christina; Rogers, Libby; Brown, Kate; Ambler, Gareth; Anderson, David; Barron, David J.; Blackshaw, Emily; Crowe, Sonya; English, Kate; Franklin, Rodney; Jesper, E; Meagher, L; Pearson, M; Rakow, Tim; Salamonowicz, M; Spiegelhalter, D; Stickley, John; Thomas, Joanne; Tibby, Shane M.; Tsang,; Utley, Martin; Witter, Thomas

doi:10.3310/hsdr05230

Cited by 13 publications

(21 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We are not able to determine variation in calibration between all of the different centres that contributed to the original PRAIS-2 development as data were not presented by centre in the original documentation of PRAIS-2 development. 13 PRAIS-2 development excluded non-elective procedures and so its accuracy in this group, or whether this would be a valuable predictor, is unknown. We demonstrated that PRAIS-2 does have good discrimination and calibration in this higher risk subgroup.…”

Section: Discussionmentioning

confidence: 99%

“…PRAIS-2 prediction score and its constituent variables PRAIS-2 score is generated from a transformed logistic regression model of 30-day mortality following cardiac surgery. 13 The model included the following perioperative variables: age, weight, diagnosis, procedure group, type of procedure, whether or not there was definite univentricular heart function, additional cardiac risk factors, acquired comorbidity, congenital comorbidity, severity of illness and an additional coefficient for procedures performed after 2013 (online supplemental table S1 shows the units or categories of each of these variables). The formula for the PRAIS-2 score (using the function from the logistic regression model) is where z is the logistic model function of the nine variables.…”

Section: Data Sourcementioning

confidence: 99%

See 1 more Smart Citation

External validation of the improving partial risk adjustment in surgery (PRAIS-2) model for 30-day mortality after paediatric cardiac surgery

et al. 2020

View full text Add to dashboard Cite

ObjectiveIndependent temporal external validation of the improving partial risk adjustment in surgery model (PRAIS-2) to predict 30-day mortality in patients undergoing paediatric cardiac surgery.DesignRetrospective analysis of prospectively collected data.SettingPaediatric cardiac surgery.InterventionPRAIS-2 validation was carried out using a two temporally different single centre (Bristol, UK) cohorts: Cohort 1 surgery undertaken from April 2004 to March 2009 and Cohort 2 from April 2015 to July 2019. For each subject PRAIS-2 score was calculated according to the original formula.ParticipantsA total of 1352 (2004-2009) and 1197 (2015-2019) paediatric cardiac surgical procedures were included in the Cohort 1 and Cohort 2, respectively (median age at the procedure 6.3 and 7.1 months).Primary and secondary outcome measuresPRAIS-2 performance was assessed in terms of discrimination by means of ROC (receiver operating characteristic) curve analysis and calibration by using the calibration belt method.ResultsPRAIS-2 score showed excellent discrimination for both cohorts (AUC 0.72 (95%CI: 0.65 to 0.80) and 0.88 (95%CI: 0.82 to 0.93), respectively). While PRAIS-2 was only marginally calibrated in Cohort 1, with a tendency to underestimate risk in lowrisk and overestimate risk in high risk procedures (P-value = 0.033), validation in Cohort 2 showed good calibration with the 95% confidence belt containing the bisector for predicted mortality (P-value = 0.143). We also observed good prediction accuracy in the non-elective procedures (N = 483;AUC 0.78 (95%CI 0.68 to 0.87); Calibration belt containing the bisector (P-value=0.589).ConclusionsIn a single centre UK-based cohort, PRAIS-2 showed excellent discrimination and calibration in predicting 30-day mortality in paediatric cardiac surgery including in those undergoing non-elective procedures. Our results support a wider adoption of PRAIS-2 score in the clinical practice.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Data Sourcementioning

confidence: 99%

External validation of the improving partial risk adjustment in surgery (PRAIS-2) model for 30-day mortality after paediatric cardiac surgery

et al. 2020

View full text Add to dashboard Cite

show abstract

“…It is extremely common to find non-cardiac health problems that impact on outcomes for children undergoing paediatric cardiac surgery. 163 Recent research has provided us with a better understanding of these diseases in terms of the broad groups which are known to be linked to operative mortality. 160,164 In the current study we applied the most recently developed peer-reviewed grouping for additional conditions or comorbidities from the UK-based PRAiS2 model, which was designed for UK NCHDA data 164 in order to consider these as potential risk factors for morbidity.…”

Section: Comorbiditymentioning

confidence: 99%

“…Our experience during a separate National Institute for Health Research project on developing a website to explain how the audit monitor reports on survival after children's heart surgery 163 highlighted the importance of involving parents in the design and content of information throughout.…”

Section: Introductionmentioning

confidence: 99%

Early morbidities following paediatric cardiac surgery: a mixed-methods study

Brown

Pagel

Ridout

et al. 2020

Health Serv Deliv Res

Self Cite

View full text Add to dashboard Cite

Background Over 5000 paediatric cardiac surgeries are performed in the UK each year and early survival has improved to > 98%. Objectives We aimed to identify the surgical morbidities that present the greatest burden for patients and health services and to develop and pilot routine monitoring and feedback. Design and setting Our multidisciplinary mixed-methods study took place over 52 months across five UK paediatric cardiac surgery centres. Participants The participants were children aged < 17 years. Methods We reviewed existing literature, ran three focus groups and undertook a family online discussion forum moderated by the Children’s Heart Federation. A multidisciplinary group, with patient and carer involvement, then ranked and selected nine key morbidities informed by clinical views on definitions and feasibility of routine monitoring. We validated a new, nurse-administered early warning tool for assessing preoperative and postoperative child development, called the brief developmental assessment, by testing this among 1200 children. We measured morbidity incidence in 3090 consecutive surgical admissions over 21 months and explored risk factors for morbidity. We measured the impact of morbidities on quality of life, clinical burden and costs to the NHS and families over 6 months in 666 children, 340 (51%) of whom had at least one morbidity. We developed and piloted methods suitable for routine monitoring of morbidity by centres and co-developed new patient information about morbidities with parents and user groups. Results Families and clinicians prioritised overlapping but also different morbidities, leading to a final list of acute neurological event, unplanned reoperation, feeding problems, renal replacement therapy, major adverse events, extracorporeal life support, necrotising enterocolitis, surgical infection and prolonged pleural effusion. The brief developmental assessment was valid in children aged between 4 months and 5 years, but not in the youngest babies or 5- to 17-year-olds. A total of 2415 (78.2%) procedures had no measured morbidity. There was a higher risk of morbidity in neonates, complex congenital heart disease, increased preoperative severity of illness and with prolonged bypass. Patients with any morbidity had a 6-month survival of 81.5% compared with 99.1% with no morbidity. Patients with any morbidity scored 5.2 points lower on their total quality of life score at 6 weeks, but this difference had narrowed by 6 months. Morbidity led to fewer days at home by 6 months and higher costs. Extracorporeal life support patients had the lowest days at home (median: 43 days out of 183 days) and highest costs (£71,051 higher than no morbidity). Limitations Monitoring of morbidity is more complex than mortality, and hence this requires resources and clinician buy-in. Conclusions Evaluation of postoperative morbidity provides important information over and above 30-day survival and should become the focus of audit and quality improvement. Future work National audit of morbidities has been initiated. Further research is needed to understand the implications of feeding problems and renal failure and to evaluate the brief developmental assessment. Funding This project was funded by the NIHR Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 8, No. 30. See the NIHR Journals Library website for further project information.

show abstract

“…These audits are designed to allow for performance to be compared with defined standards, and HQIP has published guidance on how to detect and manage outliers in such audits, which includes reference to the importance of case-mix adjustment. 15 The importance of adjustment in centre comparisons is reflected in recent National Institute for Health Research (NIHR)-funded work by Pagel et al 16 They sought to improve their risk adjustment model for 30-day mortality after heart surgery and to improve how information was communicated back to patients. Given that the impact of identifying outliers can be dramatic for centressurgery at one paediatric centre was halted under the original risk modelappropriate risk modelling was identified as important.…”

Section: Use Of Clinical Audits In Improving Carementioning

confidence: 99%

Identifying exceptional cystic fibrosis care services: combining statistical process control with focus groups

MacNeill

Pierotti

Mohammed

et al. 2019

Health Serv Deliv Res

View full text Add to dashboard Cite

Background The Cystic Fibrosis (CF) Registry collects clinical data on all patients attending specialist CF centres in the UK. These data have been used to make comparisons between centres on key outcomes such as forced expiratory volume in 1 second (FEV1) using simple rankings, which promote the assumption that those with the highest measures provide ‘better’ care. Objectives To explore whether or not using statistical ‘process control’ charts that move away from league tables and adjusting for case mix (age, where appropriate; sex; CF genotype; pancreatic sufficiency; and socioeconomic status) could identify exceptional CF care services in terms of clinically meaningful outcomes. Then, using insight from patients and clinicians on what structures, processes and policies are necessary for delivering good CF care, to explore whether or not care is associated with observed differences in outcomes. Design Cross-sectional analyses. Setting Specialist CF centres in the UK. Participants Patients aged ≥ 6 years attending specialist CF centres and clinicians at these centres. Main outcome measures FEV1% predicted. Data sources Annual reviews taken from the UK CF Registry (2007–15). Results We studied FEV1 in many different ways and in different periods. In our analyses of both adult and paediatric centres, we observed that some centres showed repeated evidence of ‘special-cause variation’, with mean FEV1 being greater than the mean in some cases and lower than the mean in others. Some of these differences were explained by statistical adjustment for different measures of case mix, such as age, socioeconomic status, genotype and pancreatic sufficiency. After adjustment, there was some remaining evidence of special-cause variation for some centres. Our data at these centres suggest that there may be an association with the use of intravenous antibiotics. Workshops and focus groups with clinicians at paediatric and adult centres identified a number of structures, processes and policies that were felt to be associated with good care. From these, questionnaires for CF centre directors were developed and disseminated. However, the response rate was low, limiting the questionnaires’ use. Focus groups with patients to gain their insights into what is necessary for the delivery of good care identified themes similar to those identified by clinicians, and a patient questionnaire was developed based on these insights. Limitations Our data analyses suggest that differences in intravenous antibiotic usage may be associated with centre-level outcomes; this needs to be explored further in partnership with the centres. Our survey of centre directors yielded a low response, making it difficult to gain useful knowledge to inform further discussions with sites. Conclusions Our findings confirm that the CF Registry can be used to identify differences in clinical outcomes between centres and that case mix might explain some of these differences. As such, adjustment for case mix is essential when trying to understand how and why centres differ from the mean. Future work Future work will involve exploring with clinicians how care is delivered so that we can understand associations between care and outcomes. Patients will also be asked for their perspectives on the care they receive. Funding The National Institute for Health Research Health Services and Delivery Research programme.

show abstract

Improving risk adjustment in the PRAiS (Partial Risk Adjustment in Surgery) model for mortality after paediatric cardiac surgery and improving public understanding of its use in monitoring outcomes

Cited by 13 publications

References 60 publications

External validation of the improving partial risk adjustment in surgery (PRAIS-2) model for 30-day mortality after paediatric cardiac surgery

External validation of the improving partial risk adjustment in surgery (PRAIS-2) model for 30-day mortality after paediatric cardiac surgery

Early morbidities following paediatric cardiac surgery: a mixed-methods study

Identifying exceptional cystic fibrosis care services: combining statistical process control with focus groups

Contact Info

Product

Resources

About