Interobserver reliability of attending physicians and bedside nurses when using an inpatient paediatric respiratory score

Biondi, Eric; Gottfried, Julie Albright; Fioravanti, Irene Dutko; Schriefer, Jan; Aligne, C. Andrew; Leonard, Michael S.

doi:10.1111/jocn.12737

Cited by 7 publications

(6 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Third, agreement between providers at this institution for assessing the variables in the tool is generally fair to moderate(14, 26, 27). Prior studies have shown pediatric clinicians have moderate(28, 29) to substantial(30) agreement in the assessment of retractions(27). External validation of this scoring system and further assessment of inter-rater agreement is needed, especially in non-ICU patients, older children, non-cardiac surgery patients, and those with new-onset respiratory illness.…”

Section: Discussionmentioning

confidence: 99%

Derivation and Validation of an Objective Effort of Breathing Score in Critically Ill Children

Shein

Hotz

Khemani

2019

Pediatric Critical Care Medicine

View full text Add to dashboard Cite

Objective: To derive and validate a score that correlates with an objective measurement of a child’s effort of breathing (EOB). Design: Secondary analysis of a previously conducted observational study. Setting: The pediatric and cardiothoracic Intensive Care Units of a quaternary-care children’s hospital. Patients: Patients more than 37 weeks gestational age to age 18 years who were undergoing extubation. Interventions: EOB was measured in patients following extubation using esophageal manometry to calculate pressure rate product (PRP). Simultaneously, members of a multidisciplinary team (nurse, physician and respiratory therapist) assessed respiratory function using a previously validated tool. Elements of the tool that were significantly associated with PRP in univariate analysis were identified and included in a multivariate model. An EOB score was derived from the results of the model using data from half of the subjects (Derivation cohort), and then validated using data from the remaining subjects (Validation cohort) by calculating the area under the receiver-operator characteristic curve (ROC AUC) for PRP >90th percentile and for the need for reintubation. Measurements and Main Results: Among 409 subjects, the median age was 5 months and nearly half were cardiac surgery patients (49.1%). Retractions, stridor and pulsus paradoxus were included in the Simple Score. ROC AUC for PRP >90th percentile was 0.8359 (95% CI: 0.7996–0.8722) in the Derivation cohort and 0.7930 (0.7524–0.8337) in the Validation cohort. ROC AUC for reintubation was 0.7280 (0.6807–0.7752) when all scores were analyzed individually, and was 0.7548 (0.6644–0.8452) if scores from three clinicians from different disciplines were summated. Results were similar regardless of provider discipline or training. Conclusions: A scoring system was derived and validated, performed acceptably to predict increased EOB or need for advanced respiratory support, and may function best when used by a team.

show abstract

Section: Discussionmentioning

confidence: 99%

Derivation and Validation of an Objective Effort of Breathing Score in Critically Ill Children

Shein

Hotz

Khemani

2019

Pediatric Critical Care Medicine

View full text Add to dashboard Cite

show abstract

“…In line with this, high interobserver agreement has been reported previously in a reliability analysis of the pediatric asthma score using the average measures of multiple observers. 15 However, we believe that to function well in daily practice, interobserver agreement of a clinical WOB score should be evaluated in the context of observations by single raters (e.g., at various time points before and after physician/nurse rotations). Indeed, in our study calculated average measure ICCs were high, contrasting with the relatively low single-measure ICCs (primary outcome), suggesting poor reliability of individual clinical judgment of WOB in our cohort.…”

Section: Discussionmentioning

confidence: 99%

Interobserver Agreement on Clinical Judgment of Work of Breathing in Spontaneously Breathing Children in the Pediatric Intensive Care Unit

Groot

Neef

Otten

et al. 2019

J Pediatr Intensive Care

View full text Add to dashboard Cite

Clinical assessment of the work of breathing (WOB) remains a cornerstone in respiratory support decision-making in the pediatric intensive care unit (PICU). In this study, we determined the interobserver agreement of 30 observers (PICU physicians and nurses) on WOB and multiple signs of effort of breathing in 10 spontaneously breathing children admitted to the PICU. By reliability analysis, the agreement on overall WOB was poor to moderate, and only three separate signs of effort of breathing (breathing rate, stridor, and grunting) showed moderate-to-good interobserver reliability. We conclude that the interobserver agreement on the clinical WOB judgment among PICU physicians and nurses is low.

show abstract

“…Although few studies have validated the PAS in the ED setting, the use of a PAS-based pathway for hospitalized patients decreased length of stay and cost of care 8 . Initial ED studies of PAS showed excellent interrater reliability (IRR) with a Pearson correlation statistic of 0.92 7 and a Cohen κ of 0.82 9 . Anecdotally, we have noted significant discrepancies between nursing, respiratory therapist, and physician PAS scoring when evaluating pediatric ED patients with acute asthma.…”

mentioning

confidence: 99%

“…8 Initial ED studies of PAS showed excellent interrater reliability (IRR) with a Pearson correlation statistic of 0.92 7 and a Cohen κ of 0.82. 9 Anecdotally, we have noted significant discrepancies between nursing, respiratory therapist, and physician PAS scoring when evaluating pediatric ED patients with acute asthma. In addition, a recent study from Johnson et al 10 demonstrated lower than previously demonstrated IRR on numerous asthma severity scores (although PAS was not evaluated).…”

mentioning

confidence: 99%

Interrater Reliability of the Pediatric Asthma Score

Gardiner

Wilkinson

2021

Pediatr Emer Care

View full text Add to dashboard Cite

Objectives:The aim of this study was to determine the interrater reliability (IRR) of the Pediatric Asthma Score (PAS) and to evaluate the discriminative performance of this score to predict the need for hospital admission among children with acute asthma. Methods:A secondary analysis of prospective data was performed to compare triage nurse and study personnel PAS scores among children aged 6 to 18 years presenting to the emergency department with acute asthma. The IRR was determined by calculation of weighted Cohen κ with differences evaluated by Wilcoxon ranked pairs. Receiver operating characteristic curves were created to evaluate the predictive ability of PAS to determine the need for hospital admission.Results: One hundred one subjects were evaluated by both study personnel and a triage nurse with PAS score recorded. The IRR of the total PAS score was determined to be moderate (κ = 0.57) and acceptable, although lower than previously reported. Individual components of the PAS score demonstrated fair to substantial agreement. Receiver operating characteristic analysis demonstrated total PAS at emergency department triage to have poor test characteristics in predicting the need for hospital admission, whether PAS was determined by study personnel, triage nurse, or an average score (area under the curve, 0.62-0.65). Conclusions:In this study, total PAS score demonstrated a moderate and acceptable level of IRR with a poor discriminative ability to determine the need for hospital admission at the time of ED triage.

show abstract

Interobserver reliability of attending physicians and bedside nurses when using an inpatient paediatric respiratory score

Abstract: The use of an Integrated Care Pathway allows for optimal interprofessional collaboration between bedside nurses and attending physicians.

Cited by 7 publications

References 23 publications

Derivation and Validation of an Objective Effort of Breathing Score in Critically Ill Children

Derivation and Validation of an Objective Effort of Breathing Score in Critically Ill Children

Interobserver Agreement on Clinical Judgment of Work of Breathing in Spontaneously Breathing Children in the Pediatric Intensive Care Unit

Interrater Reliability of the Pediatric Asthma Score

Contact Info

Product

Resources

About