The bronchiolitis severity score: An assessment of face validity, construct validity, and interobserver reliability

Abstract

Objective: To assess face validity, interobserver reliability, and the ability to discriminate escalations of care within 24-h of admission (late rescues) for the bronchiolitis severity score (BSS) for children hospitalized for acute bronchiolitis.Hypotheses: The BSS will yield variable face validity, have clinically relevant interobserver reliability (kappa > 0.7), and distinguish late rescues during hospitalization.Methods: We performed a combined retrospective and prospective, mixed methods study where (1) interobserver agreement was prospectively assessed by overall and subcategory congruence (kappa) calculations, (2) face value were qualitatively assessed from aggregate questionnaire responses, and (3) construct validity for late rescues were assessed using receiver operator characteristic (ROC) curve analyses.Results: Face validity, assessed from 39 questionnaire respondents, were generally positive for BSS utility, reliability, and usability. The BSS exhibited weak interobserver reliability (kappa = 0.22, 95% confidence interval [CI]: 0.11-0.31) calculated from 72 sequential, blinded calculations. Retrospectively, 181 children less than 2 years of age admitted to the general pediatric ward for acute bronchiolitis from November 2017 to April 2019 were identified of which 18 (9.9%) experienced late rescues. Admission BSS values were no different for children with and without late rescues (6[3,6] vs. 4[3,6]; p = .09). An ROC curve analysis revealed an area under the curve of 0.61 (95% CI: 0.48-0.75; threshold ≥6 with sensitivity = 56%, specificity = 69%) for BSS to discriminate late rescues. Conclusion:Although clinicians expressed favorable perceptions of BSS face and content validity, we noted weak interobserver reliability and limited construct validity. Further development and validation are needed to strengthen the BSS before routine use.

Characterization of Cardiopulmonary Interactions and Exploring Their Prognostic Value in Acute Bronchiolitis: A Prospective Cardiopulmonary Ultrasound Study

“…Clinical scores have proven to be inaccurate in predicting a prognosis in acute bronchiolitis, with weak interobserver reliability and limited construct validity [ 5 ]. Thus, lung ultrasound (LUS) is now emerging as a valuable tool in the assessment of the respiratory status in bronchiolitis.…”

Pre‐school wheeze hospital presentations: A demographic profile and review of clinical management

“…A score of 1–2 is mild, a score of 3–5 is moderate and a score of 6 is severe pre‐school wheeze. The severity score construction is comparable to that used for bronchiolitis (respiratory rate, retractions, dyspnoea and auscultation), 12 and for acute childhood asthma (respiratory rate, oxygen saturation, auscultation, retractions and dyspnoea) 13,14 …”

Enhancing universities students' performance through level advisers' leadership qualities

“…Cohen's Kappa index was used to estimate for the expert rating. With the inter-rater between 0.6 and 0.7 upward is acceptable (Siraj et al. , 2021) Yusoff (2019).…”

“…Cohen's Kappa index was used to estimate for the expert rating. With the inter-rater between 0.6 and 0.7 upward is acceptable (Siraj et al, 2021) Yusoff (2019. Therefore, the weak items with index less than 0.6 were reworded and restructured.…”