Inter-observer agreement in interpreting chest X-rays on children with acute lower respiratory tract infections and concurrent wheezing

Abstract

CONTEXT AND OBJECTIVE: Many children with acute lower respiratory tract infections (ALRI) present to the emergency ward with concurrent wheezing. A chest x-ray is often requested to rule out pneumonia. We assessed inter-observer agreement in interpreting x-rays on such children. DESIGNS AND SETTING: Prospective consecutive case study at Instituto de Salud del Niño, Lima, Peru. METHODS: Chest x-rays were obtained from eligible children younger than two years old with ALRI and concurrent wheezing who were seen in the emergency ward of a nationwide pediatric referral hospital. The x-rays were read independently by three different pediatric residents who were aware only that the children had a respiratory infection. All the children had received inhaled beta-adrenergic agonists before undergoing chest x-rays. Lobar and complicated pneumonia cases were excluded from the study. RESULTS: Two hundred x-rays were read. The overall kappa index was 0.2. The highest individual kappa values for specific x-ray findings ranged from 0.26 to 0.34 for rib horizontalization and from 0.14 to 0.31 for alveolar infiltrate. Inter-observer variation was intermediate for alveolar infiltrate (kappa 0.14 to 0.21) and for air bronchogram (kappa 0.13 to 0.23). Reinforcement of the bronchovascular network (kappa 0.10 to 0.16) and air trapping (kappa 0.05 to 0.20) had the lowest agreement. CONCLUSIONS: There was poor inter-observer agreement for chest x-ray interpretation on children with ALRI and concurrent wheezing seen at the emergency ward. This may preclude reliable diagnosing of pneumonia in settings where residents make management decisions regarding sick children. The effects of training on inter-observer variation need further studies.

Chest radiography of acute paediatric lower respiratory infections: experience versus interobserver variation

“…Swingler also suggested that radiological features be defined more carefully, which is what we attempted to do using a standardized questionnaire about 10 different radiological features. It appears that in an academic programme such as ours, the level of agreement found was consistent with that previously reported by others, with Cohen's kappas intermediate between two major studies . Yet, there is no universal consistency in diagnosis, and we feel that there is ample room for increasing proficiency in CR interpretation.…”

Variability and accuracy in interpretation of consolidation on chest radiography for diagnosing pneumonia in children under 5 years of age

“…Radiological findings, primarily the presence of consolidation, are frequently used as the reference standard for defining pneumonia, but variability in interpretation is well recognized . Previous studies of chest radiographs in children have reported widely varying inter‐observer variability for identification of consolidation, with kappa values 0.2–0.79, but these studies have been small, leading to imprecision of estimates, and often used highly selected patients, which may introduce spectrum bias. Factors affecting interpretation and, therefore, reader agreement of chest radiographs are not well defined.…”

Evaluation of the World Health Organization criteria for chest radiographs for pneumonia diagnosis in children

“…In young children, some of these clinical signs and symptoms can be absent [21]. Radiography still remains the best available tool to diagnose pneumonia, although observer variation is high [3,4,6,13,20].…”

“…However, several previous studies required two radiologists as observers for the gold standard for pneumonia diagnosis [1,9,27,28]. Other studies used several different combinations of readers: pediatric residents, experienced pediatricians, or pediatric pulmonologists [4,8,19,28].…”