Observer Variability in Interpretation of Abdominal Radiographs of Infants with Suspected Necrotizing Enterocolitis

Abstract

We examined (1) the observer variability (both interobserver and intraobserver) in interpretation of abdominal radiographs of infants with suspected necrotizing enterocolitis (NEC), (2) the interobserver variability for individual radiologic signs used to diagnose NEC, and (3) the influence of experience in determining the extent of observer variability. Our hypotheses were (1) there would be considerable observer variability in interpretation of abdominal radiographs of infants with suspected NEC; (2) the extent of observer variability would differ for individual radiologic signs of NEC; and (3) the extent of observer variability would be determined by the observer's experience. The participants included 12 observers: two pediatric radiologists, four attending neonatologists, three neonatal fellows, and three pediatric residents. The participating observers under similar interpretation conditions, twice independently, interpreted the same 40 pairs of abdominal radiographs from infants with suspected NEC. The interval between the two interpretations was 3 to 6 months. Intraobserver and interobserver variability was assessed by applying the Kappa statistic to the radiologic signs of NEC for the two separate interpretations. The observers were blinded to patient's identity and the clinical course. Each observer recorded the absence, suspicion, or presence of (1) intestinal distention, (2) air fluid levels, (3) bowel wall thickening, (4) pneumatosis intestinalis, (5) portal venous gas, (6) pneumoperitoneum, and (7) NEC. We found low intraobserver and interobserver agreements. There was considerable variation in observer variability for individual radiologic signs. Trained observers performed better than intraining observers. We conclude that the radiologic signs in isolation should not be considered reliable. We recommend studies to formulate more objective criteria for many of the radiographic features of NEC. Standardization and periodic enforcement of these criteria among observers could reduce observer variability. We suggest that, to decrease both false-negative and false-positive interpretation, an experienced observer should always review the radiographs of infants with suspected NEC.

Necrotizing Enterocolitis: The Mystery Goes On

“…Pneumoperitoneum is a sign that the intestine has perforated, but this may be due to either spontaneous intestinal perforation or NEC. The diagnosis of the pneumatosis intestinalis is sometimes very difficult with various radiologists having different opinions on the same radiograph [52]. Occasionally, a neonate with a very distended abdomen and complex ascites lacks bowel gas and radiographs suggest considerable fluid in the abdominal cavity without free air.…”

A systematic review of serologic tests in the diagnosis of necrotizing enterocolitis

“…However, not only does pneumatosis intestinalis have low sensitivity [7] and the potential for decreased specificity [8], but also concordance between specialist pediatric radiologists in interpreting its presence is low [9][10][11]. Although other radiologic investigations have been proposed, such as Doppler ultrasound scanning for intestinal perfusion [12], experience with these is limited, and the technique is operator dependent.…”

Necrotising Enterocolitis - epidemiology and imaging

“…It was soon recognized, that subtle radiographic signs of early NEC may appear before clinical signs of the disease and progress ahead of clinical deterioration, but also that the typical radiographic signs, pneumatosis intestinalis (PI) and portal venous gas (PVG), were often transient, pneumoperitoneum (PP) frequently missing in spite of intestinal perforation, and the overall sensitivity of AR low, especially in extremely low birth weight infants. 22,26,31,34,[37][38][39][40][41][42][43] These insights, together with the reports of early detection of PVG with ultrasound by Merritt et.al. 44 and the studies of mesenteric circulation in neonates by van Bel et.al, 45 led Fotter and Sorantin to suggested a standardized algorithm for early diagnosis and evaluation of progress, in which AR in two projections and ultrasonography was recommended for diagnosis in all cases, followed by repeated examinations at 4-6 hours interval, or at least daily, with supine and left lateral decubitus films each time, and repeated ultrasounds at every 12 to 24 hours.…”

“…44 and the studies of mesenteric circulation in neonates by van Bel et.al, 45 led Fotter and Sorantin to suggested a standardized algorithm for early diagnosis and evaluation of progress, in which AR in two projections and ultrasonography was recommended for diagnosis in all cases, followed by repeated examinations at 4-6 hours interval, or at least daily, with supine and left lateral decubitus films each time, and repeated ultrasounds at every 12 to 24 hours. 46 Apart from low sensitivity, AR also suffers from of considerable intra-and interobserver variability 43,47,48 and presumably some variability in the quality of the radiographs, depending on the skill of the radiographer. To improve diagnostic accuracy, radiographic patterns have been analysed at greater detail, defining early signs, such as ileus and elongation of bowel loops, loss of bowel wall definition, and a disorganised, atonic pattern of intestinal gas.…”

“…The selection of articles may, however, not be complete. 40 1976 X Daneman 22 1978 X X X X X X Kogutt 41 1979 X Mata 48 1980 X X X Leonard 73 1982 X *Cikrit 74 1985 X Frey 75 1987 X X *Kennedy 76 1987 X Kosloske 56 1988 X X X Brill 77 1990 X Bömelburg 78 1992 X X X Rehan 43 1999 X X X X X X Molik 57 2001 X Tam 31 2002 X X X Faingold 70 2005 X Kim 60 2005 X Sharma 23 2005 X X X Coursey 50 2009 X X X X X X X Dördelman 24 2009 X X Bonhorst 34 2011 X Dilli 61 2011 X X X X Muchantef 53 2013 X X X X X X Silva 72 2013 X X X X Muller 79 2014 X * Abstracts only accessed US has been suggested to be used for: detection of PI, PVG, and intra-abdominal fluid and evaluation of bowel peristalsis, thickness, and perfusion of the bowel wall in neonates with mild symptoms and nonspecific plain film findings;…”