P176 Expert consensus on acquisition and reporting of intestinal ultrasonography activity in Crohn’s disease. A prospective inter-rater agreement study

Abstract

Background Intestinal ultrasonography (IUS) is a promising cross-sectional imaging modality used to assess transmural disease and complications in Crohn’s disease (CD). Although recently positioned as a first-line modality for evaluation as per ECCO guidelines, standard measurements, reproducibility and nomenclature have not yet been clearly established. The aim of this study was to evaluate the inter-rater agreement for parameters identified as important by experts through Delphi consensus. Methods IUS parameters demonstrating inflammatory activity were systematically reviewed in the literature and presented to IUS experts. Individual parameters were selected by a blinded Delphi consensus panel to establish relative contribution to inflammatory activity in CD. Weighted grading of each parameter was further established by expert consensus. Image acquisition for optimal measurement was established by consensus. Two phases for evaluating inter-rater variability were undertaken. Phase 1: blind review by 8 readers of 20 de-identified CD cases. Cases with poor agreement were reviewed to clarify discrepancy and improve agreement. Phase 2: an additional 30 de-identified CD cases blindly were reviewed by 12 independent expert readers. Inter-rater agreement was evaluated for all 4 key parameters. Statistics were performed using Stata 16. Bowel wall thickness (BWT) was assessed using intraclass correlation coefficient (ICC) and the ordinal parameters using weighted Cohens Kappa. Results The Delphi process reduced 12 activity parameters to 4 key contributors including BWT, color Doppler signal (CDI), inflammatory fat and bowel wall echostratification (Figure 1). BWT was regarded as pathologic if the average of 4 measurements were > 3 mm for the small and large bowel, and grades of the additional parameters established (Table 1). Bowel wall thickness was comprised of 2 measurements in cross section and 2 in longitudinal orientation (Figure 2). Interobserver agreement was almost perfect for BWT: ICC=0.91 (95% CI 0.83 to 0.96) p = 0.001, while there was moderate agreement for CDI κ=0.60 (95% CI 0.48–0.72) p = 0.001. Agreement for inflammatory fat detection was also moderate with κ= 0.50 (95% CI 0.33–0.66) p = 0.001, while stratification was fair κ= 0.39 (95% CI 0.26–0.53) p = 0.001. Conclusion This expert consensus-based IUS activity score clearly establishes the reproducibility of this standardised approach to measure inflammatory activity in patients with CD. Using our method, BWT which is known as the most important parameter, is highly reproducible with CDI and inflammatory fat demonstrating moderate reproducibility. This score may provide the foundation for the future incorporation of IUS in research studies and clinical trials.

Standardisation of intestinal ultrasound scoring in clinical trials for luminal Crohn’s disease

“…This adaptation was the only method considered appropriate for assessment of bowel wall vascularity. Concerns amongst panellists remained regarding the inter‐rater reliability of a semi‐quantitative scoring method, which was recently reported as moderate (Cohen’s kappa = 0.6) 25,33 . The value of collapsing grades 1 and 2, and the novel method proposed in this study to improve reliability of assessment of bowel wall vascularity will be determined in an ongoing study.…”

“…Heterogeneity and lack of clarity for defining bowel wall stratification may have contributed to the observed lack of reliability in previous studies and consequent exclusion of this items from some CD activity indices. 25,[33][34][35] Potential applications of IUS in clinical trials were also explored in this study. Beyond the value for patient screening, broad applica-…”

“…In the panel discussion, neither mesenteric hyperechogenicity or fat wrapping alone were considered adequate, and the general term mesenteric inflammatory fat should be used to describe the mesenteric inflammatory changes observable with IUS. Whilst the panel felt that a binary grading system for this item was appropriate, two new grading scales proposed by Novak et al and Bhatnagar et al (based on the METRIC study) were rated as uncertain but may demonstrate greater discriminatory power 25,33‐35 …”

“…Second, panel voting and discussion was informed by an up-to-date systematic review as well as the most recent evidence from studies published between rounds of voting. 11,25,33,34,37 Limitations of this study include uncertainty on the appropriateness of some IUS component item definitions, response definitions and definite structure of an ideal activity score for luminal CD. However, this is likely reflective of current gaps in the literature, heterogeneity in IUS definitions and lack of standardisation of items incorporated in CD activity assessment.…”

“…Although which was recently reported as moderate (Cohen's kappa = 0.6). 25,33 The value of collapsing grades 1 and 2, and the novel method proposed in this study to improve reliability of assessment of bowel wall vascularity will be determined in an ongoing study.…”