Interobserver Agreement for Single Operator Choledochoscopy Imaging: Can We Do Better?

Abstract

Background. The SpyGlass Direct Visualization System (Boston Scientific, Natick, MA) is routinely used during single operator choledochoscopy (SOC) to identify biliary lesions or strictures with a diagnostic accuracy up to 88%. The objective of this study was to determine the interobserver agreement (IOA) of modified scoring criteria for diagnosing biliary lesions/strictures. Methods. 27 SPY SOC video clips were reviewed and scored by 9 interventional endoscopists based on published criteria that included the presence and severity of surface structure, vasculature visualization, lesions, and findings. Results. Overall IOA was “slight” for all variables. The K statistics are as follows: surface (K = 0.12, SE = 0.02); vessels (K = 0.14, SE = 0.02); lesions (K = 0.11, SE = 0.02); findings (K = 0.08, SE = 0.03); and final diagnosis (K = 0.08, SE = 0.02). The IOA for “findings” and “final diagnosis” was also only “slight.” The final diagnosis was malignant (11), benign (11), and indeterminate (5). Conclusion. IOA using the modified criteria of SOC images was slight to almost poor. The average accuracy was less than 50%. These findings reaffirm that imaging criteria for benign and malignant biliary pathology need to be formally established and validated.

The interobserver agreement of optical features used to differentiate benign from neoplastic biliary lesions assessed at balloon‐assisted cholangioscopy

“…We believe that the characteristics identified with at least moderate interobserver agreements, irregular margins, dark mucosa, adherent mucous, tubular surface structures, branched or disorganized surface structures, and papillary projections are very important. Moderate and substantial interobserver agreements between individual endoscopists for optical features differentiating benign from malignant biliary lesions have generally not been identified in earlier studies . We firmly believe this in part relates to the superior image quality of BAC compared with other single operator cholangioscopy systems.…”

“…Moderate and substantial interobserver agreements between individual endoscopists for optical features differentiating benign from malignant biliary lesions have generally not been identified in earlier studies. 25,26 We firmly believe this in part relates to the superior image quality of BAC compared with other single operator cholangioscopy systems. Another strength of the present study is that a high level of agreement was achieved between a large group of experienced endoscopists who were not BAC experts.…”

“…Sethi and colleagues have, for some time, recognized the importance of assessing the interobserver agreement between expert endoscopists to help validate specific optical criteria that may differentiate benign from neoplastic biliary lesions . In their earlier work using the first generation fiber‐optic SpyGlass system, the interobserver agreements for a range of criteria were poor (κ ≤ 0.2) . In a later study by the same group, four visual features established by a single expert endoscopist using first generation SpyGlass were used .…”

Endoscopic assessment and management of biliary strictures

“…However, this high diagnostic yield for EUS requires verification in other studies. Interobserver agreement for standard single-operator cholangioscopy has been shown to be modest at best, highlighting the need for validated scoring criteria 24. The more recent use of ultra-slim endoscopes for cholangioscopy shows promise,25 facilitating superior white-light image quality, larger biopsy specimens and multimodal chromendoscopy, autofluorescence and narrow band imaging.…”

Diagnosing Biliary Malignancy

“…In 2 consecutive studies, interobserver agreement of SOC visual findings and final diagnosis were only slight to fair and the accuracy was less than 50% on diagnosing malignancy. 52,53 With the forthcoming availability of digital SOC and its suspected increased utility, the development of a reliable, validated set of visual criteria will be even more valuable.…”