Interobserver agreement between eight observers using IOTA simple rules and O-RADS lexicon descriptors for adnexal masses

Abstract

Purpose To evaluate interobserver agreement in assigning imaging features and classifying adnexal masses using the IOTA simple rules versus O-RADS lexicon and identify causes of discrepancy. Methods Pelvic ultrasound (US) examinations in 114 women with 118 adnexal masses were evaluated by eight radiologists blinded to the final diagnosis (4 attendings and 4 fellows) using IOTA simple rules and O-RADS lexicon. Each feature category was analyzed for interobserver agreement using intraclass correlation coefficient (ICC) for ordinal variables and free marginal kappa for nominal variables. The two-tailed significance level (a) was set at 0.05. Results For IOTA simple rules, interobserver agreement was almost perfect for three malignant lesion categories (M2-4) and substantial for the remaining two (M1, M5) with k-values of 0.80–0.82 and 0.68–0.69, respectively. Interobserver agreement was almost perfect for two benign feature categories (B2, B3), substantial for two (B4, B5) and moderate for one (B1) with k-values of 0.81–0.90, 0.69–0.70 and 0.60, respectively. For O-RADS, interobserver agreement was almost perfect for two out of ten feature categories (ascites and peritoneal nodules) with k-values of 0.89 and 0.97. Interobserver agreement ranged from fair to substantial for the remaining eight feature categories with k-values of 0.39–0.61. Fellows and attendings had ICC values of 0.725 and 0.517, respectively. Conclusion O-RADS had variable interobserver agreement with overall good agreement. IOTA simple rules had more uniform interobserver agreement with overall excellent agreement. Greater reader experience did not improve interobserver agreement with O-RADS. Graphical abstract

Ultrasonographic diagnosis of adnexal masses: interobserver agreement in the interpretation of videos, using IOTA terminology

“…Neha Antil et al [ 17 ] recently evaluated agreement in classifying adnexal masses using Simple Rules among observers of different levels of experience (4 fellows and 4 attendings). They found that agreement calculated through interclass correlation coefficient (ICC) was excellent and, consistent with our results, it did not improve with the years of experience.…”

Effect of differences in O-RADS lexicon interpretation between senior and junior sonologists on O-RADS classification and diagnostic performance

“…The O-RADS ultrasound working group selected some terms from the O-RADS lexicon and proposed the O-RADS ultrasound risk stratification and management system in 2020 (Andreotti et al 2020 ; Cao et al 2021 ). This risk stratification and management system provides a detailed description of each category (Andreotti et al 2020 ), but in some studies, the classification system still did not achieve good interobserver agreement (Antil et al 2022 ; Guo et al 2022a , 2022b ). Differences in the interpretation of the O-RADS ultrasound lexicon can have an impact on the assessment results of O-RADS (Antil et al 2022 ).…”

“…Existing studies (Vara et al 2022 ; Xie et al 2022 ; Wu et al 2022 ; Pi et al 2021 ) have confirmed that the O-RADS ultrasound classification system has high sensitivity and diagnostic efficacy when assessed by both senior and junior sonologists, but the inter-observer agreement of this classification system fluctuates widely, ranging from fair to almost perfect. Differences in the interpretation of the O-RADS ultrasound lexicon can have an impact on the outcome of O-RADS assessment (Antil et al 2022 ). In order to assess the consistency of O-RADS ultrasound lexicon interpretation between senior and junior sonologists and to investigate its impact on O-RADS classification and diagnostic performance, we conducted a prospective study.…”

In Vivo Ultrasound Molecular Imaging in the Evaluation of Complex Ovarian Masses: A Practical Guide to Correlation with Ex Vivo Immunohistochemistry

“…During the past decade, ultrasound (US) has been the first-line imaging modality in gynecological oncology. [18][19][20] US has a lot of advantages as an ideal imaging tool: it is relatively inexpensive, highly portable, free of ionizing radiation, capable of high-resolution real-time imaging, and available worldwide. In particular, endovaginal transducers enable detailed evaluation of the uterus and ovaries in high resolution comparable to MRI.…”

“…Transvaginal ultrasound (TVUS) imaging has routinely been used in the diagnosis of OC. [18][19][20] As a first-line screening imaging tool, conventional TVUS suffers from lower sensitivity and specificity compared to other imaging modalities, e.g., MRI. [15] Conventional TVUS is usually unable to detect small solid ovarian tumors in normal volume ovaries, or to always differentiate benign ovarian tumors reliably from malignancy.…”