Predictive accuracy and reproducibility of the O-RADS US scoring system among sonologists with different training levels

Wu, Manli; Zhang, Man; Cao, Junyan; Wu, Shuangyu; Chen, Ying; Luo, Liping; Lin, Xin; Su, Manting; Zhang, Xinling

doi:10.1007/s00404-022-06752-5

“…When classifying classic benign lesions, junior sonologist preferred to follow the lesion categories in the O-RADS lexicon; therefore, some classic benign lesions were classified by junior sonologist into the subcategories of atypical benign lesions in O-RADS 2 or 3, which in turn affected the overall inter-observer agreement of the O-RADS classification ( к = 0.790). Numerous studies have confirmed that the best cutoff value for the O-RADS classification system is > O-RADS 3 (Wu et al 2022 ; Basha et al 2021 ; Cao et al 2021 ). Therefore, even when classified according to the lesion categories in the O-RADS ultrasound lexicon, most classic benign lesions could still be accurately categorized as benign.…”

Section: Discussionmentioning

confidence: 94%

“…The area under the receiver operating characteristic (ROC) curve (AUC) for the classification of benign or malignant tumors was calculated to compare the difference in diagnostic performance between senior and junior sonologists when using O-RADS. When O-RADS > 3 was defined as malignant (Wu et al 2022 ; Cao et al 2021 ), the O-RADS results were dichotomized accordingly and the sensitivity, specificity, accuracy and positive predictive value (PPV) of O-RADS classification were calculated for both observers by comparison with pathological findings. P value < 0.05 was considered statistically significant.…”

Section: Methodsmentioning

confidence: 99%

“…Existing studies (Vara et al 2022 ; Xie et al 2022 ; Wu et al 2022 ; Pi et al 2021 ) have confirmed that the O-RADS ultrasound classification system has high sensitivity and diagnostic efficacy when assessed by both senior and junior sonologists, but the inter-observer agreement of this classification system fluctuates widely, ranging from fair to almost perfect. Differences in the interpretation of the O-RADS ultrasound lexicon can have an impact on the outcome of O-RADS assessment (Antil et al 2022 ).…”

Section: Introductionmentioning

confidence: 96%

See 1 more Smart Citation

Effect of differences in O-RADS lexicon interpretation between senior and junior sonologists on O-RADS classification and diagnostic performance

Yang

¹

,

Wang

²

,

Liu

³

et al. 2023

J Cancer Res Clin Oncol

View full text Add to dashboard Cite

Purpose To assess the consistency of Ovarian-Adnexal Reporting and Data System (O-RADS) lexicon interpretation between senior and junior sonologists and to investigate its impact on O-RADS classification and diagnostic performance. Methods We prospectively studied 620 patients with adnexal lesions, all of whom underwent transvaginal or transrectal ultrasound performed by a senior sonologist (R1) who selected the O-RADS lexicon description and O-RADS category for the lesion after the examination. Meanwhile, the junior sonologist (R2) analyzed the images retained by R1 and divided the lesion in the same way. Pathological findings were used as a reference standard. kappa (к) statistics were used to assess the interobserver agreement. Results Of the 620 adnexal lesions, 532 were benign and 88 were malignant. When using the O-RADS lexicon, R1 and R2 had almost perfect agreement regarding lesion category, external contour of solid lesions, presence of papillary inside cystic lesions, and fluid echogenicity (к: 0.81–1.00). Substantial agreement in solid components, acoustic shadow, vascularity and O-RADS categories (к: 0.61–0.80). Consistency in classifying classic benign lesions in the O-RADS category was only moderate (к = 0.535). No significant difference in diagnostic performance between them using O-RADS (P = 0.1211). Conclusion There was good agreement between senior and junior sonologists in the interpretation of the O-RADS lexicon and in the classification of O-RADS, except for a moderate agreement in the interpretation and classification of classic benign lesions. Differences in O-RADS category delineation between sonologists had no significant effect on the diagnostic performance of O-RADS.

show abstract

“…The accuracy of gynecologic ultrasonography is largely dependent on the sonologist's subjective assessment. It has been observed that the correct classi cation of the adnexal lesions in expert ultrasound examination is higher than in less experienced doctors (Wu et al, 2023). Currently, there are no effective management strategies for O-RADS US 4 lesions, with the risk of malignancy exhibiting signi cant variations and some lesions found to be benign.…”

Section: Introductionmentioning

confidence: 99%

Developing a deep learning model for predicting ovarian cancer in Ovarian-Adnexal Reporting and Data System Ultrasound (O-RADS US) Category 4 lesions: A multicenter study

Xie,

Lin,

Li

et al. 2024

Preprint

0

View full text Add to dashboard Cite

Purpose To develop a deep learning (DL) model for differentiating between benign and malignant ovarian tumors of Ovarian-Adnexal Reporting and Data System Ultrasound (O-RADS US) Category 4 lesions, and validate its diagnostic performance. Methods A retrospective analysis of 1619 US images obtained from three centers from December 2014 to March 2023. DeepLabV3 and YOLOv8 were jointly used to segment, classify, and detect ovarian tumors. Precision and recall and area under the receiver operating characteristic curve (AUC) were employed to assess the model performance. Results A total of 519 patients (including 269 benign and 250 malignant masses) were enrolled in the study. The number of women included in the training, validation, and test cohorts was 426, 46, and 47, respectively. The detection models exhibited an average precision of 98.68% (95% CI: 0.95–0.99) for benign masses and 96.23% (95% CI: 0.92–0.98) for malignant masses. Moreover, in the training set, the AUC was 0.96 (95% CI: 0.94–0.97), whereas in the validation set, the AUC was 0.93(95% CI: 0.89–0.94) and 0.95 (95% CI: 0.91–0.96) in the test set. The sensitivity, specificity, accuracy, positive predictive value, and negative predictive values for the training set were 0.943,0.957,0.951,0.966, and 0.936, respectively, whereas those for the validation set were 0.905,0.935, 0.935,0.919, and 0.931, respectively. In addition, the sensitivity, specificity, accuracy, positive predictive value, and negative predictive value for the test set were 0.925, 0.955, 0.941, 0.956, and 0.927, respectively. Conclusion The constructed DL model exhibited high diagnostic performance in distinguishing benign and malignant ovarian tumors in O-RADS US category 4 lesions.

show abstract

“…The accuracy of gynecologic ultrasonography is largely dependent on the sonologist’s subjective assessment. It has been observed that the correct classification of the adnexal lesions in expert ultrasound examination is higher than in less experienced doctors (Wu et al 2023 ). Currently, there are no effective management strategies for O-RADS US 4 lesions, with the risk of malignancy exhibiting significant variations and some lesions found to be benign.…”

Section: Introductionmentioning

confidence: 99%

Developing a deep learning model for predicting ovarian cancer in Ovarian-Adnexal Reporting and Data System Ultrasound (O-RADS US) Category 4 lesions: A multicenter study

Xie,

Lin,

Li

et al. 2024

J Cancer Res Clin Oncol

0

View full text Add to dashboard Cite

Purpose To develop a deep learning (DL) model for differentiating between benign and malignant ovarian tumors of Ovarian-Adnexal Reporting and Data System Ultrasound (O-RADS US) Category 4 lesions, and validate its diagnostic performance. Methods A retrospective analysis of 1619 US images obtained from three centers from December 2014 to March 2023. DeepLabV3 and YOLOv8 were jointly used to segment, classify, and detect ovarian tumors. Precision and recall and area under the receiver operating characteristic curve (AUC) were employed to assess the model performance. Results A total of 519 patients (including 269 benign and 250 malignant masses) were enrolled in the study. The number of women included in the training, validation, and test cohorts was 426, 46, and 47, respectively. The detection models exhibited an average precision of 98.68% (95% CI: 0.95–0.99) for benign masses and 96.23% (95% CI: 0.92–0.98) for malignant masses. Moreover, in the training set, the AUC was 0.96 (95% CI: 0.94–0.97), whereas in the validation set, the AUC was 0.93(95% CI: 0.89–0.94) and 0.95 (95% CI: 0.91–0.96) in the test set. The sensitivity, specificity, accuracy, positive predictive value, and negative predictive values for the training set were 0.943,0.957,0.951,0.966, and 0.936, respectively, whereas those for the validation set were 0.905,0.935, 0.935,0.919, and 0.931, respectively. In addition, the sensitivity, specificity, accuracy, positive predictive value, and negative predictive value for the test set were 0.925, 0.955, 0.941, 0.956, and 0.927, respectively. Conclusion The constructed DL model exhibited high diagnostic performance in distinguishing benign and malignant ovarian tumors in O-RADS US category 4 lesions.

show abstract

Predictive accuracy and reproducibility of the O-RADS US scoring system among sonologists with different training levels

Cited by 7 publications

References 24 publications

Effect of differences in O-RADS lexicon interpretation between senior and junior sonologists on O-RADS classification and diagnostic performance

Effect of differences in O-RADS lexicon interpretation between senior and junior sonologists on O-RADS classification and diagnostic performance

Developing a deep learning model for predicting ovarian cancer in Ovarian-Adnexal Reporting and Data System Ultrasound (O-RADS US) Category 4 lesions: A multicenter study

Developing a deep learning model for predicting ovarian cancer in Ovarian-Adnexal Reporting and Data System Ultrasound (O-RADS US) Category 4 lesions: A multicenter study

Contact Info

Product

Resources

About