Ultrasound Experience Substantially Impacts on Diagnostic Performance and Confidence when Adnexal Masses Are Classified Using Pattern Recognition

Abstract

Aim: To determine how accurately and confidently examiners with different levels of ultrasound experience can classify adnexal masses as benign or malignant and suggest a specific histological diagnosis when evaluating ultrasound images using pattern recognition. Methods: Ultrasound images of selected adnexal masses were evaluated by 3 expert sonologists, 2 senior and 4 junior trainees. They were instructed to classify the masses using pattern recognition as benign or malignant, to state the level of confidence with which this classification was made and to suggest a specific histological diagnosis. Sensitivity, specificity, accuracy and positive and negative likelihood ratios (LR+ and LR–) with regard to malignancy were calculated. The area under the receiver operating characteristic curve (AUC) of pattern recognition was calculated by using six levels of diagnostic confidence. Results: 166 masses were examined, of which 42% were malignant. Sensitivity with regard to malignancy ranged from 80 to 86% for the experts, was 70 and 84% for the 2 senior trainees and ranged from 70 to 86% for the junior trainees. The specificity of the experts ranged from 79 to 91%, was 77 and 89% for the senior trainees and ranged from 59 to 83% for the junior trainees. The experts were uncertain about their diagnosis in 4–13% of the cases, the senior trainees in 15–20% and the junior trainees in 67–100% of the cases. The AUCs ranged from 0.861 to 0.922 for the experts, were 0.842 and 0.855 for the senior trainees, and ranged from 0.726 to 0.795 for the junior trainees. The experts suggested a correct specific histological diagnosis in 69–77% of the cases. All 6 trainees did so significantly less often (22–42% of the cases). Conclusion: Expert sonologists can accurately classify adnexal masses as benign or malignant and can successfully predict the specific histological diagnosis in many cases. Whilst less experienced operators perform reasonably well when predicting the benign or malignant nature of the mass, they do so with a very low level of diagnostic confidence and are unable to state the likely histology of a mass in most cases.

Subjective assessment of ovarian masses using pattern recognition: the impact of experience on diagnostic performance and interobserver variability

“…The best diagnostic performance, in combination with the lowest interobserver variability, was achieved by the group of level III sonologists. These results are in agreement with those of other published studies assessing interobserver variability relative to different levels of experience [17,18,20,21]. The largest differences in diagnostic performance were observed in the present series in the classification of malignant tumors.…”

“…The mean sensitivity obtained by the experts was 80%, with a specificity of 87%, while the junior trainees achieved a mean sensitivity and specificity of 55 and 80%, respectively. Similar results were reported by van Holsbeke et al [17]. In their study, ultrasound images of 166 selected adnexal masses were evaluated by three expert sonologists, two senior trainees, and four junior trainees.…”

“…The expert sonologists achieved a good diagnostic performance in classifying adnexal masses as benign or malignant and predicting a specific histological diagnosis. The less experienced operators performed reasonably well when predicting the benign or malignant nature of the mass, but they had a significantly poorer diagnostic performance in making a specific diagnosis [17].…”

“…Although many major patient management decisions are based on sonographic interpretations, little detailed information is available regarding the interobserver and intraobserver variability of such assessments and little is known about the impact of experience on the subjective assessment of adnexal masses [17,18]. A study by Yazbek et al [19] showed that gynecologists with a special interest in sonography were able to influence decision-making in such a way that fewer staging laparotomies were required and hospitalization periods were shorter in comparison with less experienced examiners.…”

“…To date, only a few studies including small numbers of different observers have assessed the results of subjective evaluation by less experienced examiners [17,18,20,21]. A recent study compared diagnostic accuracy in a group of three expert sonologists and two senior and four junior trainees.…”