Prospective Internal Validation of Mathematical Models to Predict Malignancy in Adnexal Masses: Results from the International Ovarian Tumor Analysis Study

Abstract

Purpose: To prospectively test the mathematical models for calculation of the risk of malignancy in adnexal masses that were developed on the International OvarianTumorAnalysis (IOTA) phase 1data set on a new data set and to compare their performance with that of pattern recognition, our standard method. Methods: Three IOTA centers included 507 new patients who all underwent a transvaginal ultrasound using the standardized IOTA protocol. The outcome measure was the histologic classification of excised tissue. The diagnostic performance of 11 mathematical models that had been developed on the phase 1data set and of pattern recognition was expressed as area under the receiver operating characteristic curve (AUC) and as sensitivity and specificity when using the cutoffs recommended in the studies where the models had been created. For pattern recognition, an AUC was made based on level of diagnostic confidence. Results: All IOTA models performed very well and quite similarly, with sensitivity and specificity ranging between 92% and 96% and 74% and 84%, respectively, and AUCs between 0.945 and 0.950. A least squares support vector machine with linear kernel and a logistic regression model had the largest AUCs. For pattern recognition, the AUC was 0.963, sensitivity was 90.2%, and specificity was 92.9%. Conclusion: This internal validation of mathematical models to estimate the malignancy risk in adnexal tumors shows that the IOTA models had a diagnostic performance similar to that in the original data set. Pattern recognition used by an expert sonologist remains the best method, although the difference in performance between the best mathematical model is not large.

Interobserver Agreement in Describing the Ultrasound Appearance of Adnexal Masses and in Calculating the Risk of Malignancy Using Logistic Regression Models

“…Our results showed that two experienced sonologists agreed quite well in their classification of masses as benign or malignant using the 10% risk of malignancy cutoff of LR1 and LR2, and that the diagnostic performance of LR1 and LR2 with regard to discrimination between benign and malignant tumors was similar for the two sonologists and similar to that reported by others (14,(26)(27)(28). This is reassuring, because the main purpose of using model LR1 and LR2 is to classify tumors as benign or malignant.…”

External Validation of Diagnostic Models to Estimate the Risk of Malignancy in Adnexal Masses

“…Four different types of mathematical models were developed to evaluate which type of model did best (scoring system, logistic regression models, artificial neural networks (ANN), and vector machine models). On internal and temporal validation in the centers that previously developed the models, they had excellent diagnostic performance (16,(22)(23)(24)(25). After temporal validation, all models had similar performance (AUCs between 0.945 and 0.950).…”

Terms, definitions and measurements to describe the sonographic features of the endometrium and intrauterine lesions: a consensus opinion from the International Endometrial Tumor Analysis (IETA) group

“…have been used in large multicenter studies and the results have been validated successfully 26 ; hopefully, the morphological and blood flow descriptors used in the IOTA study will gradually be adopted as the standard language to describe ovarian pathology. We hope that a similar approach to the endometrium will also prove successful.…”