Multimodal Entity Coreference for Cervical Dysplasia Diagnosis

Abstract

Cervical cancer is the second most common type of cancer for women. Existing screening programs for cervical cancer, such as Pap Smear, suffer from low sensitivity. Thus, many patients who are ill are not detected in the screening process. Using images of the cervix as an aid in cervical cancer screening has the potential to greatly improve sensitivity, and can be especially useful in resource-poor regions of the world. In this paper, we develop a data-driven computer algorithm for interpreting cervical images based on color and texture. We are able to obtain 74% sensitivity and 90% specificity when differentiating high-grade cervical lesions from low-grade lesions and normal tissue. On the same dataset, using Pap tests alone yields a sensitivity of 37% and specificity of 96%, and using HPV test alone gives a 57% sensitivity and 93% specificity. Furthermore, we develop a comprehensive algorithmic framework based on Multimodal Entity Coreference for combining various tests to perform disease classification and diagnosis. When integrating multiple tests, we adopt information gain and gradient-based approaches for learning the relative weights of different tests. In our evaluation, we present a novel algorithm that integrates cervical images, Pap, HPV, and patient age, which yields 83.21% sensitivity and 94.79% specificity, a statistically significant improvement over using any single source of information alone.

Multi-feature based benchmark for cervical dysplasia classification evaluation

“…In contrast to Pap tests [4, 5], Cervigrams are images captured by the non-invasive and low cost digital cervicography. To further improve the classification performance, Song et al [7] combined the Cervigram information with other clinical test results such as Pap and HPV; however, these other clinical tests require additional resources that may not be available in resource-poor areas of the world.…”

“…Several types of features [2, 7, 8, 9, 10] have been proposed to encode Cervigram information. Li et al [8] identified acetowhite regions by analyzing local color features.…”

“…In the work by Ji et al [10], texture features were used to recognize important vascular patterns in Cervigrams. In [2, 7], the authors combined the pyramid histogram of oriented gradients (PHOG) and the pyramid color histogram in L*A*B space (PLAB) features to perform region of interest (ROI) segmentation and CIN classification.…”

“…Neural networks, support vector machines (SVM), k-Nearest Neighbors (KNN), linear discriminant analysis (LDA), and decision trees are commonly used for studying cervical cancer [11]. Kim et al [2] applied a linear SVM to classify Cervigrams into CIN1/normal or CIN2/3+, while Song et al [7] utilized KNN coupled with a majority voting algorithm to perform the CIN classification. Zhang et al [12] proposed a discriminative sparse representation for tissue classification in Cervigrams.…”

Multi-test cervical cancer diagnosis with missing data estimation

“…Some of the most important visual observations in cervigrams include the acetowhite region, and features within that region, such as mosaicism, punctation, and atypical vessels; it is also important to distinguish these possibly disease-related features from benign features such as polyps or cysts. Figure 3 shows some example images of those observations [14]. The identification of these different characteristics within a cervigram could help with diagnosis.…”