Content-Based Retrieval of Focal Liver Lesions Using Bag-of-Visual-Words Representations of Single- and Multiphase Contrast-Enhanced CT Images

Abstract

This paper is aimed at developing and evaluating a content-based retrieval method for contrastenhanced liver computed tomographic (CT) images using bag-of-visual-words (BoW) representations of single and multiple phases. The BoW histograms are extracted using the raw intensity as local patch descriptor for each enhance phase by densely sampling the image patches within the liver lesion regions. The distance metric learning algorithms are employed to obtain the semantic similarity on the Hellinger kernel feature map of the BoW histograms. The different visual vocabularies for BoW and learned distance metrics are evaluated in a contrast-enhanced CT image dataset comprised of 189 patients with three types of focal liver lesions, including 87 hepatomas, 62 cysts, and 60 hemangiomas. For each single enhance phase, the mean of average precision (mAP) of BoW representations for retrieval can reach above 90 % which is significantly higher than that of intensity histogram and Gabor filters. Furthermore, the combined BoW representations of the three enhance phases can improve mAP to 94.5 %. These preliminary results demonstrate that the BoW representation is effective and feasible for retrieval of liver lesions in contrast-enhanced CT images.

Content based medical image retrieval using dictionary learning

“…This feature extraction method is more suitable for medical image databases because of the rich information of medical images available at the center of images. Unlike natural images, most of the medical images are taken under standardized conditions [5], which makes them possess somewhat rich information around the center. This is, however, not a prerequisite for our method.…”

“…In the Bag-Of-Words (BOW) [5] framework, the image patches were sampled densely or sparsely by "interest points" detectors and were depicted by local patch descriptors like SIFT. These descriptors were used to classify liver lesions in CT images.…”

“…In a typical CBIR system in medical domain, subtle differences between images can not be considered irrelevant. Consequently, a Content Based Medical Image Retrieval (CBMIR) system having a kind of invariance (with respect to any transformation) is of value [5], [6].…”

Visual Character N-Grams for Classification and Retrieval of Radiological Images

“…This approach is also been shown useful in the classification of focal liver lesions on CT images in [44]. In [45] raw intensities without normalization are used as local patch descriptors. The raw patches are then sampled densely with the stride of one pixel in the liver lesion region to form the BoW representation.…”

Text- and Content-Based Medical Image Retrieval in the VISCERAL Retrieval Benchmark

“…In the literature, there are many methods that can automatically extract the visual features to characterize the medical images [2,[26][27][28]. The Bag of Visual Words (BoVW) [29,30] method, which is one of the popular methods for visual content-based image retrieval, is applied as our first content-based retrieval method.…”