Content-Based Image Retrieval in Radiology: Current Status and Future Directions

Abstract

Diagnostic radiology requires accurate interpretation of complex signals in medical images. Content-based image retrieval (CBIR) techniques could be valuable to radiologists in assessing medical images by identifying similar images in large archives that could assist with decision support. Many advances have occurred in CBIR, and a variety of systems have appeared in nonmedical domains; however, permeation of these methods into radiology has been limited. Our goal in this review is to survey CBIR methods and systems from the perspective of application to radiology and to identify approaches developed in nonmedical applications that could be translated to radiology. Radiology images pose specific challenges compared with images in the consumer domain; they contain varied, rich, and often subtle features that need to be recognized in assessing image similarity. Radiology images also provide rich opportunities for CBIR: rich metadata about image semantics are provided by radiologists, and this information is not yet being used to its fullest advantage in CBIR systems. By integrating pixel-based and metadata-based image feature analysis, substantial advances of CBIR in medicine could ensue, with CBIR systems becoming an important tool in radiology practice.

A Multimodal Search Engine for Medical Imaging Studies

“…Multimodal information retrieval has had its impact in a multitude of fields, and several tools and techniques for CBMIR have emerged over the last two decades [4], [9]. In [10], the authors cover the state of the art on multimodal medical information retrieval in three perspectives, one of which is the latest research done in CBMIR.…”

“…Further advancements have granted the ability to search using pictures, audio, and other kinds of multimedia content as part of the query. Indeed, the continued efforts in image processing, medical informatics, and information retrieval are creating suitable conditions for the integration of multimodal information retrieval in the workflows of clinicians, lecturers, and researchers [4,5]. Multimodality in the generic context of information retrieval refers to the theories, algorithms, systems, and challenges of indexing and retrieving multiple modes (kinds) of data, which may include meta-data, free text, images, or other multimedia sources [6].…”

“…These systems, however, should not rely on visual similarity alone: the combination of medical image feature sets with non-visual data, such as DICOM meta-data and structured reports, is proven to be highly beneficial for medical decision support systems [4], as the fusion of multiple modalities can provide complementary information and increase the accuracy of the overall decision-making process [7]. Visual feature extraction and comparison techniques are already contemplated by many content-based image retrieval (CBIR) systems, but these capabilities are only the tip of the iceberg.…”

Content-Based Retrieval of Focal Liver Lesions Using Bag-of-Visual-Words Representations of Single- and Multiphase Contrast-Enhanced CT Images

“…In general, two key issues are at the forefront of the development of the CBIR system: (1) representing the image contents and (2) defining the similarity between images [9]. The intensity and texture features in liver CT images are considered as the important cues for the computerized detection and classification of liver lesions.…”

Towards a Repository for Standardized Medical Image and Signal Case Data Annotated with Ground Truth

“…In a recent review on the current status and future directions of image processing in radiology that was published in the Journal of Digital Imaging, the authors conclude by providing strategies for moving forward in the next decade, which are the creation of standardized tools for data collection and the facilitation of data sharing [1]. Indeed, in signal-or image-based computer-aided diagnosis (CAD), there is also a strong need for medical reference data repositories [2][3][4].…”