Content-Based Brain Tumor Retrieval for MR Images Using Transfer Learning

Abstract

This paper presents an automatic content-based image retrieval (CBIR) system for brain tumors on T1-weighted contrast-enhanced magnetic resonance images (CE-MRI). The key challenge in CBIR systems for MR images is the semantic gap between the low-level visual information captured by the MRI machine and the high-level information perceived by the human evaluator. The traditional feature extraction methods focus only on low-level or high-level features and use some handcrafted features to reduce this gap. It is necessary to design a feature extraction framework to reduce this gap without using handcrafted features by encoding/combining low-level and high-level features. Deep learning is very powerful for feature representation that can depict low-level and high-level information completely and embed the phase of feature extraction in self-learning. Therefore, we propose a deep convolutional neural network VGG19-based novel feature extraction framework and apply closed-form metric learning to measure the similarity between the query image and database images. Furthermore, we adopt transfer learning and propose a block-wise fine-tuning strategy to enhance the retrieval performance. The extensive experiments are performed on a publicly available CE-MRI dataset that consists of three types of brain tumors (i.e., glioma, meningioma, and pituitary tumor) collected from 233 patients with a total of 3064 images across the axial, coronal, and sagittal views. Our method is more generic, as we do not use any handcrafted features; it requires minimal preprocessing, tested as robust on fivefold cross-validation, can achieve a fivefold mean average precision of 96.13%, and outperforms the state-of-the-art CBIR systems on the CE-MRI dataset. INDEX TERMS Brain tumor retrieval, block-wise fine-tuning, closed-form metric learning, convolutional neural networks, feature extraction, transfer learning.

State-of-the-Art CNN Optimizer for Brain Tumor Segmentation in Magnetic Resonance Images

“…In [22], a 3 CNN layer-based deep learning method was proposed to classify different brain tumour types and grades. Deep-CNN-based transfer learning and fine-tuning was used to segment brain tumors [23]. A recent study clearly outlined the components of CNN (layers, ReLU, dropout, response, and pooling) and its working mechanism [24]; this study is a comparison of scale-invariant feature transform (SIFT) and sparse coding for ImageNet LSVRC (Large Scale Visual Recognition Challenge) held in 2010.…”

An Efficient Medical Image Retrieval and Classification using Deep Neural Network

“…In (35) presented the CBIR system for brain tumor images. The T1-weighted CE-MRI dataset images were taken for validating the image retrieval performance.…”

“…The average retrieval precision of the proposed method is compared to the existing techniques to know the effectiveness of the proposed method over medical image retrieval. The existing methods used for the comparison are LMVCoP (33) and CBIR-CNN (35) . In LMVCoP (33) , two different feature extraction approaches are used such as LDP and GLCM.…”

“…The deep CNN based method is used for feature extraction and transfer learning, block-wise fine-tuning strategies are utilized to perform the medical image retrieval. The image retrieval of LMVCoP (33) and CBIR-CNN (35) are analyzed over the OASIS and CE-MRI datasets respectively. Table 9.…”

“…Table 9. ARP comparison of proposed method with existing method OASIS dataset CE-MRI dataset LMVCoP (33) Proposed method CBIR-CNN (35) Proposed method 71.58 88.25 96.13 98.33…”