Increasing numbers of MRI brain scans, improvements in image resolution, and advancements in MRI acquisition technology are causing significant increases in the demand for and burden on radiologists' efforts in terms of reading and interpreting brain MRIs. Content-based image retrieval (CBIR) is an emerging technology for reducing this burden by supporting the reading of medical images. High dimensionality is a major challenge in developing a CBIR system that is applicable for 3D brain MRIs. In this study, we propose a system called diseaseoriented data concentration with metric learning (DDCML). In DDCML, we introduce deep metric learning to a 3D convolutional autoencoder (CAE). Our proposed DDCML scheme achieves a high dimensional compression rate (4096:1) while preserving the disease-related anatomical features that are important for medical image classification. The low-dimensional representation obtained by DDCML improved the clustering performance by 29.1% compared to plain 3D-CAE in terms of discriminating Alzheimer's disease patients from healthy subjects, and successfully reproduced the relationships of the severity of disease categories that were not included in the training.

To build a robust and practical content-based image retrieval (CBIR) system applicable to clinical brain MRI databases, we propose a new framework, disease-oriented image embedding with pseudo-scanner standardization (DI-PSS). It consists of two core techniques: data harmonization to absorb differences caused by different scanning environments and an algorithm to generate low-dimensional embeddings suitable for disease classification. Until now, there have been very few studies aimed at CBIR of brain MRI. Even in the harmonization of scanners, which is an important prerequisite technique for CBIR, only a limited number of studies have been conducted on T1-weighted MRI, which has collected a vast amount of clinical data. Recently proposed methods need to correctly estimate the domain (i.e., dataset, scanner) of each data in advance to remove environment-dependent information from low-dimensional embedding, which is not an easy task. With DI-PSS, each brain image is pseudo-transformed into a brain image taken with a given reference scanner. Then, 3D convolutioinal autoencoders (3D-CAE) trained with deep metric learning generate low-dimensional embeddings that better reflect the characteristics of the disease. In this study, DI-PSS reduced the variability of distance in low-dimensional embedding between Alzheimer's disease (AD) and clinically normal (CN) patients, caused by differences in scanners and datasets, by 15.8-22.6% and 18.0-29.9%, respectively, compared to the baseline. This improved the ability of spectral clustering to classify AD and CN by 6.2% in average accuracy and 10.7% in macro-F1. Our method has the advantage of not requiring difficult domain prediction tasks in advance, and can effectively utilize the big data of T1-weighted MR images. Given the potential of the DI-PSS for harmonizing images scanned by MRI scanners that were not used to scan the training data, it is well suited for application to a large number of legacy MRIs captured in heterogeneous environments.

No abstract

To build a robust and practical content-based image retrieval (CBIR) system that is applicable to a clinical brain MRI database, we propose a new framework -Disease-oriented image embedding with pseudo-scanner standardization (DI-PSS) -that consists of two core techniques, data harmonization and a dimension reduction algorithm. Our DI-PSS uses skull stripping and CycleGAN-based image transformations that map to a standard brain followed by transformation into a brain image taken with a given reference scanner. Then, our 3D convolutioinal autoencoders (3D-CAE) with deep metric learning acquires a low-dimensional embedding that better reflects the characteristics of the disease. The effectiveness of our proposed framework was tested on the T1-weighted MRIs selected from the Alzheimer's Disease Neuroimaging Initiative and the Parkinson's Progression Markers Initiative. We confirmed that our PSS greatly reduced the variability of low-dimensional embeddings caused by different scanner and datasets. Compared with the baseline condition, our PSS reduced the variability in the distance from Alzheimer's disease (AD) to clinically normal (CN) and Parkinson disease (PD) cases by 15.8-22.6% and 18.0-29.9%, respectively. These properties allow DI-PSS to generate lower dimensional representations that are more amenable to disease classification. In AD and CN classification experiments based on spectral clustering, PSS improved the average accuracy and macro-F1 by 6.2% and 10.7%, respectively. Given the potential of the DI-PSS for harmonizing images scanned by MRI scanners that were not used to scan the training data, we expect that the DI-PSS is suitable for application to a large number of legacy MRIs scanned in heterogeneous environments.