Convolutional Neural Networks With Intermediate Loss for 3D Super-Resolution of CT and MRI Scans

Abstract

Computed Tomography (CT) scanners that are commonly-used in hospitals and medical centers nowadays produce low-resolution images, e.g. one voxel in the image corresponds to at most onecubic millimeter of tissue. In order to accurately segment tumors and make treatment plans, radiologists and oncologists need CT scans of higher resolution. The same problem appears in Magnetic Resonance Imaging (MRI). In this paper, we propose an approach for the single-image super-resolution of 3D CT or MRI scans. Our method is based on deep convolutional neural networks (CNNs) composed of 10 convolutional layers and an intermediate upscaling layer that is placed after the first 6 convolutional layers. Our first CNN, which increases the resolution on two axes (width and height), is followed by a second CNN, which increases the resolution on the third axis (depth). Different from other methods, we compute the loss with respect to the ground-truth high-resolution image right after the upscaling layer, in addition to computing the loss after the last convolutional layer. The intermediate loss forces our network to produce a better output, closer to the ground-truth. A widely-used approach to obtain sharp results is to add Gaussian blur using a fixed standard deviation. In order to avoid overfitting to a fixed standard deviation, we apply Gaussian smoothing with various standard deviations, unlike other approaches. We evaluate the proposed method in the context of 2D and 3D super-resolution of CT and MRI scans from two databases, comparing it to related works from the literature and baselines based on various interpolation schemes, using 2× and 4× scaling factors. The empirical study shows that our approach attains superior results to all other methods. Moreover, our subjective image quality assessment by human observers reveals that both doctors and regular annotators chose our method in favor of Lanczos interpolation in 97.55% cases for an upscaling factor of 2× and in 96.69% cases for an upscaling factor of 4×. In order to allow others to reproduce our state-of-the-art results, we provide our code as open source at https://github.com/lilygeorgescu/3d-super-res-cnn. INDEX TERMSConvolutional neural networks, single-image super-resolution, CT images, MRI images, medical image super-resolution.

Accurate and Efficient Intracranial Hemorrhage Detection and Subtype Classification in 3D CT Scans with Convolutional and Long Short-Term Memory Neural Networks

“…Going at deeper convolutional layers, we typically observe more complex features, e.g., texture patterns, object parts, which are obtained by convolving lower-level features. Convolutional neural networks achieved state-of-the-art results on a broad range of computer vision [ 3 , 11 , 50 , 52 , 53 , 54 , 55 ] and medical imaging [ 12 , 19 , 24 ] tasks. In many cases, their success is in large part due to the availability of pre-trained models on large-scale datasets, which are highly transferable to other tasks, requiring only some fine-tuning.…”

“…After the initial success of deep learning [ 10 ] in object recognition from images [ 3 , 11 ], deep neural networks have been adopted for a broad range of tasks in medical imaging, ranging from cell segmentation [ 12 ] and cancer detection [ 13 , 14 , 15 , 16 , 17 ] to intracranial hemorrhage detection [ 5 , 8 , 18 , 19 , 20 , 21 , 22 ] and CT/MRI super-resolution [ 23 , 24 , 25 , 26 ]. Since we address the task of intracranial hemorrhage detection, we consider related works that are focused on the same task as ours [ 5 , 6 , 7 , 8 , 18 , 19 , 20 , 21 , 22 , 27 , 28 , 29 , 30 ], as well as works that study intracranial hemorrhage segmentation [ 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 ].…”

The Current Situation and Future Prospects of Simulators in Dental Education

“…Now, deep learning has been widely used in medical image analysis and modeling [113]. Through this type of method, we can easily extract relevant regions from computed tomography or MRI data and reconstruct mesh data [114]. These technologies can be transferred to oral cavity modeling, which can greatly facilitate model construction in a virtual surgery system and quickly establish a personalized oral cavity model.…”

Non-Linear Anisotropic Diffusion for Memory-Efficient Computed Tomography Super-Resolution Reconstruction

“…The second strategy is a post-processing step applied after the reconstruction of the 3D volume. For this strategy, the SR techniques can either be applied on the 2D slices of the volume [59,53,24], or on the 3D volume itself [47,28,23,19]. Both of these approaches are pipelined frameworks, where errors can accumulate between the CT reconstruction stage and the super-resolution stage, so that the final volume may actually be inconsistent with the original projection data, which may not be acceptable in some situations.…”