Regressing Heatmaps for Multiple Landmark Localization Using CNNs

A survey on deep learning in medical image analysis

“…Several researchers have shown that designing architectures incorporating unique task-specific properties can obtain better results than straightforward CNNs. Two examples which we encountered several times are multi-view Gao et al (2016d) Frame labeling US CNN 4 class frame classification using transfer learning with pre-trained networks Kumar et al (2016) Frame labeling US CNN 12 standard anatomical planes, CNN extracts features for support vector machine Rajchl et al (2016b) Segmentation with non expert labels MRI CNN Crowd-sourcing annotation efforts to segment brain structures Rajchl et al (2016a) Segmentation given bounding box MRI CNN CNN and CRF for segmentation of structures Ravishankar et al (2016a) Quantification US CNN Hybrid system using CNN and texture features to find abdominal circumference Yu et al (2016b) Left ventricle segmentation US CNN Frame-by-frame segmentation by dynamically fine-tuning CNN to the latest frame Wound segmentation photographs CNN Additional detection of infection risk and healing progress Ypsilantis et al (2015) Chemotherapy response prediction PET CNN CNN outperforms classical radiomics features in patients with esophageal cancer Zheng et al (2015) Carotid artery bifurcation detection CT CNN Two stage detection process, CNNs combined with Haar features Alansary et al (2016) Placenta segmentation MRI CNN 3D multi-stream CNN with extension for motion correction Fritscher et al (2016) Head&Neck tumor segmentation CT CNN 3 orthogonal patches in 2D CNNs, combined with other features Jaumard- Hakoun et al (2016) Tongue contour extraction US RBM Analysis of tongue motion during speech, combines auto-encoders with RBMs Payer et al (2016) Hand landmark detection X-ray CNN Various architectures are compared Quinn et al (2016) Disease detection microscopy CNN Smartphone mounted on microscope detects malaria, tuberculosis & parasite eggs Smistad and Løvstakken (2016) Vessel detection and segmentation US CNN Femoral and carotid vessels analyzed with standard fCNN Twinanda et al (2017) Task recognition in laparoscopy Videos CNN Fine-tuned AlexNet applied to video frames Xu et al (2016c) Cervical dysplasia cervigrams CNN Fine-tuned pre-trained network with added non-imaging features Xue et al (2016) Esophageal microvessel classification Microscopy CNN Simple CNN used for feature extraction Zhang et al (2016a) Image reconstruction CT CNN Reconstructing from limited angle measurements, reducing reconstruction artefacts Lekadir et al (2017) Carotid plaque classification US CNN Simple CNN for characterization of carotid plaque composition in ultrasound …”

Detecting Anatomical Landmarks From Limited Medical Imaging Data Using Two-Stage Task-Oriented Deep Neural Networks

“…Although multi-atlas registration can improve the robustness and accuracy, it is very time intensive to perform multiple registrations for each testing subject. Different from the former two sets of methods, learning based methods utilize learning algorithms in machine learning domain for landmark detection and have demonstrated superiority in anatomical landmark detection for medical images [6], [19]. …”

“…Besides, FCN based methods achieve impressive performance in object detection and landmark detection [31], [32], [33], [34]. Recently, Payer et al [6] have tested several FCN architectures for detecting anatomical landmarks with limited medical imaging data, where each landmark position is marked as a heat map corresponding to the original image. Reasonable detection performance is obtained, which helps illustrate the effectiveness of FCN in detecting anatomical landmarks.…”

“…, with millions of images [1], [2], [3]. Although several deep learning based landmark detection methods have been proposed in medical image analysis [4], [5], [6], [7], it is still challenging to detect anatomical landmarks for medical images, due to limited training data at hand. Also, the total number of weights to be learned in deep neural networks for 3D medical images is much larger than that for 2D natural images.…”

“…On the other hand, recent studies [10], [6] have adopted an end-to-end learning strategy for landmark detection via convolutional neural networks (CNN) or fully convolutional networks (FCN), with an entire image as input and the landmark coordinates (or heat maps of landmarks) as output. Due to limited ( e.g.…”