Integrating spatial configuration into heatmap regression based CNNs for landmark localization

Payer, Christian; Štern, Darko; Bischof, Horst; Urschler, Martin

doi:10.1016/j.media.2019.03.007

Cited by 245 publications

(224 citation statements)

References 29 publications

Supporting

Mentioning

183

Contrasting

Order By: Relevance

“…Hence we argue that the distance transform regression learns geometric properties of shapes. In addition to that, we find that Gaussian heat-map regression is a common task in localizing anatomical landmarks [7]. Parallels can be drawn between Gaussian heat-map regression and Euclidean distance map regression for soft-localization of organ-specific landmarks.…”

Section: Methodsmentioning

confidence: 62%

Shape-Aware Complementary-Task Learning for Multi-organ Segmentation

Navarro

Shit

Ezhov

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Multi-organ segmentation in whole-body computed tomography (CT) is a constant pre-processing step which finds its application in organ-specific image retrieval, radiotherapy planning, and interventional image analysis. We address this problem from an organ-specific shapeprior learning perspective. We introduce the idea of complementary-task learning to enforce shape-prior leveraging the existing target labels. We propose two complementary-tasks namely i) distance map regression and ii) contour map detection to explicitly encode the geometric properties of each organ. We evaluate the proposed solution on the public VIS-CERAL dataset containing CT scans of multiple organs. We report a significant improvement of overall dice score from 0.8849 to 0.9018 due to the incorporation of complementary-task learning.

show abstract

Section: Methodsmentioning

confidence: 62%

Shape-Aware Complementary-Task Learning for Multi-organ Segmentation

Navarro

Shit

Ezhov

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…The landmark locations are then straightforwardly derived from the heat-maps as the points with maximal intensity in each channel. This kind of approach www.nature.com/scientificreports www.nature.com/scientificreports/ appears particularly appropriate for medical image processing 16,17 . Note that this concept of heat-map is related to the concept of heat-maps mentioned in the DAN and MCAM methods.…”

Section: Abbreviation Name Descriptionmentioning

confidence: 99%

“…This objective takes place in a larger framework in biomedical engineering and computer vision where localizing anatomical landmarks on biomedical images, sometimes referred to as detecting keypoints, has already been considered in other contexts. As an illustration, this has for instance been considered for the aortic valve on Computer Tomography scans 13 , for cephalometric landmarks on lateral cephalograms 14,15 , for finger joints on X-ray and MRI or for spine landmarks on volumetric Computer Tomography scans 16,17 . It has also been considered as a key element of more global registration processes, such as between fundus photographs and MRI for the eye 18 and between series of biological microscopic images 19 .…”

mentioning

confidence: 99%

Automatic vocal tract landmark localization from midsagittal MRI data

Eslami

Neuschaefer‐Rube

Serrurier

2020

Sci Rep

View full text Add to dashboard Cite

the various speech sounds of a language are obtained by varying the shape and position of the articulators surrounding the vocal tract. Analyzing their variations is crucial for understanding speech production, diagnosing speech disorders and planning therapy. identifying key anatomical landmarks of these structures on medical images is a pre-requisite for any quantitative analysis and the rising amount of data generated in the field calls for an automatic solution. The challenge lies in the high inter-and intra-speaker variability, the mutual interaction between the articulators and the moderate quality of the images. This study addresses this issue for the first time and tackles it by means of Deep Learning. it proposes a dedicated network architecture named Flat-net and its performance are evaluated and compared with eleven state-of-the-art methods from the literature. the dataset contains midsagittal anatomical Magnetic Resonance Images for 9 speakers sustaining 62 articulations with 21 annotated anatomical landmarks per image. Results show that the Flat-net approach outperforms the former methods, leading to an overall Root Mean Square Error of 3.6 pixels/0.36 cm obtained in a leave-oneout procedure over the speakers. the implementation codes are also shared publicly on GitHub.www.nature.com/scientificreports www.nature.com/scientificreports/ animals 22 . This short review emphasizes the importance of anatomical landmark localization from images for a large variety of applications and this study lengthens this non-exhaustive list with speech production.Within the existing literature, two fields of application appear more particularly active and connected to our problem. The first is the localization of landmarks for the face, rich of an abundant research from landmark identification on two-dimensional photographs 23,24 to landmark identification on three-dimensional Ultrasounds for fetuses for instance 25,26 . A recent and comprehensive review is provided by Wu et al. 27 . It represents a challenging issue due to the high variability of the shapes, poses, occlusions, and lighting conditions. Similarly, localizing the position of the joints of the body on images to estimate the human pose is also a long-standing problem 28 . It is also a challenging issue in computer vision due to the high variability of the postures, body shapes, actions, clothes and scenes.The goal and contribution of this study is to propose a fully-automated end-to-end image analysis methods for localizing key anatomical landmarks in the vocal tract area from midsagittal MRI data. As emphasized earlier, the need for such a method in the field is required and has never been attempted so far. It is aimed to be used in the future for new speakers for which no prior data are available. Considering the existing methods of the literature and the recent rise of data science to solve such problems, Deep Learning (DL) appears as the inescapable approach 29 . Indeed, DL approaches in image processing tasks appear to outperform most of traditional t...

show abstract

“…We utilize an hourglass network which is an encoder-decoder model initially introduced for human pose estimation [27] and address both ROI and landmark localization tasks. Several other studies in medical imaging domain also leveraged a similar approach by applying U-Net [33] to the landmark localization problem [12,31]. However, the encoder-decoder networks are computationally heavy during the training phase since they regress a tensor of high-resolution heatmaps which is challenging for medical images that are typically of a large size.…”

Section: Related Workmentioning

confidence: 99%

“…Anatomical landmark localization is a challenging problem that appears in many medical image analysis problems [31]. One particular realm where the localization of landmarks is of high importance is the analysis of knee plain radiographs at different stages of osteoarthritis (OA) -the most common joint disorder and 11 th highest disability factor in the world [2].…”

Section: Introductionmentioning

confidence: 99%

KNEEL: Knee Anatomical Landmark Localization Using Hourglass Networks

Tiulpin

Melekhov

Saarakkala

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

This paper addresses the challenge of localization of anatomical landmarks in knee X-ray images at different stages of osteoarthritis (OA). Landmark localization can be viewed as regression problem, where the landmark position is directly predicted by using the region of interest or even full-size images leading to large memory footprint, especially in case of high resolution medical images. In this work, we propose an efficient deep neural networks framework with an hourglass architecture utilizing a soft-argmax layer to directly predict normalized coordinates of the landmark points. We provide an extensive evaluation of different regularization techniques and various loss functions to understand their influence on the localization performance. Furthermore, we introduce the concept of transfer learning from low-budget annotations, and experimentally demonstrate that such approach is improving the accuracy of landmark localization. Compared to the prior methods, we validate our model on two datasets that are independent from the train data and assess the performance of the method for different stages of OA severity. The proposed approach demonstrates better generalization performance compared to the current state-of-the-art.

show abstract

Integrating spatial configuration into heatmap regression based CNNs for landmark localization

Cited by 245 publications

References 29 publications

Shape-Aware Complementary-Task Learning for Multi-organ Segmentation

Shape-Aware Complementary-Task Learning for Multi-organ Segmentation

Automatic vocal tract landmark localization from midsagittal MRI data

KNEEL: Knee Anatomical Landmark Localization Using Hourglass Networks

Contact Info

Product

Resources

About