Training CNNs for Image Registration from Few Samples with Model-based Data Augmentation

Uzunova, Hristina; Wilms, Matthias; Handels, Heinz; Ehrhardt, Jan

doi:10.1007/978-3-319-66182-7_26

Cited by 70 publications

(52 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the training process, fifteen volumes of randomly rotated tumors were generated for each iteration of the training. Such type of image augmentation is commonly used in the training of deep learning for natural images as well as medical images 31 , 32 . The target output was cytolytic activity score (CytAct), defined by the expression of granzyme A ( GZMA ) and perforin 1 ( PRF1 ) normalized by z-score 19 .…”

Section: Methodsmentioning

confidence: 99%

Tumor immune profiles noninvasively estimated by FDG PET with deep learning correlate with immunotherapy response in lung adenocarcinoma

Park

Choi

et al. 2020

Theranostics

View full text Add to dashboard Cite

Rationale: The clinical application of biomarkers reflecting tumor immune microenvironment is hurdled by the invasiveness of obtaining tissues despite its importance in immunotherapy. We developed a deep learning-based biomarker which noninvasively estimates a tumor immune profile with fluorodeoxyglucose positron emission tomography (FDG-PET) in lung adenocarcinoma (LUAD). Methods: A deep learning model to predict cytolytic activity score (CytAct) using semi-automatically segmented tumors on FDG-PET trained by a publicly available dataset paired with tissue RNA sequencing (n = 93). This model was validated in two independent cohorts of LUAD: SNUH (n = 43) and The Cancer Genome Atlas (TCGA) cohort (n = 16). The model was applied to the immune checkpoint blockade (ICB) cohort, which consists of patients with metastatic LUAD who underwent ICB treatment (n = 29). Results: The predicted CytAct showed a positive correlation with CytAct of RNA sequencing in validation cohorts (Spearman rho = 0.32, p = 0.04 in SNUH cohort; spearman rho = 0.47, p = 0.07 in TCGA cohort). In ICB cohort, the higher predicted CytAct of individual lesion was associated with more decrement in tumor size after ICB treatment (Spearman rho = -0.54, p < 0.001). Higher minimum predicted CytAct in each patient associated with significantly prolonged progression free survival and overall survival (Hazard ratio 0.25, p = 0.001 and 0.18, p = 0.004, respectively). In patients with multiple lesions, ICB responders had significantly lower variance of predicted CytActs ( p = 0.005). Conclusion: The deep learning model that predicts CytAct using FDG-PET of LUAD was validated in independent cohorts. Our approach may be used to noninvasively assess an immune profile and predict outcomes of LUAD patients treated with ICB.

show abstract

Section: Methodsmentioning

confidence: 99%

Tumor immune profiles noninvasively estimated by FDG PET with deep learning correlate with immunotherapy response in lung adenocarcinoma

Park

Choi

et al. 2020

Theranostics

View full text Add to dashboard Cite

show abstract

“…For instance, De Craene et al [7] optimized a 4D velocity field parameterized by B-Spline spatio-temporal kernels to introduce temporal consistency, and Shi et al [14] combined different MR sequences to estimate myocardial motion using a series of free-form deformations (FFD) [12]. In recent years, some deep learning works [15,18] have also been proposed for medical image registration. They either trained networks to learn similarity metrics or simulated transformations as ground truth to learn the regression.…”

Section: Related Workmentioning

confidence: 99%

Joint Learning of Motion Estimation and Segmentation for Cardiac MR Image Sequences

Chen

Bai

Schlemper

et al. 2018

Medical Image Computing and Computer Assisted Intervention – MICCAI 2018

118

125

View full text Add to dashboard Cite

Cardiac motion estimation and segmentation play important roles in quantitatively assessing cardiac function and diagnosing cardiovascular diseases. In this paper, we propose a novel deep learning method for joint estimation of motion and segmentation from cardiac MR image sequences. The proposed network consists of two branches: a cardiac motion estimation branch which is built on a novel unsupervised Siamese style recurrent spatial transformer network, and a cardiac segmentation branch that is based on a fully convolutional network. In particular, a joint multi-scale feature encoder is learned by optimizing the segmentation branch and the motion estimation branch simultaneously. This enables the weakly-supervised segmentation by taking advantage of features that are unsupervisedly learned in the motion estimation branch from a large amount of unannotated data. Experimental results using cardiac MRI images from 220 subjects show that the joint learning of both tasks is complementary and the proposed models outperform the competing methods significantly in terms of accuracy and speed.

show abstract

“…Recent works on neural network based image registration fall into two major categories, the first treats network based registration as a regression problem on a given ground truth deformation field such as in [23]- [27]. These methods, unlike ours, can be used as fast approximations to other registration models.…”

Section: Related Workmentioning

confidence: 99%

TeTrIS: Template Transformer Networks for Image Segmentation With Shape Priors

Lee

Petersen

Pawlowski

et al. 2019

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

In this paper we introduce and compare different approaches for incorporating shape prior information into neural network based image segmentation. Specifically, we introduce the concept of template transformer networks where a shape template is deformed to match the underlying structure of interest through an end-to-end trained spatial transformer network. This has the advantage of explicitly enforcing shape priors and is free of discretisation artefacts by providing a soft partial volume segmentation. We also introduce a simple yet effective way of incorporating priors in state-of-the-art pixel-wise binary classification methods such as fully convolutional networks and U-net. Here, the template shape is given as an additional input channel, incorporating this information significantly reduces false positives. We report results on synthetic data and sub-voxel segmentation of coronary lumen structures in cardiac computed tomography showing the benefit of incorporating priors in neural network based image segmentation.

show abstract

Training CNNs for Image Registration from Few Samples with Model-based Data Augmentation

Cited by 70 publications

References 11 publications

Tumor immune profiles noninvasively estimated by FDG PET with deep learning correlate with immunotherapy response in lung adenocarcinoma

Tumor immune profiles noninvasively estimated by FDG PET with deep learning correlate with immunotherapy response in lung adenocarcinoma

Joint Learning of Motion Estimation and Segmentation for Cardiac MR Image Sequences

TeTrIS: Template Transformer Networks for Image Segmentation With Shape Priors

Contact Info

Product

Resources

About