Re-Identification and growth detection of pulmonary nodules without image registration using 3D siamese neural networks

Rafael-Palou, Xavier; Aubanell, Anton; Bonavita, Ilaria; Ceresa, Mario; Piella, Gemma; Ribas, Vicent; Ballester, Miguel Ángel González

doi:10.1016/j.media.2020.101823

Cited by 27 publications

(15 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this study we used a longitudinal lung CT dataset (Rafael-Palou et al, 2020) for the follow-up analysis of incidental pulmonary nodules. In total, the cohort contains 161 patients (10 more cases compared to the previous version) with two thoracic CT scans per patient.…”

Section: Vh-lungmentioning

confidence: 99%

“…Recently, deep learning and in particular deep convolutional neural networks (CNN) have shown a great ability to automatically extract high-level representations from image data (Najafabadi et al, 2015). This has enabled performance improvements over conventional approaches in various medical imaging problems, such as nodule detection (Setio et al, 2017), segmentation (Messay et al, 2015), re-identification (Rafael-Palou et al, 2020) and malignancy classification (Ciompi et al, 2017).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Uncertainty-aware Hierarchical Probabilistic Network for Early Prediction, Quantification and Segmentation of Pulmonary Tumour Growth

Rafael-Palou¹,

Aubanell²,

Ceresa³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Early detection and quantification of tumour growth would help clinicians to prescribe more accurate treatments and provide better surgical planning. However, the multifactorial and heterogeneous nature of lung tumour progression hampers identification of growth patterns. In this study, we present a novel method based on a deep hierarchical generative and probabilistic framework that, according to radiological guidelines, predicts tumour growth, quantifies its size and provides a semantic appearance of the future nodule. Unlike previous deterministic solutions, the generative characteristic of our approach also allows us to estimate the uncertainty in the predictions, especially important for complex and doubtful cases. Results of evaluating this method on an independent test set reported a tumour growth balanced accuracy of 74%, a tumour growth size MAE of 1.77 mm and a tumour segmentation Dice score of 78%. These surpassed the performances of equivalent deterministic and alternative generative solutions (i.e. probabilistic U-Net, Bayesian test dropout and Pix2Pix GAN) confirming the suitability of our approach.

show abstract

Section: Vh-lungmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

An Uncertainty-aware Hierarchical Probabilistic Network for Early Prediction, Quantification and Segmentation of Pulmonary Tumour Growth

Rafael-Palou¹,

Aubanell²,

Ceresa³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Liu et al [32] extended similar 2D Siamese networks in a coarseto-fine fashion. While, Rafael-Palou et al [40] performed 3D Siamese networks with CT series, only shallow network architectures were evaluated on tracking lung nodules. However, we follow SiamRPN++ [29] to use Siamese networks with 3D DenseNet backbones and apply it to conduct universal lesion tracking in whole body CT images.…”

Section: Related Workmentioning

confidence: 99%

“…Monitoring treatment response by identifying and measuring corresponding lesions is critical in radiological workflows [20,40,1,47]. Manually conducting these procedures is labor-intensive, as expert clinicians must review multi-Figure 1.…”

Section: Introductionmentioning

confidence: 99%

“…However, registration algorithms [22,23,36,37,5] are usually less sensitive to local image changes; thus, they can be inaccurate to track small-sized lesions or lesions with large shape changes. On the other hand, appearance-based trackers [40,14] handle size and appearance changes by projecting lesion images into an embedding space [62,59], where images of the same lesion have similar embeddings and images of different lesions are different from one another. However, these appearance-based trackers may mismatch lesions with visually similar but spurious backgrounds.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Lesion Tracker: Monitoring Lesions in 4D Longitudinal Imaging Studies

Cai¹,

Tang²,

Yan³

et al. 2020

Preprint

View full text Add to dashboard Cite

Monitoring treatment response in longitudinal studies plays an important role in clinical practice. Accurately identifying lesions across serial imaging follow-up is the core to the monitoring procedure. Typically this incorporates both image and anatomical considerations. However, matching lesions manually is labor-intensive and time-consuming. In this work, we present deep lesion tracker (DLT), a deep learning approach that uses both appearance-and anatomical-based signals. To incorporate anatomical constraints, we propose an anatomical signal encoder, which prevents lesions being matched with visually similar but spurious regions. In addition, we present a new formulation for Siamese networks that avoids the heavy computational loads of 3D cross-correlation. To present our network with greater varieties of images, we also propose a self-supervised learning (SSL) strategy to train trackers with unpaired images, overcoming barriers to data collection. To train and evaluate our tracker, we introduce and release the first lesion tracking benchmark, consisting of 3891 lesion pairs from the public DeepLesion database. The proposed method, DLT, locates lesion centers with a mean error distance of 7mm. This is 5% better than a leading registration algorithm while running 14 times faster on whole CT volumes. We demonstrate even greater improvements over detector or similarity-learning alternatives. DLT also generalizes well on an external clinical test set of 100 longitudinal studies, achieving 88% accuracy. Finally, we plug DLT into an automatic tumor monitoring workflow where it leads to an accuracy of 85% in assessing lesion treatment responses, which is only 0.46% lower than the accuracy of manual inputs.

show abstract

Siamese Encoder-based Spatial-Temporal Mixer for Growth Trend Prediction of Lung Nodules on CT Scans

Fang

Wang²,

Li³

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Re-Identification and growth detection of pulmonary nodules without image registration using 3D siamese neural networks

Cited by 27 publications

References 43 publications

An Uncertainty-aware Hierarchical Probabilistic Network for Early Prediction, Quantification and Segmentation of Pulmonary Tumour Growth

An Uncertainty-aware Hierarchical Probabilistic Network for Early Prediction, Quantification and Segmentation of Pulmonary Tumour Growth

Deep Lesion Tracker: Monitoring Lesions in 4D Longitudinal Imaging Studies

Siamese Encoder-based Spatial-Temporal Mixer for Growth Trend Prediction of Lung Nodules on CT Scans

Contact Info

Product

Resources

About