Spatio-Temporal Learning from Longitudinal Data for Multiple Sclerosis Lesion Segmentation

Denner, Stefan; Khakzar, Ashkan; Sajid, Moiz; Saleh, Mahdi; Špiclin, Žiga; Kim, Seong-Tae; Navab, Nassir

doi:10.1007/978-3-030-72084-1_11

Cited by 11 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, results showed the usefulness of longitudinal segmentation as the U-Net

network with the baseline PET and baseline lesion segmentation as inputs had significantly better results than the U-Net

network (0.58 vs. 0.50). These results are in accordance with Denner et al [ 19 ], who observed performance improvement with longitudinal networks using two time-points. In addition, on PET images, lesion contrast can be very different between two acquisitions, due to the patient’s treatment response, as shown in Figure 2 .…”

Section: Discussionsupporting

confidence: 93%

“…However, most of these deep-learning techniques focus on a single acquisition, while lesion segmentation on multiple time points is required to assess treatment response. Recently, methods were developed for the monitoring of multiple sclerosis lesions [ 19 , 20 , 21 ] and the assessment of rectal cancer response [ 22 ] on longitudinal MRI images. Denner et al [ 19 ] proposed a U-Net with input channels for each acquisition and an auxiliary self-supervised registration task to guide lesion segmentation.…”

Section: Introductionmentioning

confidence: 99%

“…Recently, methods were developed for the monitoring of multiple sclerosis lesions [ 19 , 20 , 21 ] and the assessment of rectal cancer response [ 22 ] on longitudinal MRI images. Denner et al [ 19 ] proposed a U-Net with input channels for each acquisition and an auxiliary self-supervised registration task to guide lesion segmentation. Jin et al [ 22 ] used two networks (one for each acquisition) linked by a sub-network for response prediction.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Automatic Segmentation of Metastatic Breast Cancer Lesions on 18F-FDG PET/CT Longitudinal Acquisitions for Treatment Response Assessment

Moreau

Rousseau

Fourcade

et al. 2021

Cancers

View full text Add to dashboard Cite

Metastatic breast cancer patients receive lifelong medication and are regularly monitored for disease progression. The aim of this work was to (1) propose networks to segment breast cancer metastatic lesions on longitudinal whole-body PET/CT and (2) extract imaging biomarkers from the segmentations and evaluate their potential to determine treatment response. Baseline and follow-up PET/CT images of 60 patients from the EPICUREseinmeta study were used to train two deep-learning models to segment breast cancer metastatic lesions: One for baseline images and one for follow-up images. From the automatic segmentations, four imaging biomarkers were computed and evaluated: SULpeak, Total Lesion Glycolysis (TLG), PET Bone Index (PBI) and PET Liver Index (PLI). The first network obtained a mean Dice score of 0.66 on baseline acquisitions. The second network obtained a mean Dice score of 0.58 on follow-up acquisitions. SULpeak, with a 32% decrease between baseline and follow-up, was the biomarker best able to assess patients’ response (sensitivity 87%, specificity 87%), followed by TLG (43% decrease, sensitivity 73%, specificity 81%) and PBI (8% decrease, sensitivity 69%, specificity 69%). Our networks constitute promising tools for the automatic segmentation of lesions in patients with metastatic breast cancer allowing treatment response assessment with several biomarkers.

show abstract

“…However, results showed the usefulness of longitudinal segmentation as the U-Net

network with the baseline PET and baseline lesion segmentation as inputs had significantly better results than the U-Net

Section: Discussionsupporting

confidence: 93%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Segmentation of Metastatic Breast Cancer Lesions on 18F-FDG PET/CT Longitudinal Acquisitions for Treatment Response Assessment

Moreau

Rousseau

Fourcade

et al. 2021

Cancers

View full text Add to dashboard Cite

show abstract

“…Inspired by the work of Balakrishnan et al ( 2019 ) to compute the deformation field (DF), Salem et al ( 2020 ) developed a new approach to simultaneously learn the nonlinear DF between follow-up and baseline and from the learned DF and input images learn the segmentation mask. Denner et al ( 2021 ) used the same shared encoder and different decoders to learn the tasks of segmentation and non-rigid registration. To improve the lesion map segmentation, Gessert et al ( 2020 ) extended the 4D context by adding a temporal history and adding convGRU to aggregate the 3D representations from encoders to be passed to the decoder for the final prediction map.…”

Section: Introductionmentioning

confidence: 99%

Improving the detection of new lesions in multiple sclerosis with a cascaded 3D fully convolutional neural network approach

et al. 2022

View full text Add to dashboard Cite

Longitudinal magnetic resonance imaging (MRI) has an important role in multiple sclerosis (MS) diagnosis and follow-up. Specifically, the presence of new lesions on brain MRI scans is considered a robust predictive biomarker for the disease progression. New lesions are a high-impact prognostic factor to predict evolution to MS or risk of disability accumulation over time. However, the detection of this disease activity is performed visually by comparing the follow-up and baseline scans. Due to the presence of small lesions, misregistration, and high inter-/intra-observer variability, this detection of new lesions is prone to errors. In this direction, one of the last Medical Image Computing and Computer Assisted Intervention (MICCAI) challenges was dealing with this automatic new lesion quantification. The MSSEG-2: MS new lesions segmentation challenge offers an evaluation framework for this new lesion segmentation task with a large database (100 patients, each with two-time points) compiled from the OFSEP (Observatoire français de la sclérose en plaques) cohort, the French MS registry, including 3D T2-w fluid-attenuated inversion recovery (T2-FLAIR) images from different centers and scanners. Apart from a change in centers, MRI scanners, and acquisition protocols, there are more challenges that hinder the automated detection process of new lesions such as the need for large annotated datasets, which may be not easily available, or the fact that new lesions are small areas producing a class imbalance problem that could bias trained models toward the non-lesion class. In this article, we present a novel automated method for new lesion detection of MS patient images. Our approach is based on a cascade of two 3D patch-wise fully convolutional neural networks (FCNNs). The first FCNN is trained to be more sensitive revealing possible candidate new lesion voxels, while the second FCNN is trained to reduce the number of misclassified voxels coming from the first network. 3D T2-FLAIR images from the two-time points were pre-processed and linearly co-registered. Afterward, a fully CNN, where its inputs were only the baseline and follow-up images, was trained to detect new MS lesions. Our approach obtained a mean segmentation dice similarity coefficient of 0.42 with a detection F1-score of 0.5. Compared to the challenge participants, we obtained one of the highest precision scores (PPVL = 0.52), the best PPVL rate (0.53), and a lesion detection sensitivity (SensL of 0.53).

show abstract

Multiple Sclerosis Lesion Segmentation - A Survey of Supervised CNN-Based Methods

Zhang

Oguz

2021

Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries

View full text Add to dashboard Cite

Spatio-Temporal Learning from Longitudinal Data for Multiple Sclerosis Lesion Segmentation

Cited by 11 publications

References 24 publications

Automatic Segmentation of Metastatic Breast Cancer Lesions on 18F-FDG PET/CT Longitudinal Acquisitions for Treatment Response Assessment

Automatic Segmentation of Metastatic Breast Cancer Lesions on 18F-FDG PET/CT Longitudinal Acquisitions for Treatment Response Assessment

Improving the detection of new lesions in multiple sclerosis with a cascaded 3D fully convolutional neural network approach

Multiple Sclerosis Lesion Segmentation - A Survey of Supervised CNN-Based Methods

Contact Info

Product

Resources

About