An Ensemble Approach for Patient Prognosis of Head and Neck Tumor Using Multimodal Data

Saeed, Numan; Majzoub, Roba Al; Sobirov, Ikboljon; Yaqub, Mohammad

doi:10.1007/978-3-030-98253-9_26

Cited by 17 publications

(11 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Outcome prediction: summary of participants' methods The following describes the approach of each team participating in Task 2 (and 3 for some), in the order of the Task 2 ranking. In [54], Saeed et al (team "BiomedIA") first experimented with the clinical variables and determined that better prediction was achieved using only variables with complete values, compared to using all variables with imputing missing values. They elected to first implement a fusion of PET and CT images by averaging them into a new single PET/CT image that would be further cropped (2 different sizes of 50x50x50 and 80x80x80 were tested) to form the main input to their solution based on a 3D CNN (Deep-CR) trained to extract features which were then fed into Multi-Task Logistic Regression (MTLR, a sequence of logistic regression models created at various timelines to evaluate the probability of the event happening) improved by integrating neural networks to achieve nonlinearity, along with the clinical variables.…”

Section: Results: Reporting Of Challenge Outcomementioning

confidence: 99%

“…Individual participants' papers reporting their methods and results were submitted to the challenge organizers. Reviews were organized by the organizers and the papers of the participants are published in the LNCS challenges proceedings [60,1,52,56,59,12,53,62,21,32,43,37,54,6,67,16,9,48,49,58,40,51,17,46,65,39,33,45,27]. When participating in multiple tasks, participants could submit one or multiple papers.…”

Section: Introduction: Research Contextmentioning

confidence: 99%

See 1 more Smart Citation

Overview of the HECKTOR Challenge at MICCAI 2020: Automatic Head and Neck Tumor Segmentation in PET/CT

Andrearczyk

Oreiller

Jreige

et al. 2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

This paper presents an overview of the second edition of the HEad and neCK TumOR (HECKTOR) challenge, organized as a satellite event of the 24th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) 2021. The challenge is composed of three tasks related to the automatic analysis of PET/CT images for patients with Head and Neck cancer (H&N), focusing on the oropharynx region. Task 1 is the automatic segmentation of H&N primary Gross Tumor Volume (GTVt) in FDG-PET/CT images. Task 2 is the automatic prediction of Progression Free Survival (PFS) from the same FDG-PET/CT. Finally, Task 3 is the same as Task 2 with ground truth GTVt annotations provided to the participants. The data were collected from six centers for a total of 325 images, split into 224 training and 101 testing cases. The interest in the challenge was highlighted by the important participation with 103 registered teams and 448 result submissions. The best methods obtained a Dice Similarity Coefficient (DSC) of 0.7591 in the first task, and a Concordance index (C-index) of 0.7196 and 0.6978 in Tasks 2 and 3, respectively. In all tasks, simplicity of the approach was found to be key to ensure generalization performance. The comparison of the PFS prediction performance in Tasks 2 and 3 suggests that providing the GTVt contour was not crucial to achieve best results, which indicates that fully automatic methods can be used. This potentially obviates the need for GTVt contouring, opening avenues for reproducible and large scale radiomics studies including thousands potential subjects. equal contribution equal contribution

show abstract

Section: Results: Reporting Of Challenge Outcomementioning

confidence: 99%

Section: Introduction: Research Contextmentioning

confidence: 99%

Overview of the HECKTOR Challenge at MICCAI 2020: Automatic Head and Neck Tumor Segmentation in PET/CT

Andrearczyk

Oreiller

Jreige

et al. 2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…On the other hand, Z-score normalization was used for PET images. Furthermore, the images are cropped down to 80 × 80 × 48mm 3 as in [21] for two main purposes; the first is to fairly compare our results to the state-of-the-art in [21], which also used images with these dimensions. The second is that this reduction of image dimensions, in turn, speeds up training and inference processes and allows to run multiple experiments.…”

Section: Data Preprocessingmentioning

confidence: 99%

“…In [21], authors tackled the prognosis task for oropharyngeal squamous cell carcinoma patients using CT and PET images, along with clinical data. Their proposed solution was ranked the first in the progression-free survival prediction task of MICCAI 2021 HEad and neCK TumOR (HECKTOR) segmentation and outcome prediction challenge [20].…”

Section: Introductionmentioning

confidence: 99%

TMSS: An End-to-End Transformer-Based Multimodal Network for Segmentation and Survival Prediction

Saeed

Sobirov

Majzoub

et al. 2022

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

When oncologists estimate cancer patient survival, they rely on multimodal data. Even though some multimodal deep learning methods have been proposed in the literature, the majority rely on having two or more independent networks that share knowledge at a later stage in the overall model. On the other hand, oncologists do not do this in their analysis but rather fuse the information in their brain from multiple sources such as medical images and patient history. This work proposes a deep learning method that mimics oncologists' analytical behavior when quantifying cancer and estimating patient survival. We propose TMSS, an end-to-end Transformer based Multimodal network for Segmentation and Survival predication that leverages the superiority of transformers that lies in their abilities to handle different modalities. The model was trained and validated for segmentation and prognosis tasks on the training dataset from the HEad & NeCK TumOR segmentation and the outcome prediction in PET/CT images challenge (HECKTOR). We show that the proposed prognostic model significantly outperforms state-of-the-art methods with a concordance index of 0.763 ± 0.14 while achieving a comparable dice score of 0.772 ± 0.030 to a standalone segmentation model. The code is publicly available at https://t.ly/V-_W.

show abstract

“…Bounding box information comes with the dataset for localization of the tumor region, which was used to crop the scans and the mask down to the size of 144 × 144 × 144mm 3 with consistency between the scans. Further cropping down to 80 × 80 × 48 was performed around the tumor region for faster and better performance as in [24]. This highlights the tumor region, allowing the models to learn more easily.…”

Section: Datasets and Preprocessingmentioning

confidence: 99%

Segmentation with Super Images: A New 2D Perspective on 3D Medical Image Analysis

Sobirov¹,

Saeed²,

Yaqub³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Deep learning is showing an increasing number of audience in medical imaging research. In the segmentation task of medical images, we oftentimes rely on volumetric data, and thus require the use of 3D architectures which are praised for their ability to capture more features from the depth dimension. Yet, these architectures are generally more ineffective in time and compute compared to their 2D counterpart on account of 3D convolutions, max pooling, up-convolutions, and other operations used in these networks. Moreover, there are limited to no 3D pretrained model weights, and pretraining is generally challenging. To alleviate these issues, we propose to cast volumetric data to 2D super images and use 2D networks for the segmentation task. The method processes the 3D image by stitching slices side-by-side to generate a super resolution image. While the depth information is lost, we expect that deep neural networks can still capture and learn these features. Our goal in this work is to introduce a new perspective when dealing with volumetric data, and test our hypothesis using vanilla networks. We hope that this approach, while achieving close enough results to 3D networks using only 2D counterparts, can attract more related research in the future, especially in medical image analysis since volumetric data is comparably limited.

show abstract

An Ensemble Approach for Patient Prognosis of Head and Neck Tumor Using Multimodal Data

Cited by 17 publications

References 11 publications

Overview of the HECKTOR Challenge at MICCAI 2020: Automatic Head and Neck Tumor Segmentation in PET/CT

Overview of the HECKTOR Challenge at MICCAI 2020: Automatic Head and Neck Tumor Segmentation in PET/CT

TMSS: An End-to-End Transformer-Based Multimodal Network for Segmentation and Survival Prediction

Segmentation with Super Images: A New 2D Perspective on 3D Medical Image Analysis

Contact Info

Product

Resources

About