Temporal Context Matters: Enhancing Single Image Prediction with Disease Progression Representations

Konwer, Aishik; Xu, Xuan; Bae, Joseph; Chen, Chao; Prasanna, Prateek

doi:10.1109/cvpr52688.2022.01826

Cited by 10 publications

(6 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The set of images can be regarded as time-series images data that contain abundant temporal relevant diagnostic information. Integrating temporal information into medical imaging learning has significance for enhancing the diagnosis, prognosis, and disease progression analysis [175][176][177] . Some previous works used the CNN and recurrent neural network to mine the temporal and spatial information simultaneously [176,178] .…”

Section: Longitudinal Images Datamentioning

confidence: 99%

“…Integrating temporal information into medical imaging learning has significance for enhancing the diagnosis, prognosis, and disease progression analysis [175][176][177] . Some previous works used the CNN and recurrent neural network to mine the temporal and spatial information simultaneously [176,178] . However, with our investigation, only a few works are involved in employing a pre-training approach in this field.…”

Section: Longitudinal Images Datamentioning

confidence: 99%

See 1 more Smart Citation

Pre-training in Medical Data: A Survey

et al. 2023

View full text Add to dashboard Cite

Medical data refers to health-related information associated with regular patient care or as part of a clinical trial program. There are many categories of such data, such as clinical imaging data, bio-signal data, electronic health records (EHR), and multi-modality medical data. With the development of deep neural networks in the last decade, the emerging pre-training paradigm has become dominant in that it has significantly improved machine learning methods’ performance in a data-limited scenario. In recent years, studies of pre-training in the medical domain have achieved significant progress. To summarize these technology advancements, this work provides a comprehensive survey of recent advances for pre-training on several major types of medical data. In this survey, we summarize a large number of related publications and the existing benchmarking in the medical domain. Especially, the survey briefly describes how some pre-training methods are applied to or developed for medical data. From a data-driven perspective, we examine the extensive use of pre-training in many medical scenarios. Moreover, based on the summary of recent pre-training studies, we identify several challenges in this field to provide insights for future studies.

show abstract

Section: Longitudinal Images Datamentioning

confidence: 99%

Section: Longitudinal Images Datamentioning

confidence: 99%

Pre-training in Medical Data: A Survey

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Domain adaptation refers to the training of a neural network to jointly generate both discriminative and domain-invariant features in order to model different source and target data distributions. Many approaches resort to distribution matching through source reweighting [17], geometric transformation [4] or application of MMD and CORAL losses [19,27,44]. Rather than adapting distributions, we are more interested in adapting the partial modality (source domain) feature representations to the full modality (target domain) representations.…”

Section: Related Workmentioning

confidence: 99%

Enhancing Modality-Agnostic Representations via Meta-Learning for Brain Tumor Segmentation

Konwer¹,

Hu²,

Xu³

et al. 2023

Preprint

View full text Add to dashboard Cite

In the medical vision domain, different imaging modalities provide complementary information. However, in practice, not all modalities may be available during inference. Previous approaches, e.g., knowledge distillation or image synthesis, often assume the availability of full modalities for all patients during training; this is unrealistic and impractical owing to the variability in data collection across sites. We propose a novel approach to learn enhanced modality-agnostic representations by employing a novel meta-learning strategy in training, even when only a fraction of full modality patients are available. Meta-learning enhances partial modality representations to full modality representations by meta-training on partial modality data and meta-testing on limited full modality samples. Additionally, we co-supervise this feature enrichment by introducing an auxiliary adversarial learning branch. More specifically, a missing modality detector is used as a discriminator to mimic the full modality setting. Our segmentation framework significantly outperforms state-of-the-art brain tumor segmentation techniques in missing modality scenarios, as demonstrated on two brain tumor MRI datasets.

show abstract

“…Nevertheless, the majority of studies primarily concentrate on single time-point CT images (pre- or post-treatment), neglecting the temporal dynamics and alterations which can be elucidated through longitudinal contrast-enhanced CT images [7]. Several methods for predicting treatment response based on longitudinal medical images have been proposed, demonstrating that disease progression patterns represented by longitudinal data can improve the performance [8, 9]. These works can be primarily categorized into deep feature contrast (DFC) based methods and deep feature fusion (DFF) based ones.…”

Section: Introductionmentioning

confidence: 99%

LOMIA-T: A Transformer-based LOngitudinal Medical Image Analysis framework for predicting treatment response of esophageal cancer

Sun,

Li,

Chen

et al. 2024

Preprint

View full text Add to dashboard Cite

Deep learning models based on medical images have made significant strides in predicting treatment outcomes. However, previous methods have primarily concentrated on single time-point images, neglecting the temporal dynamics and changes inherent in longitudinal medical images. Thus, we propose a Transformer-based longitudinal image analysis framework (LOMIA-T) to contrast and fuse latent representations from pre- and post-treatment medical images for predicting treatment response. Specifically, we first design a treatment response-based contrastive loss to enhance latent representation by discerning evolutionary processes across various disease stages. Then, we integrate latent representations from pre- and post-treatment CT images using a cross-attention mechanism. Considering the redundancy in the dual-branch output features induced by the cross-attention mechanism, we propose a clinically interpretable feature fusion strategy to predict treatment response. Experimentally, the proposed framework outperforms several state-of-the-art longitudinal image analysis methods on an in-house Esophageal Squamous Cell Carcinoma (ESCC) dataset, encompassing 170 pre- and post-treatment contrast-enhanced CT image pairs from ESCC patients underwent neoadjuvant chemoradiotherapy. Ablation experiments validate the efficacy of the proposed treatment response-based contrastive loss and feature fusion strategy. The codes will be made available at https://github.com/syc19074115/LOMIA-T.

show abstract

Temporal Context Matters: Enhancing Single Image Prediction with Disease Progression Representations

Cited by 10 publications

References 54 publications

Pre-training in Medical Data: A Survey

Pre-training in Medical Data: A Survey

Enhancing Modality-Agnostic Representations via Meta-Learning for Brain Tumor Segmentation

LOMIA-T: A Transformer-based LOngitudinal Medical Image Analysis framework for predicting treatment response of esophageal cancer

Contact Info

Product

Resources

About