Automated and accurate classification of Whole Slide Image (WSI) is of great significance for the early diagnosis and treatment of cancer, which can be realized by Multi-Instance Learning (MIL). However, the current MIL method easily suffers from over-fitting due to the weak supervision of slide-level labels. In addition, it is difficult to distinguish discriminative instances in a WSI bag in the absence of pixel-level annotations. To address these problems, we propose a novel Clustering-Based Multi-Instance Learning method (CBMIL) for WSI classification. The CBMIL constructs feature set from phenotypic clusters to augment data for training the aggregation network. Meanwhile, a contrastive learning task is incorporated into the CBMIL for multi-task learning, which helps to regularize the feature aggregation process. In addition, the centroid of each phenotypic cluster is updated by the model, and the weights of the WSI patches are calculated by their similarity to the phenotypic centroids to highlight the significant patches. Our method is evaluated on two public WSI datasets (CAMELYON16 and TCGA-Lung) for binary tumor and cancer sub-types classification and achieves better performance and great interpretability compared with the state-of-the-art methods. The code is available at: https://github.com/wwu98934/CBMIL.
Multi-modality 3D medical images play an important role in the clinical practice. Due to the effectiveness of exploring the complementary information among different modalities, multi-modality learning has attracted increased attention recently, which can be realized by Deep Learning (DL) models. However, it remains a challenging task for two reasons. First, the prediction confidence of multi-modality learning network cannot be guaranteed when the model is trained with weaklysupervised volume-level labels. Second, it is difficult to effectively exploit the complementary information across modalities and also preserve the modality-specific properties when fusion. In this paper, we present a novel Reinforcement Learning (RL) driven approach to comprehensively address these challenges, where two Recurrent Neural Networks (RNN) based agents are utilized to choose reliable and informative features within modality (intra-learning) and explore complementary representations across modalities (inter-learning) with the guidance of dynamic weights. These agents are trained via Proximal Policy Optimization (PPO) with the confidence increment of the prediction as the reward. We take the 3D image classification as an example and conduct experiments on a multi-modality brain tumor MRI data. Our approach outperforms other methods when employing the proposed RL-based multimodality representation learning.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.